Repository: alibaba/cluster-contrast-reid Branch: main Commit: 57b62e95eb3a Files: 44 Total size: 98.0 MB Directory structure: gitextract_7h4tsjq6/ ├── LICENSE ├── README.md ├── clustercontrast/ │ ├── __init__.py │ ├── datasets/ │ │ ├── __init__.py │ │ ├── dukemtmcreid.py │ │ ├── market1501.py │ │ ├── msmt17.py │ │ ├── personx.py │ │ └── veri.py │ ├── evaluation_metrics/ │ │ ├── __init__.py │ │ ├── classification.py │ │ └── ranking.py │ ├── evaluators.py │ ├── models/ │ │ ├── __init__.py │ │ ├── cm.py │ │ ├── dsbn.py │ │ ├── kmeans.py │ │ ├── pooling.py │ │ ├── resnet.py │ │ ├── resnet_ibn.py │ │ └── resnet_ibn_a.py │ ├── trainers.py │ └── utils/ │ ├── __init__.py │ ├── data/ │ │ ├── __init__.py │ │ ├── base_dataset.py │ │ ├── preprocessor.py │ │ ├── sampler.py │ │ └── transforms.py │ ├── faiss_rerank.py │ ├── faiss_utils.py │ ├── infomap_cluster.py │ ├── infomap_utils.py │ ├── logging.py │ ├── meters.py │ ├── osutils.py │ ├── rerank.py │ └── serialization.py ├── examples/ │ ├── cluster_contrast_train_usl.py │ ├── cluster_contrast_train_usl_infomap.py │ ├── logs/ │ │ └── log.txt │ ├── pretrained/ │ │ └── resnet50-19c8e357.pth │ └── test.py ├── run_code.sh └── setup.py ================================================ FILE CONTENTS ================================================ ================================================ FILE: LICENSE ================================================ MIT License Copyright (c) 2021 Alibaba Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ================================================ FILE: README.md ================================================ ![Python >=3.5](https://img.shields.io/badge/Python->=3.6-blue.svg) ![PyTorch >=1.6](https://img.shields.io/badge/PyTorch->=1.6-yellow.svg) # Cluster Contrast for Unsupervised Person Re-Identification The *official* repository for [Cluster Contrast for Unsupervised Person Re-Identification](https://arxiv.org/pdf/2103.11568v3.pdf). We achieve state-of-the-art performances on **unsupervised learning** tasks for object re-ID, including person re-ID and vehicle re-ID. **Our unified framework** ![framework](figs/frameworkv2.png) ## Updates ***11/19/2021*** 1. Memory dictionary update changed from bath hard update to momentum update. Because the bath hard update is sensitive to parameters, good results need to adjust many parameters, which is not robust enough. 2. Add the results of the InforMap clustering algorithm. Compared with the DBSCAN clustering algorithm, it can achieve better results. At the same time, we found through experiments that it is more robust on each data set. ## Notes In the process of doing experiments, we found that some settings have a greater impact on the results. Share them here to prevent everyone from stepping on the pit when applying our method. 1. The dataloader sampler uses RandomMultipleGallerySampler, see the code implementation for details. At the same time, we also provide RandomMultipleGallerySamplerNoCam sampler, which can be used in non-ReID fields. 2. Add batch normalization to the final output layer of the network, see the code for details. 3. we obtain a total number of P × Z images in the mini batch. P represents the number of categories, Z represents the number of instances of each category. mini batch = P x Z, P is set to 16, Z changes with the mini batch. ## Requirements ### Installation ```shell git clone https://github.com/alibaba/cluster-contrast-reid.git cd ClusterContrast python setup.py develop ``` ### Prepare Datasets ```shell cd examples && mkdir data ``` Download the person datasets Market-1501,MSMT17,PersonX,DukeMTMC-reID and the vehicle datasets VeRi-776 from [aliyun](https://virutalbuy-public.oss-cn-hangzhou.aliyuncs.com/share/data.zip). Then unzip them under the directory like ``` ClusterContrast/examples/data ├── market1501 │ └── Market-1501-v15.09.15 ├── msmt17 │ └── MSMT17_V1 ├── personx │ └── PersonX ├── dukemtmcreid │ └── DukeMTMC-reID └── veri └── VeRi ``` ### Prepare ImageNet Pre-trained Models for IBN-Net When training with the backbone of [IBN-ResNet](https://arxiv.org/abs/1807.09441), you need to download the ImageNet-pretrained model from this [link](https://drive.google.com/drive/folders/1thS2B8UOSBi_cJX6zRy6YYRwz_nVFI_S) and save it under the path of `examples/pretrained/`. ImageNet-pretrained models for **ResNet-50** will be automatically downloaded in the python script. ## Training We utilize 4 GTX-2080TI GPUs for training. For more parameter configuration, please check **`run_code.sh`**. **examples:** Market-1501: 1. Using DBSCAN: ```shell CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl.py -b 256 -a resnet50 -d market1501 --iters 200 --momentum 0.1 --eps 0.6 --num-instances 16 ``` 2. Using InfoMap: ```shell CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl_infomap.py -b 256 -a resnet50 -d market1501 --iters 200 --momentum 0.1 --eps 0.5 --k1 15 --k2 4 --num-instances 16 ``` MSMT17: 1. Using DBSCAN: ```shell CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl.py -b 256 -a resnet50 -d msmt17 --iters 400 --momentum 0.1 --eps 0.6 --num-instances 16 ``` 2. Using InfoMap: ```shell CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl_infomap.py -b 256 -a resnet50 -d msmt17 --iters 400 --momentum 0.1 --eps 0.5 --k1 15 --k2 4 --num-instances 16 ``` DukeMTMC-reID: 1. Using DBSCAN: ```shell CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl.py -b 256 -a resnet50 -d dukemtmcreid --iters 200 --momentum 0.1 --eps 0.6 --num-instances 16 ``` 2. Using InfoMap: ```shell CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl_infomap.py -b 256 -a resnet50 -d dukemtmcreid --iters 200 --momentum 0.1 --eps 0.5 --k1 15 --k2 4 --num-instances 16 ``` VeRi-776 1. Using DBSCAN: ```shell CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl.py -b 256 -a resnet50 -d veri --iters 400 --momentum 0.1 --eps 0.6 --num-instances 16 --height 224 --width 224 ``` 2. Using InfoMap: ```shell CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl_infomap.py -b 256 -a resnet50 -d veri --iters 400 --momentum 0.1 --eps 0.5 --k1 15 --k2 4 --num-instances 16 --height 224 --width 224 ``` ## Evaluation We utilize 1 GTX-2080TI GPU for testing. **Note that** + use `--width 128 --height 256` (default) for person datasets, and `--height 224 --width 224` for vehicle datasets; + use `-a resnet50` (default) for the backbone of ResNet-50, and `-a resnet_ibn50a` for the backbone of IBN-ResNet. To evaluate the model, run: ```shell CUDA_VISIBLE_DEVICES=0 \ python examples/test.py \ -d $DATASET --resume $PATH ``` **Some examples:** ```shell ### Market-1501 ### CUDA_VISIBLE_DEVICES=0 \ python examples/test.py \ -d market1501 --resume logs/spcl_usl/market_resnet50/model_best.pth.tar ``` ## Results ![framework](figs/resultsv2.png) You can download the above models in the paper from [aliyun](https://virutalbuy-public.oss-cn-hangzhou.aliyuncs.com/share/cluster-contrast.zip) ## Citation If you find this code useful for your research, please cite our paper ``` @article{dai2021cluster, title={Cluster Contrast for Unsupervised Person Re-Identification}, author={Dai, Zuozhuo and Wang, Guangyuan and Zhu, Siyu and Yuan, Weihao and Tan, Ping}, journal={arXiv preprint arXiv:2103.11568}, year={2021} } ``` # Acknowledgements Thanks to Yixiao Ge for opening source of his excellent works [SpCL](https://github.com/yxgeee/SpCL). ================================================ FILE: clustercontrast/__init__.py ================================================ from __future__ import absolute_import from . import datasets from . import evaluation_metrics from . import models from . import utils from . import evaluators from . import trainers __version__ = '0.1.0' ================================================ FILE: clustercontrast/datasets/__init__.py ================================================ from __future__ import absolute_import import warnings from .market1501 import Market1501 from .msmt17 import MSMT17 from .personx import PersonX from .veri import VeRi from .dukemtmcreid import DukeMTMCreID __factory = { 'market1501': Market1501, 'msmt17': MSMT17, 'personx': PersonX, 'veri': VeRi, 'dukemtmcreid': DukeMTMCreID } def names(): return sorted(__factory.keys()) def create(name, root, *args, **kwargs): """ Create a dataset instance. Parameters ---------- name : str The dataset name. root : str The path to the dataset directory. split_id : int, optional The index of data split. Default: 0 num_val : int or float, optional When int, it means the number of validation identities. When float, it means the proportion of validation to all the trainval. Default: 100 download : bool, optional If True, will download the dataset. Default: False """ if name not in __factory: raise KeyError("Unknown dataset:", name) return __factory[name](root, *args, **kwargs) def get_dataset(name, root, *args, **kwargs): warnings.warn("get_dataset is deprecated. Use create instead.") return create(name, root, *args, **kwargs) ================================================ FILE: clustercontrast/datasets/dukemtmcreid.py ================================================ import glob import os.path as osp import re from ..utils.data import BaseImageDataset def process_dir(dir_path, relabel=False): img_paths = glob.glob(osp.join(dir_path, "*.jpg")) pattern = re.compile(r"([-\d]+)_c(\d)") # get all identities pid_container = set() for img_path in img_paths: pid, _ = map(int, pattern.search(img_path).groups()) if pid == -1: continue pid_container.add(pid) pid2label = {pid: label for label, pid in enumerate(pid_container)} data = [] for img_path in img_paths: pid, camid = map(int, pattern.search(img_path).groups()) if (pid not in pid_container) or (pid == -1): continue assert 1 <= camid <= 8 camid -= 1 if relabel: pid = pid2label[pid] data.append((img_path, pid, camid)) return data class DukeMTMCreID(BaseImageDataset): """DukeMTMC-reID. Reference: - Ristani et al. Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking. ECCVW 2016. - Zheng et al. Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in vitro. ICCV 2017. URL: ``_ Dataset statistics: - identities: 1404 (train + query). - images:16522 (train) + 2228 (query) + 17661 (gallery). - cameras: 8. """ dataset_dir = "DukeMTMC-reID" def __init__(self, root, verbose=True): super(DukeMTMCreID, self).__init__() self.root = osp.abspath(osp.expanduser(root)) self.dataset_dir = osp.join(self.root, self.dataset_dir) self.train_dir = osp.join(self.dataset_dir, 'bounding_box_train') self.query_dir = osp.join(self.dataset_dir, 'query') self.gallery_dir = osp.join(self.dataset_dir, 'bounding_box_test') train = process_dir(dir_path=self.train_dir, relabel=True) query = process_dir(dir_path=self.query_dir, relabel=False) gallery = process_dir(dir_path=self.gallery_dir, relabel=False) self.train = train self.query = query self.gallery = gallery self.num_train_pids, self.num_train_imgs, self.num_train_cams = self.get_imagedata_info(self.train) self.num_query_pids, self.num_query_imgs, self.num_query_cams = self.get_imagedata_info(self.query) self.num_gallery_pids, self.num_gallery_imgs, self.num_gallery_cams = self.get_imagedata_info(self.gallery) def _check_before_run(self): """Check if all files are available before going deeper""" if not osp.exists(self.dataset_dir): raise RuntimeError("'{}' is not available".format(self.dataset_dir)) if not osp.exists(self.train_dir): raise RuntimeError("'{}' is not available".format(self.train_dir)) if not osp.exists(self.query_dir): raise RuntimeError("'{}' is not available".format(self.query_dir)) if not osp.exists(self.gallery_dir): raise RuntimeError("'{}' is not available".format(self.gallery_dir)) ================================================ FILE: clustercontrast/datasets/market1501.py ================================================ from __future__ import print_function, absolute_import import os.path as osp import glob import re from ..utils.data import BaseImageDataset class Market1501(BaseImageDataset): """ Market1501 Reference: Zheng et al. Scalable Person Re-identification: A Benchmark. ICCV 2015. URL: http://www.liangzheng.org/Project/project_reid.html Dataset statistics: # identities: 1501 (+1 for background) # images: 12936 (train) + 3368 (query) + 15913 (gallery) """ dataset_dir = 'Market-1501-v15.09.15' def __init__(self, root, verbose=True, **kwargs): super(Market1501, self).__init__() self.dataset_dir = osp.join(root, self.dataset_dir) self.train_dir = osp.join(self.dataset_dir, 'bounding_box_train') self.query_dir = osp.join(self.dataset_dir, 'query') self.gallery_dir = osp.join(self.dataset_dir, 'bounding_box_test') self._check_before_run() train = self._process_dir(self.train_dir, relabel=True) query = self._process_dir(self.query_dir, relabel=False) gallery = self._process_dir(self.gallery_dir, relabel=False) if verbose: print("=> Market1501 loaded") self.print_dataset_statistics(train, query, gallery) self.train = train self.query = query self.gallery = gallery self.num_train_pids, self.num_train_imgs, self.num_train_cams = self.get_imagedata_info(self.train) self.num_query_pids, self.num_query_imgs, self.num_query_cams = self.get_imagedata_info(self.query) self.num_gallery_pids, self.num_gallery_imgs, self.num_gallery_cams = self.get_imagedata_info(self.gallery) def _check_before_run(self): """Check if all files are available before going deeper""" if not osp.exists(self.dataset_dir): raise RuntimeError("'{}' is not available".format(self.dataset_dir)) if not osp.exists(self.train_dir): raise RuntimeError("'{}' is not available".format(self.train_dir)) if not osp.exists(self.query_dir): raise RuntimeError("'{}' is not available".format(self.query_dir)) if not osp.exists(self.gallery_dir): raise RuntimeError("'{}' is not available".format(self.gallery_dir)) def _process_dir(self, dir_path, relabel=False): img_paths = glob.glob(osp.join(dir_path, '*.jpg')) pattern = re.compile(r'([-\d]+)_c(\d)') pid_container = set() for img_path in img_paths: pid, _ = map(int, pattern.search(img_path).groups()) if pid == -1: continue # junk images are just ignored pid_container.add(pid) pid2label = {pid: label for label, pid in enumerate(pid_container)} dataset = [] for img_path in img_paths: pid, camid = map(int, pattern.search(img_path).groups()) if pid == -1: continue # junk images are just ignored assert 0 <= pid <= 1501 # pid == 0 means background assert 1 <= camid <= 6 camid -= 1 # index starts from 0 if relabel: pid = pid2label[pid] dataset.append((img_path, pid, camid)) return dataset ================================================ FILE: clustercontrast/datasets/msmt17.py ================================================ from __future__ import print_function, absolute_import import os.path as osp import glob import re from ..utils.data import BaseImageDataset def _process_dir(dir_path, relabel=False): img_paths = glob.glob(osp.join(dir_path, '*.jpg')) pattern = re.compile(r'([-\d]+)_c(\d+)') pid_container = set() for img_path in img_paths: pid, _ = map(int, pattern.search(img_path).groups()) if pid == -1: continue # junk images are just ignored pid_container.add(pid) pid2label = {pid: label for label, pid in enumerate(pid_container)} dataset = [] for img_path in img_paths: pid, camid = map(int, pattern.search(img_path).groups()) if pid == -1: continue # junk images are just ignored assert 1 <= camid <= 15 camid -= 1 # index starts from 0 if relabel: pid = pid2label[pid] dataset.append((img_path, pid, camid)) return dataset class MSMT17(BaseImageDataset): dataset_dir = 'MSMT17_V1' def __init__(self, root, verbose=True, **kwargs): super(MSMT17, self).__init__() self.dataset_dir = osp.join(root, self.dataset_dir) self.train_dir = osp.join(self.dataset_dir, 'bounding_box_train') self.query_dir = osp.join(self.dataset_dir, 'query') self.gallery_dir = osp.join(self.dataset_dir, 'bounding_box_test') self._check_before_run() train = _process_dir(self.train_dir, relabel=True) query = _process_dir(self.query_dir, relabel=False) gallery = _process_dir(self.gallery_dir, relabel=False) if verbose: print("=> MSMT17_V1 loaded") self.print_dataset_statistics(train, query, gallery) self.train = train self.query = query self.gallery = gallery self.num_train_pids, self.num_train_imgs, self.num_train_cams = self.get_imagedata_info(self.train) self.num_query_pids, self.num_query_imgs, self.num_query_cams = self.get_imagedata_info(self.query) self.num_gallery_pids, self.num_gallery_imgs, self.num_gallery_cams = self.get_imagedata_info(self.gallery) def _check_before_run(self): """Check if all files are available before going deeper""" if not osp.exists(self.dataset_dir): raise RuntimeError("'{}' is not available".format(self.dataset_dir)) if not osp.exists(self.train_dir): raise RuntimeError("'{}' is not available".format(self.train_dir)) if not osp.exists(self.query_dir): raise RuntimeError("'{}' is not available".format(self.query_dir)) if not osp.exists(self.gallery_dir): raise RuntimeError("'{}' is not available".format(self.gallery_dir)) ================================================ FILE: clustercontrast/datasets/personx.py ================================================ from __future__ import print_function, absolute_import import os.path as osp import glob import re from ..utils.data import BaseImageDataset class PersonX(BaseImageDataset): """ PersonX Reference: Sun et al. Dissecting Person Re-identification from the Viewpoint of Viewpoint. CVPR 2019. Dataset statistics: # identities: 1266 # images: 9840 (train) + 5136 (query) + 30816 (gallery) """ dataset_dir = 'PersonX' def __init__(self, root, verbose=True, **kwargs): super(PersonX, self).__init__() self.dataset_dir = osp.join(root, self.dataset_dir) self.train_dir = osp.join(self.dataset_dir, 'bounding_box_train') self.query_dir = osp.join(self.dataset_dir, 'query') self.gallery_dir = osp.join(self.dataset_dir, 'bounding_box_test') self._check_before_run() train = self._process_dir(self.train_dir, relabel=True) query = self._process_dir(self.query_dir, relabel=False) gallery = self._process_dir(self.gallery_dir, relabel=False) if verbose: print("=> PersonX loaded") self.print_dataset_statistics(train, query, gallery) self.train = train self.query = query self.gallery = gallery self.num_train_pids, self.num_train_imgs, self.num_train_cams = self.get_imagedata_info(self.train) self.num_query_pids, self.num_query_imgs, self.num_query_cams = self.get_imagedata_info(self.query) self.num_gallery_pids, self.num_gallery_imgs, self.num_gallery_cams = self.get_imagedata_info(self.gallery) def _check_before_run(self): """Check if all files are available before going deeper""" if not osp.exists(self.dataset_dir): raise RuntimeError("'{}' is not available".format(self.dataset_dir)) if not osp.exists(self.train_dir): raise RuntimeError("'{}' is not available".format(self.train_dir)) if not osp.exists(self.query_dir): raise RuntimeError("'{}' is not available".format(self.query_dir)) if not osp.exists(self.gallery_dir): raise RuntimeError("'{}' is not available".format(self.gallery_dir)) def _process_dir(self, dir_path, relabel=False): img_paths = glob.glob(osp.join(dir_path, '*.jpg')) pattern = re.compile(r'([-\d]+)_c([-\d]+)') cam2label = {3: 1, 4: 2, 8: 3, 10: 4, 11: 5, 12: 6} pid_container = set() for img_path in img_paths: pid, _ = map(int, pattern.search(img_path).groups()) pid_container.add(pid) pid2label = {pid: label for label, pid in enumerate(pid_container)} dataset = [] for img_path in img_paths: pid, camid = map(int, pattern.search(img_path).groups()) assert (camid in cam2label.keys()) camid = cam2label[camid] camid -= 1 # index starts from 0 if relabel: pid = pid2label[pid] dataset.append((img_path, pid, camid)) return dataset ================================================ FILE: clustercontrast/datasets/veri.py ================================================ from __future__ import absolute_import from __future__ import division from __future__ import print_function import glob import re import os.path as osp from ..utils.data import BaseImageDataset class VeRi(BaseImageDataset): """ VeRi Reference: Liu, X., Liu, W., Ma, H., Fu, H.: Large-scale vehicle re-identification in urban surveillance videos. In: IEEE % International Conference on Multimedia and Expo. (2016) accepted. Dataset statistics: # identities: 776 vehicles(576 for training and 200 for testing) # images: 37778 (train) + 11579 (query) """ dataset_dir = 'VeRi' def __init__(self, root, verbose=True, **kwargs): super(VeRi, self).__init__() self.dataset_dir = osp.join(root, self.dataset_dir) self.train_dir = osp.join(self.dataset_dir, 'image_train') self.query_dir = osp.join(self.dataset_dir, 'image_query') self.gallery_dir = osp.join(self.dataset_dir, 'image_test') self.check_before_run() train = self.process_dir(self.train_dir, relabel=True) query = self.process_dir(self.query_dir, relabel=False) gallery = self.process_dir(self.gallery_dir, relabel=False) if verbose: print('=> VeRi loaded') self.print_dataset_statistics(train, query, gallery) self.train = train self.query = query self.gallery = gallery self.num_train_pids, self.num_train_imgs, self.num_train_cams = self.get_imagedata_info(self.train) self.num_query_pids, self.num_query_imgs, self.num_query_cams = self.get_imagedata_info(self.query) self.num_gallery_pids, self.num_gallery_imgs, self.num_gallery_cams = self.get_imagedata_info(self.gallery) def check_before_run(self): """Check if all files are available before going deeper""" if not osp.exists(self.dataset_dir): raise RuntimeError('"{}" is not available'.format(self.dataset_dir)) if not osp.exists(self.train_dir): raise RuntimeError('"{}" is not available'.format(self.train_dir)) if not osp.exists(self.query_dir): raise RuntimeError('"{}" is not available'.format(self.query_dir)) if not osp.exists(self.gallery_dir): raise RuntimeError('"{}" is not available'.format(self.gallery_dir)) def process_dir(self, dir_path, relabel=False): img_paths = glob.glob(osp.join(dir_path, '*.jpg')) pattern = re.compile(r'([-\d]+)_c([-\d]+)') pid_container = set() for img_path in img_paths: pid, _ = map(int, pattern.search(img_path).groups()) if pid == -1: continue # junk images are just ignored pid_container.add(pid) pid2label = {pid: label for label, pid in enumerate(pid_container)} dataset = [] for img_path in img_paths: pid, camid = map(int, pattern.search(img_path).groups()) if pid == -1: continue # junk images are just ignored assert 0 <= pid <= 776 # pid == 0 means background assert 1 <= camid <= 20 camid -= 1 # index starts from 0 if relabel: pid = pid2label[pid] dataset.append((img_path, pid, camid)) return dataset ================================================ FILE: clustercontrast/evaluation_metrics/__init__.py ================================================ from __future__ import absolute_import from .classification import accuracy from .ranking import cmc, mean_ap __all__ = [ 'accuracy', 'cmc', 'mean_ap' ] ================================================ FILE: clustercontrast/evaluation_metrics/classification.py ================================================ from __future__ import absolute_import import torch from ..utils import to_torch def accuracy(output, target, topk=(1,)): with torch.no_grad(): output, target = to_torch(output), to_torch(target) maxk = max(topk) batch_size = target.size(0) _, pred = output.topk(maxk, 1, True, True) pred = pred.t() correct = pred.eq(target.view(1, -1).expand_as(pred)) ret = [] for k in topk: correct_k = correct[:k].view(-1).float().sum(dim=0, keepdim=True) ret.append(correct_k.mul_(1. / batch_size)) return ret ================================================ FILE: clustercontrast/evaluation_metrics/ranking.py ================================================ from __future__ import absolute_import from collections import defaultdict import numpy as np from sklearn.metrics import average_precision_score from ..utils import to_numpy def _unique_sample(ids_dict, num): mask = np.zeros(num, dtype=np.bool) for _, indices in ids_dict.items(): i = np.random.choice(indices) mask[i] = True return mask def cmc(distmat, query_ids=None, gallery_ids=None, query_cams=None, gallery_cams=None, topk=100, separate_camera_set=False, single_gallery_shot=False, first_match_break=False): distmat = to_numpy(distmat) m, n = distmat.shape # Fill up default values if query_ids is None: query_ids = np.arange(m) if gallery_ids is None: gallery_ids = np.arange(n) if query_cams is None: query_cams = np.zeros(m).astype(np.int32) if gallery_cams is None: gallery_cams = np.ones(n).astype(np.int32) # Ensure numpy array query_ids = np.asarray(query_ids) gallery_ids = np.asarray(gallery_ids) query_cams = np.asarray(query_cams) gallery_cams = np.asarray(gallery_cams) # Sort and find correct matches indices = np.argsort(distmat, axis=1) matches = (gallery_ids[indices] == query_ids[:, np.newaxis]) # Compute CMC for each query ret = np.zeros(topk) num_valid_queries = 0 for i in range(m): # Filter out the same id and same camera valid = ((gallery_ids[indices[i]] != query_ids[i]) | (gallery_cams[indices[i]] != query_cams[i])) if separate_camera_set: # Filter out samples from same camera valid &= (gallery_cams[indices[i]] != query_cams[i]) if not np.any(matches[i, valid]): continue if single_gallery_shot: repeat = 10 gids = gallery_ids[indices[i][valid]] inds = np.where(valid)[0] ids_dict = defaultdict(list) for j, x in zip(inds, gids): ids_dict[x].append(j) else: repeat = 1 for _ in range(repeat): if single_gallery_shot: # Randomly choose one instance for each id sampled = (valid & _unique_sample(ids_dict, len(valid))) index = np.nonzero(matches[i, sampled])[0] else: index = np.nonzero(matches[i, valid])[0] delta = 1. / (len(index) * repeat) for j, k in enumerate(index): if k - j >= topk: break if first_match_break: ret[k - j] += 1 break ret[k - j] += delta num_valid_queries += 1 if num_valid_queries == 0: raise RuntimeError("No valid query") return ret.cumsum() / num_valid_queries def mean_ap(distmat, query_ids=None, gallery_ids=None, query_cams=None, gallery_cams=None): distmat = to_numpy(distmat) m, n = distmat.shape # Fill up default values if query_ids is None: query_ids = np.arange(m) if gallery_ids is None: gallery_ids = np.arange(n) if query_cams is None: query_cams = np.zeros(m).astype(np.int32) if gallery_cams is None: gallery_cams = np.ones(n).astype(np.int32) # Ensure numpy array query_ids = np.asarray(query_ids) gallery_ids = np.asarray(gallery_ids) query_cams = np.asarray(query_cams) gallery_cams = np.asarray(gallery_cams) # Sort and find correct matches indices = np.argsort(distmat, axis=1) matches = (gallery_ids[indices] == query_ids[:, np.newaxis]) # Compute AP for each query aps = [] for i in range(m): # Filter out the same id and same camera valid = ((gallery_ids[indices[i]] != query_ids[i]) | (gallery_cams[indices[i]] != query_cams[i])) y_true = matches[i, valid] y_score = -distmat[i][indices[i]][valid] if not np.any(y_true): continue aps.append(average_precision_score(y_true, y_score)) if len(aps) == 0: raise RuntimeError("No valid query") return np.mean(aps) ================================================ FILE: clustercontrast/evaluators.py ================================================ from __future__ import print_function, absolute_import import time import collections from collections import OrderedDict import numpy as np import torch import random import copy from .evaluation_metrics import cmc, mean_ap from .utils.meters import AverageMeter from .utils.rerank import re_ranking from .utils import to_torch def extract_cnn_feature(model, inputs): inputs = to_torch(inputs).cuda() outputs = model(inputs) outputs = outputs.data.cpu() return outputs def extract_features(model, data_loader, print_freq=50): model.eval() batch_time = AverageMeter() data_time = AverageMeter() features = OrderedDict() labels = OrderedDict() end = time.time() with torch.no_grad(): for i, (imgs, fnames, pids, _, _) in enumerate(data_loader): data_time.update(time.time() - end) outputs = extract_cnn_feature(model, imgs) for fname, output, pid in zip(fnames, outputs, pids): features[fname] = output labels[fname] = pid batch_time.update(time.time() - end) end = time.time() if (i + 1) % print_freq == 0: print('Extract Features: [{}/{}]\t' 'Time {:.3f} ({:.3f})\t' 'Data {:.3f} ({:.3f})\t' .format(i + 1, len(data_loader), batch_time.val, batch_time.avg, data_time.val, data_time.avg)) return features, labels def pairwise_distance(features, query=None, gallery=None): if query is None and gallery is None: n = len(features) x = torch.cat(list(features.values())) x = x.view(n, -1) dist_m = torch.pow(x, 2).sum(dim=1, keepdim=True) * 2 dist_m = dist_m.expand(n, n) - 2 * torch.mm(x, x.t()) return dist_m x = torch.cat([features[f].unsqueeze(0) for f, _, _ in query], 0) y = torch.cat([features[f].unsqueeze(0) for f, _, _ in gallery], 0) m, n = x.size(0), y.size(0) x = x.view(m, -1) y = y.view(n, -1) dist_m = torch.pow(x, 2).sum(dim=1, keepdim=True).expand(m, n) + \ torch.pow(y, 2).sum(dim=1, keepdim=True).expand(n, m).t() dist_m.addmm_(1, -2, x, y.t()) return dist_m, x.numpy(), y.numpy() def evaluate_all(query_features, gallery_features, distmat, query=None, gallery=None, query_ids=None, gallery_ids=None, query_cams=None, gallery_cams=None, cmc_topk=(1, 5, 10), cmc_flag=False): if query is not None and gallery is not None: query_ids = [pid for _, pid, _ in query] gallery_ids = [pid for _, pid, _ in gallery] query_cams = [cam for _, _, cam in query] gallery_cams = [cam for _, _, cam in gallery] else: assert (query_ids is not None and gallery_ids is not None and query_cams is not None and gallery_cams is not None) # Compute mean AP mAP = mean_ap(distmat, query_ids, gallery_ids, query_cams, gallery_cams) print('Mean AP: {:4.1%}'.format(mAP)) if (not cmc_flag): return mAP cmc_configs = { 'market1501': dict(separate_camera_set=False, single_gallery_shot=False, first_match_break=True),} cmc_scores = {name: cmc(distmat, query_ids, gallery_ids, query_cams, gallery_cams, **params) for name, params in cmc_configs.items()} print('CMC Scores:') for k in cmc_topk: print(' top-{:<4}{:12.1%}'.format(k, cmc_scores['market1501'][k-1])) return cmc_scores['market1501'], mAP class Evaluator(object): def __init__(self, model): super(Evaluator, self).__init__() self.model = model def evaluate(self, data_loader, query, gallery, cmc_flag=False, rerank=False): features, _ = extract_features(self.model, data_loader) distmat, query_features, gallery_features = pairwise_distance(features, query, gallery) results = evaluate_all(query_features, gallery_features, distmat, query=query, gallery=gallery, cmc_flag=cmc_flag) if (not rerank): return results print('Applying person re-ranking ...') distmat_qq, _, _ = pairwise_distance(features, query, query) distmat_gg, _, _ = pairwise_distance(features, gallery, gallery) distmat = re_ranking(distmat.numpy(), distmat_qq.numpy(), distmat_gg.numpy()) return evaluate_all(query_features, gallery_features, distmat, query=query, gallery=gallery, cmc_flag=cmc_flag) ================================================ FILE: clustercontrast/models/__init__.py ================================================ from __future__ import absolute_import from .resnet import * from .resnet_ibn import * __factory = { 'resnet18': resnet18, 'resnet34': resnet34, 'resnet50': resnet50, 'resnet101': resnet101, 'resnet152': resnet152, 'resnet_ibn50a': resnet_ibn50a, 'resnet_ibn101a': resnet_ibn101a, } def names(): return sorted(__factory.keys()) def create(name, *args, **kwargs): """ Create a model instance. Parameters ---------- name : str Model name. Can be one of 'inception', 'resnet18', 'resnet34', 'resnet50', 'resnet101', and 'resnet152'. pretrained : bool, optional Only applied for 'resnet*' models. If True, will use ImageNet pretrained model. Default: True cut_at_pooling : bool, optional If True, will cut the model before the last global pooling layer and ignore the remaining kwargs. Default: False num_features : int, optional If positive, will append a Linear layer after the global pooling layer, with this number of output units, followed by a BatchNorm layer. Otherwise these layers will not be appended. Default: 256 for 'inception', 0 for 'resnet*' norm : bool, optional If True, will normalize the feature to be unit L2-norm for each sample. Otherwise will append a ReLU layer after the above Linear layer if num_features > 0. Default: False dropout : float, optional If positive, will append a Dropout layer with this dropout rate. Default: 0 num_classes : int, optional If positive, will append a Linear layer at the end as the classifier with this number of output units. Default: 0 """ if name not in __factory: raise KeyError("Unknown model:", name) return __factory[name](*args, **kwargs) ================================================ FILE: clustercontrast/models/cm.py ================================================ import collections import numpy as np from abc import ABC import torch import torch.nn.functional as F from torch import nn, autograd class CM(autograd.Function): @staticmethod def forward(ctx, inputs, targets, features, momentum): ctx.features = features ctx.momentum = momentum ctx.save_for_backward(inputs, targets) outputs = inputs.mm(ctx.features.t()) return outputs @staticmethod def backward(ctx, grad_outputs): inputs, targets = ctx.saved_tensors grad_inputs = None if ctx.needs_input_grad[0]: grad_inputs = grad_outputs.mm(ctx.features) # momentum update for x, y in zip(inputs, targets): ctx.features[y] = ctx.momentum * ctx.features[y] + (1. - ctx.momentum) * x ctx.features[y] /= ctx.features[y].norm() return grad_inputs, None, None, None def cm(inputs, indexes, features, momentum=0.5): return CM.apply(inputs, indexes, features, torch.Tensor([momentum]).to(inputs.device)) class CM_Hard(autograd.Function): @staticmethod def forward(ctx, inputs, targets, features, momentum): ctx.features = features ctx.momentum = momentum ctx.save_for_backward(inputs, targets) outputs = inputs.mm(ctx.features.t()) return outputs @staticmethod def backward(ctx, grad_outputs): inputs, targets = ctx.saved_tensors grad_inputs = None if ctx.needs_input_grad[0]: grad_inputs = grad_outputs.mm(ctx.features) batch_centers = collections.defaultdict(list) for instance_feature, index in zip(inputs, targets.tolist()): batch_centers[index].append(instance_feature) for index, features in batch_centers.items(): distances = [] for feature in features: distance = feature.unsqueeze(0).mm(ctx.features[index].unsqueeze(0).t())[0][0] distances.append(distance.cpu().numpy()) median = np.argmin(np.array(distances)) ctx.features[index] = ctx.features[index] * ctx.momentum + (1 - ctx.momentum) * features[median] ctx.features[index] /= ctx.features[index].norm() return grad_inputs, None, None, None def cm_hard(inputs, indexes, features, momentum=0.5): return CM_Hard.apply(inputs, indexes, features, torch.Tensor([momentum]).to(inputs.device)) class ClusterMemory(nn.Module, ABC): def __init__(self, num_features, num_samples, temp=0.05, momentum=0.2, use_hard=False): super(ClusterMemory, self).__init__() self.num_features = num_features self.num_samples = num_samples self.momentum = momentum self.temp = temp self.use_hard = use_hard self.register_buffer('features', torch.zeros(num_samples, num_features)) def forward(self, inputs, targets): inputs = F.normalize(inputs, dim=1).cuda() if self.use_hard: outputs = cm_hard(inputs, targets, self.features, self.momentum) else: outputs = cm(inputs, targets, self.features, self.momentum) outputs /= self.temp loss = F.cross_entropy(outputs, targets) return loss ================================================ FILE: clustercontrast/models/dsbn.py ================================================ import torch import torch.nn as nn # Domain-specific BatchNorm class DSBN2d(nn.Module): def __init__(self, planes): super(DSBN2d, self).__init__() self.num_features = planes self.BN_S = nn.BatchNorm2d(planes) self.BN_T = nn.BatchNorm2d(planes) def forward(self, x): if (not self.training): return self.BN_T(x) bs = x.size(0) assert (bs%2==0) split = torch.split(x, int(bs/2), 0) out1 = self.BN_S(split[0].contiguous()) out2 = self.BN_T(split[1].contiguous()) out = torch.cat((out1, out2), 0) return out class DSBN1d(nn.Module): def __init__(self, planes): super(DSBN1d, self).__init__() self.num_features = planes self.BN_S = nn.BatchNorm1d(planes) self.BN_T = nn.BatchNorm1d(planes) def forward(self, x): if (not self.training): return self.BN_T(x) bs = x.size(0) assert (bs%2==0) split = torch.split(x, int(bs/2), 0) out1 = self.BN_S(split[0].contiguous()) out2 = self.BN_T(split[1].contiguous()) out = torch.cat((out1, out2), 0) return out def convert_dsbn(model): for _, (child_name, child) in enumerate(model.named_children()): assert(not next(model.parameters()).is_cuda) if isinstance(child, nn.BatchNorm2d): m = DSBN2d(child.num_features) m.BN_S.load_state_dict(child.state_dict()) m.BN_T.load_state_dict(child.state_dict()) setattr(model, child_name, m) elif isinstance(child, nn.BatchNorm1d): m = DSBN1d(child.num_features) m.BN_S.load_state_dict(child.state_dict()) m.BN_T.load_state_dict(child.state_dict()) setattr(model, child_name, m) else: convert_dsbn(child) def convert_bn(model, use_target=True): for _, (child_name, child) in enumerate(model.named_children()): assert(not next(model.parameters()).is_cuda) if isinstance(child, DSBN2d): m = nn.BatchNorm2d(child.num_features) if use_target: m.load_state_dict(child.BN_T.state_dict()) else: m.load_state_dict(child.BN_S.state_dict()) setattr(model, child_name, m) elif isinstance(child, DSBN1d): m = nn.BatchNorm1d(child.num_features) if use_target: m.load_state_dict(child.BN_T.state_dict()) else: m.load_state_dict(child.BN_S.state_dict()) setattr(model, child_name, m) else: convert_bn(child, use_target=use_target) ================================================ FILE: clustercontrast/models/kmeans.py ================================================ # Written by Yixiao Ge import warnings import faiss import torch from ..utils import to_numpy, to_torch __all__ = ["label_generator_kmeans"] @torch.no_grad() def label_generator_kmeans(features, num_classes=500, cuda=True): assert num_classes, "num_classes for kmeans is null" # k-means cluster by faiss cluster = faiss.Kmeans( features.size(-1), num_classes, niter=300, verbose=True, gpu=cuda ) cluster.train(to_numpy(features)) _, labels = cluster.index.search(to_numpy(features), 1) labels = labels.reshape(-1) centers = to_torch(cluster.centroids).float() # labels = to_torch(labels).long() # k-means does not have outlier points assert not (-1 in labels) return labels, centers, num_classes, None ================================================ FILE: clustercontrast/models/pooling.py ================================================ # Credit to https://github.com/JDAI-CV/fast-reid/blob/master/fastreid/layers/pooling.py from abc import ABC import torch import torch.nn.functional as F from torch import nn __all__ = [ "GeneralizedMeanPoolingPFpn", "GeneralizedMeanPoolingList", "GeneralizedMeanPoolingP", "AdaptiveAvgMaxPool2d", "FastGlobalAvgPool2d", "avg_pooling", "max_pooling", ] class GeneralizedMeanPoolingList(nn.Module, ABC): r"""Applies a 2D power-average adaptive pooling over an input signal composed of several input planes. The function computed is: :math:`f(X) = pow(sum(pow(X, p)), 1/p)` - At p = infinity, one gets Max Pooling - At p = 1, one gets Average Pooling The output is of size H x W, for any input size. The number of output features is equal to the number of input planes. Args: output_size: the target output size of the image of the form H x W. Can be a tuple (H, W) or a single H for a square image H x H H and W can be either a ``int``, or ``None`` which means the size will be the same as that of the input. """ def __init__(self, output_size=1, eps=1e-6): super(GeneralizedMeanPoolingList, self).__init__() self.output_size = output_size self.eps = eps def forward(self, x_list): outs = [] for x in x_list: x = x.clamp(min=self.eps) out = torch.nn.functional.adaptive_avg_pool2d(x, self.output_size) outs.append(out) return torch.stack(outs, -1).mean(-1) def __repr__(self): return ( self.__class__.__name__ + "(" + "output_size=" + str(self.output_size) + ")" ) class GeneralizedMeanPooling(nn.Module, ABC): r"""Applies a 2D power-average adaptive pooling over an input signal composed of several input planes. The function computed is: :math:`f(X) = pow(sum(pow(X, p)), 1/p)` - At p = infinity, one gets Max Pooling - At p = 1, one gets Average Pooling The output is of size H x W, for any input size. The number of output features is equal to the number of input planes. Args: output_size: the target output size of the image of the form H x W. Can be a tuple (H, W) or a single H for a square image H x H H and W can be either a ``int``, or ``None`` which means the size will be the same as that of the input. """ def __init__(self, norm, output_size=1, eps=1e-6): super(GeneralizedMeanPooling, self).__init__() assert norm > 0 self.p = float(norm) self.output_size = output_size self.eps = eps def forward(self, x): x = x.clamp(min=self.eps).pow(self.p) return torch.nn.functional.adaptive_avg_pool2d(x, self.output_size).pow( 1.0 / self.p ) def __repr__(self): return ( self.__class__.__name__ + "(" + str(self.p) + ", " + "output_size=" + str(self.output_size) + ")" ) class GeneralizedMeanPoolingP(GeneralizedMeanPooling, ABC): """ Same, but norm is trainable """ def __init__(self, norm=3, output_size=1, eps=1e-6): super(GeneralizedMeanPoolingP, self).__init__(norm, output_size, eps) self.p = nn.Parameter(torch.ones(1) * norm) class GeneralizedMeanPoolingFpn(nn.Module, ABC): r"""Applies a 2D power-average adaptive pooling over an input signal composed of several input planes. The function computed is: :math:`f(X) = pow(sum(pow(X, p)), 1/p)` - At p = infinity, one gets Max Pooling - At p = 1, one gets Average Pooling The output is of size H x W, for any input size. The number of output features is equal to the number of input planes. Args: output_size: the target output size of the image of the form H x W. Can be a tuple (H, W) or a single H for a square image H x H H and W can be either a ``int``, or ``None`` which means the size will be the same as that of the input. """ def __init__(self, norm, output_size=1, eps=1e-6): super(GeneralizedMeanPoolingFpn, self).__init__() assert norm > 0 self.p = float(norm) self.output_size = output_size self.eps = eps def forward(self, x_lists): outs = [] for x in x_lists: x = x.clamp(min=self.eps).pow(self.p) out = torch.nn.functional.adaptive_avg_pool2d(x, self.output_size).pow( 1.0 / self.p ) outs.append(out) return torch.cat(outs, 1) def __repr__(self): return ( self.__class__.__name__ + "(" + str(self.p) + ", " + "output_size=" + str(self.output_size) + ")" ) class GeneralizedMeanPoolingPFpn(GeneralizedMeanPoolingFpn, ABC): """ Same, but norm is trainable """ def __init__(self, norm=3, output_size=1, eps=1e-6): super(GeneralizedMeanPoolingPFpn, self).__init__(norm, output_size, eps) self.p = nn.Parameter(torch.ones(1) * norm) class AdaptiveAvgMaxPool2d(nn.Module, ABC): def __init__(self): super(AdaptiveAvgMaxPool2d, self).__init__() self.avgpool = FastGlobalAvgPool2d() def forward(self, x): x_avg = self.avgpool(x, self.output_size) x_max = F.adaptive_max_pool2d(x, 1) x = x_max + x_avg return x class FastGlobalAvgPool2d(nn.Module, ABC): def __init__(self, flatten=False): super(FastGlobalAvgPool2d, self).__init__() self.flatten = flatten def forward(self, x): if self.flatten: in_size = x.size() return x.view((in_size[0], in_size[1], -1)).mean(dim=2) else: return ( x.view(x.size(0), x.size(1), -1) .mean(-1) .view(x.size(0), x.size(1), 1, 1) ) def avg_pooling(): return nn.AdaptiveAvgPool2d(1) # return FastGlobalAvgPool2d() def max_pooling(): return nn.AdaptiveMaxPool2d(1) class Flatten(nn.Module): def forward(self, input): return input.view(input.size(0), -1) __pooling_factory = { "avg": avg_pooling, "max": max_pooling, "gem": GeneralizedMeanPoolingP, "gemFpn": GeneralizedMeanPoolingPFpn, "gemList": GeneralizedMeanPoolingList, "avg+max": AdaptiveAvgMaxPool2d, } def pooling_names(): return sorted(__pooling_factory.keys()) def build_pooling_layer(name): """ Create a pooling layer. Parameters ---------- name : str The backbone name. """ if name not in __pooling_factory: raise KeyError("Unknown pooling layer:", name) return __pooling_factory[name]() ================================================ FILE: clustercontrast/models/resnet.py ================================================ from __future__ import absolute_import from torch import nn from torch.nn import functional as F from torch.nn import init import torchvision import torch from .pooling import build_pooling_layer __all__ = ['ResNet', 'resnet18', 'resnet34', 'resnet50', 'resnet101', 'resnet152'] class ResNet(nn.Module): __factory = { 18: torchvision.models.resnet18, 34: torchvision.models.resnet34, 50: torchvision.models.resnet50, 101: torchvision.models.resnet101, 152: torchvision.models.resnet152, } def __init__(self, depth, pretrained=True, cut_at_pooling=False, num_features=0, norm=False, dropout=0, num_classes=0, pooling_type='avg'): super(ResNet, self).__init__() self.pretrained = pretrained self.depth = depth self.cut_at_pooling = cut_at_pooling # Construct base (pretrained) resnet if depth not in ResNet.__factory: raise KeyError("Unsupported depth:", depth) resnet = ResNet.__factory[depth](pretrained=pretrained) resnet.layer4[0].conv2.stride = (1, 1) resnet.layer4[0].downsample[0].stride = (1, 1) self.base = nn.Sequential( resnet.conv1, resnet.bn1, resnet.relu, resnet.maxpool, resnet.layer1, resnet.layer2, resnet.layer3, resnet.layer4) self.gap = build_pooling_layer(pooling_type) if not self.cut_at_pooling: self.num_features = num_features self.norm = norm self.dropout = dropout self.has_embedding = num_features > 0 self.num_classes = num_classes out_planes = resnet.fc.in_features # Append new layers if self.has_embedding: self.feat = nn.Linear(out_planes, self.num_features) self.feat_bn = nn.BatchNorm1d(self.num_features) init.kaiming_normal_(self.feat.weight, mode='fan_out') init.constant_(self.feat.bias, 0) else: # Change the num_features to CNN output channels self.num_features = out_planes self.feat_bn = nn.BatchNorm1d(self.num_features) self.feat_bn.bias.requires_grad_(False) if self.dropout > 0: self.drop = nn.Dropout(self.dropout) if self.num_classes > 0: self.classifier = nn.Linear(self.num_features, self.num_classes, bias=False) init.normal_(self.classifier.weight, std=0.001) init.constant_(self.feat_bn.weight, 1) init.constant_(self.feat_bn.bias, 0) if not pretrained: self.reset_params() def forward(self, x): bs = x.size(0) x = self.base(x) x = self.gap(x) x = x.view(x.size(0), -1) if self.cut_at_pooling: return x if self.has_embedding: bn_x = self.feat_bn(self.feat(x)) else: bn_x = self.feat_bn(x) if (self.training is False): bn_x = F.normalize(bn_x) return bn_x if self.norm: bn_x = F.normalize(bn_x) elif self.has_embedding: bn_x = F.relu(bn_x) if self.dropout > 0: bn_x = self.drop(bn_x) if self.num_classes > 0: prob = self.classifier(bn_x) else: return bn_x return prob def reset_params(self): for m in self.modules(): if isinstance(m, nn.Conv2d): init.kaiming_normal_(m.weight, mode='fan_out') if m.bias is not None: init.constant_(m.bias, 0) elif isinstance(m, nn.BatchNorm2d): init.constant_(m.weight, 1) init.constant_(m.bias, 0) elif isinstance(m, nn.BatchNorm1d): init.constant_(m.weight, 1) init.constant_(m.bias, 0) elif isinstance(m, nn.Linear): init.normal_(m.weight, std=0.001) if m.bias is not None: init.constant_(m.bias, 0) def resnet18(**kwargs): return ResNet(18, **kwargs) def resnet34(**kwargs): return ResNet(34, **kwargs) def resnet50(**kwargs): return ResNet(50, **kwargs) def resnet101(**kwargs): return ResNet(101, **kwargs) def resnet152(**kwargs): return ResNet(152, **kwargs) ================================================ FILE: clustercontrast/models/resnet_ibn.py ================================================ from __future__ import absolute_import from torch import nn from torch.nn import functional as F from torch.nn import init import torchvision import torch from .pooling import build_pooling_layer from .resnet_ibn_a import resnet50_ibn_a, resnet101_ibn_a __all__ = ['ResNetIBN', 'resnet_ibn50a', 'resnet_ibn101a'] class ResNetIBN(nn.Module): __factory = { '50a': resnet50_ibn_a, '101a': resnet101_ibn_a } def __init__(self, depth, pretrained=True, cut_at_pooling=False, num_features=0, norm=False, dropout=0, num_classes=0, pooling_type='avg'): super(ResNetIBN, self).__init__() self.depth = depth self.pretrained = pretrained self.cut_at_pooling = cut_at_pooling resnet = ResNetIBN.__factory[depth](pretrained=pretrained) resnet.layer4[0].conv2.stride = (1, 1) resnet.layer4[0].downsample[0].stride = (1, 1) self.base = nn.Sequential( resnet.conv1, resnet.bn1, resnet.relu, resnet.maxpool, resnet.layer1, resnet.layer2, resnet.layer3, resnet.layer4) self.gap = build_pooling_layer(pooling_type) if not self.cut_at_pooling: self.num_features = num_features self.norm = norm self.dropout = dropout self.has_embedding = num_features > 0 self.num_classes = num_classes out_planes = resnet.fc.in_features # Append new layers if self.has_embedding: self.feat = nn.Linear(out_planes, self.num_features) self.feat_bn = nn.BatchNorm1d(self.num_features) init.kaiming_normal_(self.feat.weight, mode='fan_out') init.constant_(self.feat.bias, 0) else: # Change the num_features to CNN output channels self.num_features = out_planes self.feat_bn = nn.BatchNorm1d(self.num_features) self.feat_bn.bias.requires_grad_(False) if self.dropout > 0: self.drop = nn.Dropout(self.dropout) if self.num_classes > 0: self.classifier = nn.Linear(self.num_features, self.num_classes, bias=False) init.normal_(self.classifier.weight, std=0.001) init.constant_(self.feat_bn.weight, 1) init.constant_(self.feat_bn.bias, 0) if not pretrained: self.reset_params() def forward(self, x): x = self.base(x) x = self.gap(x) x = x.view(x.size(0), -1) if self.cut_at_pooling: return x if self.has_embedding: bn_x = self.feat_bn(self.feat(x)) else: bn_x = self.feat_bn(x) if self.training is False: bn_x = F.normalize(bn_x) return bn_x if self.norm: bn_x = F.normalize(bn_x) elif self.has_embedding: bn_x = F.relu(bn_x) if self.dropout > 0: bn_x = self.drop(bn_x) if self.num_classes > 0: prob = self.classifier(bn_x) else: return bn_x return prob def reset_params(self): for m in self.modules(): if isinstance(m, nn.Conv2d): init.kaiming_normal_(m.weight, mode='fan_out') if m.bias is not None: init.constant_(m.bias, 0) elif isinstance(m, nn.BatchNorm2d): init.constant_(m.weight, 1) init.constant_(m.bias, 0) elif isinstance(m, nn.BatchNorm1d): init.constant_(m.weight, 1) init.constant_(m.bias, 0) elif isinstance(m, nn.Linear): init.normal_(m.weight, std=0.001) if m.bias is not None: init.constant_(m.bias, 0) def resnet_ibn50a(**kwargs): return ResNetIBN('50a', **kwargs) def resnet_ibn101a(**kwargs): return ResNetIBN('101a', **kwargs) ================================================ FILE: clustercontrast/models/resnet_ibn_a.py ================================================ import torch import torch.nn as nn import math import torch.utils.model_zoo as model_zoo __all__ = ['ResNet', 'resnet50_ibn_a', 'resnet101_ibn_a'] model_urls = { 'ibn_resnet50a': './examples/pretrained/resnet50_ibn_a.pth.tar', 'ibn_resnet101a': './examples/pretrained/resnet101_ibn_a.pth.tar', } def conv3x3(in_planes, out_planes, stride=1): "3x3 convolution with padding" return nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=stride, padding=1, bias=False) class BasicBlock(nn.Module): expansion = 1 def __init__(self, inplanes, planes, stride=1, downsample=None): super(BasicBlock, self).__init__() self.conv1 = conv3x3(inplanes, planes, stride) self.bn1 = nn.BatchNorm2d(planes) self.relu = nn.ReLU(inplace=True) self.conv2 = conv3x3(planes, planes) self.bn2 = nn.BatchNorm2d(planes) self.downsample = downsample self.stride = stride def forward(self, x): residual = x out = self.conv1(x) out = self.bn1(out) out = self.relu(out) out = self.conv2(out) out = self.bn2(out) if self.downsample is not None: residual = self.downsample(x) out += residual out = self.relu(out) return out class IBN(nn.Module): def __init__(self, planes): super(IBN, self).__init__() half1 = int(planes/2) self.half = half1 half2 = planes - half1 self.IN = nn.InstanceNorm2d(half1, affine=True) self.BN = nn.BatchNorm2d(half2) def forward(self, x): split = torch.split(x, self.half, 1) out1 = self.IN(split[0].contiguous()) out2 = self.BN(split[1].contiguous()) out = torch.cat((out1, out2), 1) return out class Bottleneck(nn.Module): expansion = 4 def __init__(self, inplanes, planes, ibn=False, stride=1, downsample=None): super(Bottleneck, self).__init__() self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=1, bias=False) if ibn: self.bn1 = IBN(planes) else: self.bn1 = nn.BatchNorm2d(planes) self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride, padding=1, bias=False) self.bn2 = nn.BatchNorm2d(planes) self.conv3 = nn.Conv2d(planes, planes * self.expansion, kernel_size=1, bias=False) self.bn3 = nn.BatchNorm2d(planes * self.expansion) self.relu = nn.ReLU(inplace=True) self.downsample = downsample self.stride = stride def forward(self, x): residual = x out = self.conv1(x) out = self.bn1(out) out = self.relu(out) out = self.conv2(out) out = self.bn2(out) out = self.relu(out) out = self.conv3(out) out = self.bn3(out) if self.downsample is not None: residual = self.downsample(x) out += residual out = self.relu(out) return out class ResNet(nn.Module): def __init__(self, block, layers, num_classes=1000): scale = 64 self.inplanes = scale super(ResNet, self).__init__() self.conv1 = nn.Conv2d(3, scale, kernel_size=7, stride=2, padding=3, bias=False) self.bn1 = nn.BatchNorm2d(scale) self.relu = nn.ReLU(inplace=True) self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1) self.layer1 = self._make_layer(block, scale, layers[0]) self.layer2 = self._make_layer(block, scale*2, layers[1], stride=2) self.layer3 = self._make_layer(block, scale*4, layers[2], stride=2) self.layer4 = self._make_layer(block, scale*8, layers[3], stride=2) self.avgpool = nn.AvgPool2d(7) self.fc = nn.Linear(scale * 8 * block.expansion, num_classes) for m in self.modules(): if isinstance(m, nn.Conv2d): n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels m.weight.data.normal_(0, math.sqrt(2. / n)) elif isinstance(m, nn.BatchNorm2d): m.weight.data.fill_(1) m.bias.data.zero_() elif isinstance(m, nn.InstanceNorm2d): m.weight.data.fill_(1) m.bias.data.zero_() def _make_layer(self, block, planes, blocks, stride=1): downsample = None if stride != 1 or self.inplanes != planes * block.expansion: downsample = nn.Sequential( nn.Conv2d(self.inplanes, planes * block.expansion, kernel_size=1, stride=stride, bias=False), nn.BatchNorm2d(planes * block.expansion), ) layers = [] ibn = True if planes == 512: ibn = False layers.append(block(self.inplanes, planes, ibn, stride, downsample)) self.inplanes = planes * block.expansion for i in range(1, blocks): layers.append(block(self.inplanes, planes, ibn)) return nn.Sequential(*layers) def forward(self, x): x = self.conv1(x) x = self.bn1(x) x = self.relu(x) x = self.maxpool(x) x = self.layer1(x) x = self.layer2(x) x = self.layer3(x) x = self.layer4(x) x = self.avgpool(x) x = x.view(x.size(0), -1) x = self.fc(x) return x def resnet50_ibn_a(pretrained=False, **kwargs): """Constructs a ResNet-50 model. Args: pretrained (bool): If True, returns a model pre-trained on ImageNet """ model = ResNet(Bottleneck, [3, 4, 6, 3], **kwargs) if pretrained: state_dict = torch.load(model_urls['ibn_resnet50a'], map_location=torch.device('cpu'))['state_dict'] state_dict = remove_module_key(state_dict) model.load_state_dict(state_dict) return model def resnet101_ibn_a(pretrained=False, **kwargs): """Constructs a ResNet-101 model. Args: pretrained (bool): If True, returns a model pre-trained on ImageNet """ model = ResNet(Bottleneck, [3, 4, 23, 3], **kwargs) if pretrained: state_dict = torch.load(model_urls['ibn_resnet101a'], map_location=torch.device('cpu'))['state_dict'] state_dict = remove_module_key(state_dict) model.load_state_dict(state_dict) return model def remove_module_key(state_dict): for key in list(state_dict.keys()): if 'module' in key: state_dict[key.replace('module.','')] = state_dict.pop(key) return state_dict ================================================ FILE: clustercontrast/trainers.py ================================================ from __future__ import print_function, absolute_import import time from .utils.meters import AverageMeter class ClusterContrastTrainer(object): def __init__(self, encoder, memory=None): super(ClusterContrastTrainer, self).__init__() self.encoder = encoder self.memory = memory def train(self, epoch, data_loader, optimizer, print_freq=10, train_iters=400): self.encoder.train() batch_time = AverageMeter() data_time = AverageMeter() losses = AverageMeter() end = time.time() for i in range(train_iters): # load data inputs = data_loader.next() data_time.update(time.time() - end) # process inputs inputs, labels, indexes = self._parse_data(inputs) # forward f_out = self._forward(inputs) # print("f_out shape: {}".format(f_out.shape)) # compute loss with the hybrid memory # loss = self.memory(f_out, indexes) loss = self.memory(f_out, labels) optimizer.zero_grad() loss.backward() optimizer.step() losses.update(loss.item()) # print log batch_time.update(time.time() - end) end = time.time() if (i + 1) % print_freq == 0: print('Epoch: [{}][{}/{}]\t' 'Time {:.3f} ({:.3f})\t' 'Data {:.3f} ({:.3f})\t' 'Loss {:.3f} ({:.3f})' .format(epoch, i + 1, len(data_loader), batch_time.val, batch_time.avg, data_time.val, data_time.avg, losses.val, losses.avg)) def _parse_data(self, inputs): imgs, _, pids, _, indexes = inputs return imgs.cuda(), pids.cuda(), indexes.cuda() def _forward(self, inputs): return self.encoder(inputs) ================================================ FILE: clustercontrast/utils/__init__.py ================================================ from __future__ import absolute_import import torch def to_numpy(tensor): if torch.is_tensor(tensor): return tensor.cpu().numpy() elif type(tensor).__module__ != 'numpy': raise ValueError("Cannot convert {} to numpy array" .format(type(tensor))) return tensor def to_torch(ndarray): if type(ndarray).__module__ == 'numpy': return torch.from_numpy(ndarray) elif not torch.is_tensor(ndarray): raise ValueError("Cannot convert {} to torch tensor" .format(type(ndarray))) return ndarray ================================================ FILE: clustercontrast/utils/data/__init__.py ================================================ from __future__ import absolute_import from .base_dataset import BaseDataset, BaseImageDataset from .preprocessor import Preprocessor class IterLoader: def __init__(self, loader, length=None): self.loader = loader self.length = length self.iter = None def __len__(self): if self.length is not None: return self.length return len(self.loader) def new_epoch(self): self.iter = iter(self.loader) def next(self): try: return next(self.iter) except: self.iter = iter(self.loader) return next(self.iter) ================================================ FILE: clustercontrast/utils/data/base_dataset.py ================================================ # encoding: utf-8 import numpy as np class BaseDataset(object): """ Base class of reid dataset """ def get_imagedata_info(self, data): pids, cams = [], [] for _, pid, camid in data: pids += [pid] cams += [camid] pids = set(pids) cams = set(cams) num_pids = len(pids) num_cams = len(cams) num_imgs = len(data) return num_pids, num_imgs, num_cams def print_dataset_statistics(self): raise NotImplementedError @property def images_dir(self): return None class BaseImageDataset(BaseDataset): """ Base class of image reid dataset """ def print_dataset_statistics(self, train, query, gallery): num_train_pids, num_train_imgs, num_train_cams = self.get_imagedata_info(train) num_query_pids, num_query_imgs, num_query_cams = self.get_imagedata_info(query) num_gallery_pids, num_gallery_imgs, num_gallery_cams = self.get_imagedata_info(gallery) print("Dataset statistics:") print(" ----------------------------------------") print(" subset | # ids | # images | # cameras") print(" ----------------------------------------") print(" train | {:5d} | {:8d} | {:9d}".format(num_train_pids, num_train_imgs, num_train_cams)) print(" query | {:5d} | {:8d} | {:9d}".format(num_query_pids, num_query_imgs, num_query_cams)) print(" gallery | {:5d} | {:8d} | {:9d}".format(num_gallery_pids, num_gallery_imgs, num_gallery_cams)) print(" ----------------------------------------") ================================================ FILE: clustercontrast/utils/data/preprocessor.py ================================================ from __future__ import absolute_import import os import os.path as osp from torch.utils.data import DataLoader, Dataset import numpy as np import random import math from PIL import Image class Preprocessor(Dataset): def __init__(self, dataset, root=None, transform=None): super(Preprocessor, self).__init__() self.dataset = dataset self.root = root self.transform = transform def __len__(self): return len(self.dataset) def __getitem__(self, indices): return self._get_single_item(indices) def _get_single_item(self, index): fname, pid, camid = self.dataset[index] fpath = fname if self.root is not None: fpath = osp.join(self.root, fname) img = Image.open(fpath).convert('RGB') if self.transform is not None: img = self.transform(img) return img, fname, pid, camid, index ================================================ FILE: clustercontrast/utils/data/sampler.py ================================================ from __future__ import absolute_import from collections import defaultdict import math import numpy as np import copy import random import torch from torch.utils.data.sampler import ( Sampler, SequentialSampler, RandomSampler, SubsetRandomSampler, WeightedRandomSampler) def No_index(a, b): assert isinstance(a, list) return [i for i, j in enumerate(a) if j != b] class RandomIdentitySampler(Sampler): def __init__(self, data_source, num_instances): self.data_source = data_source self.num_instances = num_instances self.index_dic = defaultdict(list) for index, (_, pid, _) in enumerate(data_source): self.index_dic[pid].append(index) self.pids = list(self.index_dic.keys()) self.num_samples = len(self.pids) def __len__(self): return self.num_samples * self.num_instances def __iter__(self): indices = torch.randperm(self.num_samples).tolist() ret = [] for i in indices: pid = self.pids[i] t = self.index_dic[pid] if len(t) >= self.num_instances: t = np.random.choice(t, size=self.num_instances, replace=False) else: t = np.random.choice(t, size=self.num_instances, replace=True) ret.extend(t) return iter(ret) class RandomMultipleGallerySampler(Sampler): def __init__(self, data_source, num_instances=4): super().__init__(data_source) self.data_source = data_source self.index_pid = defaultdict(int) self.pid_cam = defaultdict(list) self.pid_index = defaultdict(list) self.num_instances = num_instances for index, (_, pid, cam) in enumerate(data_source): if pid < 0: continue self.index_pid[index] = pid self.pid_cam[pid].append(cam) self.pid_index[pid].append(index) self.pids = list(self.pid_index.keys()) self.num_samples = len(self.pids) def __len__(self): return self.num_samples * self.num_instances def __iter__(self): indices = torch.randperm(len(self.pids)).tolist() ret = [] for kid in indices: i = random.choice(self.pid_index[self.pids[kid]]) _, i_pid, i_cam = self.data_source[i] ret.append(i) pid_i = self.index_pid[i] cams = self.pid_cam[pid_i] index = self.pid_index[pid_i] select_cams = No_index(cams, i_cam) if select_cams: if len(select_cams) >= self.num_instances: cam_indexes = np.random.choice(select_cams, size=self.num_instances-1, replace=False) else: cam_indexes = np.random.choice(select_cams, size=self.num_instances-1, replace=True) for kk in cam_indexes: ret.append(index[kk]) else: select_indexes = No_index(index, i) if not select_indexes: continue if len(select_indexes) >= self.num_instances: ind_indexes = np.random.choice(select_indexes, size=self.num_instances-1, replace=False) else: ind_indexes = np.random.choice(select_indexes, size=self.num_instances-1, replace=True) for kk in ind_indexes: ret.append(index[kk]) return iter(ret) class RandomMultipleGallerySamplerNoCam(Sampler): def __init__(self, data_source, num_instances=4): super().__init__(data_source) self.data_source = data_source self.index_pid = defaultdict(int) self.pid_index = defaultdict(list) self.num_instances = num_instances for index, (_, pid, cam) in enumerate(data_source): if pid < 0: continue self.index_pid[index] = pid self.pid_index[pid].append(index) self.pids = list(self.pid_index.keys()) self.num_samples = len(self.pids) def __len__(self): return self.num_samples * self.num_instances def __iter__(self): indices = torch.randperm(len(self.pids)).tolist() ret = [] for kid in indices: i = random.choice(self.pid_index[self.pids[kid]]) _, i_pid, i_cam = self.data_source[i] ret.append(i) pid_i = self.index_pid[i] index = self.pid_index[pid_i] select_indexes = No_index(index, i) if not select_indexes: continue if len(select_indexes) >= self.num_instances: ind_indexes = np.random.choice(select_indexes, size=self.num_instances-1, replace=False) else: ind_indexes = np.random.choice(select_indexes, size=self.num_instances-1, replace=True) for kk in ind_indexes: ret.append(index[kk]) return iter(ret) ================================================ FILE: clustercontrast/utils/data/transforms.py ================================================ from __future__ import absolute_import from torchvision.transforms import * from PIL import Image import random import math import numpy as np class RectScale(object): def __init__(self, height, width, interpolation=Image.BILINEAR): self.height = height self.width = width self.interpolation = interpolation def __call__(self, img): w, h = img.size if h == self.height and w == self.width: return img return img.resize((self.width, self.height), self.interpolation) class RandomSizedRectCrop(object): def __init__(self, height, width, interpolation=Image.BILINEAR): self.height = height self.width = width self.interpolation = interpolation def __call__(self, img): for attempt in range(10): area = img.size[0] * img.size[1] target_area = random.uniform(0.64, 1.0) * area aspect_ratio = random.uniform(2, 3) h = int(round(math.sqrt(target_area * aspect_ratio))) w = int(round(math.sqrt(target_area / aspect_ratio))) if w <= img.size[0] and h <= img.size[1]: x1 = random.randint(0, img.size[0] - w) y1 = random.randint(0, img.size[1] - h) img = img.crop((x1, y1, x1 + w, y1 + h)) assert(img.size == (w, h)) return img.resize((self.width, self.height), self.interpolation) # Fallback scale = RectScale(self.height, self.width, interpolation=self.interpolation) return scale(img) class RandomErasing(object): """ Randomly selects a rectangle region in an image and erases its pixels. 'Random Erasing Data Augmentation' by Zhong et al. See https://arxiv.org/pdf/1708.04896.pdf Args: probability: The probability that the Random Erasing operation will be performed. sl: Minimum proportion of erased area against input image. sh: Maximum proportion of erased area against input image. r1: Minimum aspect ratio of erased area. mean: Erasing value. """ def __init__(self, probability=0.5, sl=0.02, sh=0.4, r1=0.3, mean=(0.4914, 0.4822, 0.4465)): self.probability = probability self.mean = mean self.sl = sl self.sh = sh self.r1 = r1 def __call__(self, img): if random.uniform(0, 1) >= self.probability: return img for attempt in range(100): area = img.size()[1] * img.size()[2] target_area = random.uniform(self.sl, self.sh) * area aspect_ratio = random.uniform(self.r1, 1 / self.r1) h = int(round(math.sqrt(target_area * aspect_ratio))) w = int(round(math.sqrt(target_area / aspect_ratio))) if w < img.size()[2] and h < img.size()[1]: x1 = random.randint(0, img.size()[1] - h) y1 = random.randint(0, img.size()[2] - w) if img.size()[0] == 3: img[0, x1:x1 + h, y1:y1 + w] = self.mean[0] img[1, x1:x1 + h, y1:y1 + w] = self.mean[1] img[2, x1:x1 + h, y1:y1 + w] = self.mean[2] else: img[0, x1:x1 + h, y1:y1 + w] = self.mean[0] return img return img ================================================ FILE: clustercontrast/utils/faiss_rerank.py ================================================ #!/usr/bin/env python3 # -*- coding: utf-8 -*- """ CVPR2017 paper:Zhong Z, Zheng L, Cao D, et al. Re-ranking Person Re-identification with k-reciprocal Encoding[J]. 2017. url:http://openaccess.thecvf.com/content_cvpr_2017/papers/Zhong_Re-Ranking_Person_Re-Identification_CVPR_2017_paper.pdf Matlab version: https://github.com/zhunzhong07/person-re-ranking """ import os, sys import time import numpy as np from scipy.spatial.distance import cdist import gc import faiss import torch import torch.nn.functional as F from .faiss_utils import search_index_pytorch, search_raw_array_pytorch, \ index_init_gpu, index_init_cpu def k_reciprocal_neigh(initial_rank, i, k1): forward_k_neigh_index = initial_rank[i,:k1+1] backward_k_neigh_index = initial_rank[forward_k_neigh_index,:k1+1] fi = np.where(backward_k_neigh_index==i)[0] return forward_k_neigh_index[fi] def compute_jaccard_distance(target_features, k1=20, k2=6, print_flag=True, search_option=0, use_float16=False): end = time.time() if print_flag: print('Computing jaccard distance...') ngpus = faiss.get_num_gpus() N = target_features.size(0) mat_type = np.float16 if use_float16 else np.float32 if (search_option==0): # GPU + PyTorch CUDA Tensors (1) res = faiss.StandardGpuResources() res.setDefaultNullStreamAllDevices() _, initial_rank = search_raw_array_pytorch(res, target_features, target_features, k1) initial_rank = initial_rank.cpu().numpy() elif (search_option==1): # GPU + PyTorch CUDA Tensors (2) res = faiss.StandardGpuResources() index = faiss.GpuIndexFlatL2(res, target_features.size(-1)) index.add(target_features.cpu().numpy()) _, initial_rank = search_index_pytorch(index, target_features, k1) res.syncDefaultStreamCurrentDevice() initial_rank = initial_rank.cpu().numpy() elif (search_option==2): # GPU index = index_init_gpu(ngpus, target_features.size(-1)) index.add(target_features.cpu().numpy()) _, initial_rank = index.search(target_features.cpu().numpy(), k1) else: # CPU index = index_init_cpu(target_features.size(-1)) index.add(target_features.cpu().numpy()) _, initial_rank = index.search(target_features.cpu().numpy(), k1) nn_k1 = [] nn_k1_half = [] for i in range(N): nn_k1.append(k_reciprocal_neigh(initial_rank, i, k1)) nn_k1_half.append(k_reciprocal_neigh(initial_rank, i, int(np.around(k1/2)))) V = np.zeros((N, N), dtype=mat_type) for i in range(N): k_reciprocal_index = nn_k1[i] k_reciprocal_expansion_index = k_reciprocal_index for candidate in k_reciprocal_index: candidate_k_reciprocal_index = nn_k1_half[candidate] if (len(np.intersect1d(candidate_k_reciprocal_index,k_reciprocal_index)) > 2/3*len(candidate_k_reciprocal_index)): k_reciprocal_expansion_index = np.append(k_reciprocal_expansion_index,candidate_k_reciprocal_index) k_reciprocal_expansion_index = np.unique(k_reciprocal_expansion_index) ## element-wise unique dist = 2-2*torch.mm(target_features[i].unsqueeze(0).contiguous(), target_features[k_reciprocal_expansion_index].t()) if use_float16: V[i,k_reciprocal_expansion_index] = F.softmax(-dist, dim=1).view(-1).cpu().numpy().astype(mat_type) else: V[i,k_reciprocal_expansion_index] = F.softmax(-dist, dim=1).view(-1).cpu().numpy() del nn_k1, nn_k1_half if k2 != 1: V_qe = np.zeros_like(V, dtype=mat_type) for i in range(N): V_qe[i,:] = np.mean(V[initial_rank[i,:k2],:], axis=0) V = V_qe del V_qe del initial_rank invIndex = [] for i in range(N): invIndex.append(np.where(V[:,i] != 0)[0]) #len(invIndex)=all_num jaccard_dist = np.zeros((N, N), dtype=mat_type) for i in range(N): temp_min = np.zeros((1, N), dtype=mat_type) # temp_max = np.zeros((1,N), dtype=mat_type) indNonZero = np.where(V[i, :] != 0)[0] indImages = [] indImages = [invIndex[ind] for ind in indNonZero] for j in range(len(indNonZero)): temp_min[0, indImages[j]] = temp_min[0, indImages[j]]+np.minimum(V[i, indNonZero[j]], V[indImages[j], indNonZero[j]]) # temp_max[0,indImages[j]] = temp_max[0,indImages[j]]+np.maximum(V[i,indNonZero[j]],V[indImages[j],indNonZero[j]]) jaccard_dist[i] = 1-temp_min/(2-temp_min) # jaccard_dist[i] = 1-temp_min/(temp_max+1e-6) del invIndex, V pos_bool = (jaccard_dist < 0) jaccard_dist[pos_bool] = 0.0 if print_flag: print("Jaccard distance computing time cost: {}".format(time.time()-end)) return jaccard_dist ================================================ FILE: clustercontrast/utils/faiss_utils.py ================================================ import os import numpy as np import faiss import torch def swig_ptr_from_FloatTensor(x): assert x.is_contiguous() assert x.dtype == torch.float32 return faiss.cast_integer_to_float_ptr( x.storage().data_ptr() + x.storage_offset() * 4) def swig_ptr_from_LongTensor(x): assert x.is_contiguous() assert x.dtype == torch.int64, 'dtype=%s' % x.dtype return faiss.cast_integer_to_idx_t_ptr( x.storage().data_ptr() + x.storage_offset() * 8) def search_index_pytorch(index, x, k, D=None, I=None): """call the search function of an index with pytorch tensor I/O (CPU and GPU supported)""" assert x.is_contiguous() n, d = x.size() assert d == index.d if D is None: D = torch.empty((n, k), dtype=torch.float32, device=x.device) else: assert D.size() == (n, k) if I is None: I = torch.empty((n, k), dtype=torch.int64, device=x.device) else: assert I.size() == (n, k) torch.cuda.synchronize() xptr = swig_ptr_from_FloatTensor(x) Iptr = swig_ptr_from_LongTensor(I) Dptr = swig_ptr_from_FloatTensor(D) index.search_c(n, xptr, k, Dptr, Iptr) torch.cuda.synchronize() return D, I def search_raw_array_pytorch(res, xb, xq, k, D=None, I=None, metric=faiss.METRIC_L2): assert xb.device == xq.device nq, d = xq.size() if xq.is_contiguous(): xq_row_major = True elif xq.t().is_contiguous(): xq = xq.t() # I initially wrote xq:t(), Lua is still haunting me :-) xq_row_major = False else: raise TypeError('matrix should be row or column-major') xq_ptr = swig_ptr_from_FloatTensor(xq) nb, d2 = xb.size() assert d2 == d if xb.is_contiguous(): xb_row_major = True elif xb.t().is_contiguous(): xb = xb.t() xb_row_major = False else: raise TypeError('matrix should be row or column-major') xb_ptr = swig_ptr_from_FloatTensor(xb) if D is None: D = torch.empty(nq, k, device=xb.device, dtype=torch.float32) else: assert D.shape == (nq, k) assert D.device == xb.device if I is None: I = torch.empty(nq, k, device=xb.device, dtype=torch.int64) else: assert I.shape == (nq, k) assert I.device == xb.device D_ptr = swig_ptr_from_FloatTensor(D) I_ptr = swig_ptr_from_LongTensor(I) faiss.bruteForceKnn(res, metric, xb_ptr, xb_row_major, nb, xq_ptr, xq_row_major, nq, d, k, D_ptr, I_ptr) return D, I def index_init_gpu(ngpus, feat_dim): flat_config = [] for i in range(ngpus): cfg = faiss.GpuIndexFlatConfig() cfg.useFloat16 = False cfg.device = i flat_config.append(cfg) res = [faiss.StandardGpuResources() for i in range(ngpus)] indexes = [faiss.GpuIndexFlatL2(res[i], feat_dim, flat_config[i]) for i in range(ngpus)] index = faiss.IndexShards(feat_dim) for sub_index in indexes: index.add_shard(sub_index) index.reset() return index def index_init_cpu(feat_dim): return faiss.IndexFlatL2(feat_dim) ================================================ FILE: clustercontrast/utils/infomap_cluster.py ================================================ import numpy as np from tqdm import tqdm import infomap import faiss import math import multiprocessing as mp from clustercontrast.utils.infomap_utils import Timer def l2norm(vec): """ 归一化 :param vec: :return: """ vec /= np.linalg.norm(vec, axis=1).reshape(-1, 1) return vec def intdict2ndarray(d, default_val=-1): arr = np.zeros(len(d)) + default_val for k, v in d.items(): arr[k] = v return arr def read_meta(fn_meta, start_pos=0, verbose=True): """ idx2lb：每一个顶点对应一个类 lb2idxs：每个类对应一个id """ lb2idxs = {} idx2lb = {} with open(fn_meta) as f: for idx, x in enumerate(f.readlines()[start_pos:]): lb = int(x.strip()) if lb not in lb2idxs: lb2idxs[lb] = [] lb2idxs[lb] += [idx] idx2lb[idx] = lb inst_num = len(idx2lb) cls_num = len(lb2idxs) if verbose: print('[{}] #cls: {}, #inst: {}'.format(fn_meta, cls_num, inst_num)) return lb2idxs, idx2lb class knn_faiss(): """ 内积暴力循环 归一化特征的内积等价于余弦相似度 """ def __init__(self, feats, k, knn_method='faiss-cpu', verbose=True): self.verbose = verbose with Timer('[{}] build index {}'.format(knn_method, k), verbose): feats = feats.astype('float32') size, dim = feats.shape if knn_method == 'faiss-gpu': i = math.ceil(size / 1000000) if i > 1: i = (i - 1) * 4 res = faiss.StandardGpuResources() res.setTempMemory(i * 1024 * 1024 * 1024) index = faiss.GpuIndexFlatIP(res, dim) else: index = faiss.IndexFlatIP(dim) index.add(feats) with Timer('[{}] query topk {}'.format(knn_method, k), verbose): sims, nbrs = index.search(feats, k=k) self.knns = [(np.array(nbr, dtype=np.int32), 1 - np.array(sim, dtype=np.float32)) for nbr, sim in zip(nbrs, sims)] def filter_by_th(self, i): th_nbrs = [] th_dists = [] nbrs, dists = self.knns[i] for n, dist in zip(nbrs, dists): if 1 - dist < self.th: continue th_nbrs.append(n) th_dists.append(dist) th_nbrs = np.array(th_nbrs) th_dists = np.array(th_dists) return th_nbrs, th_dists def get_knns(self, th=None): if th is None or th <= 0.: return self.knns # TODO: optimize the filtering process by numpy # nproc = mp.cpu_count() nproc = 1 with Timer('filter edges by th {} (CPU={})'.format(th, nproc), self.verbose): self.th = th self.th_knns = [] tot = len(self.knns) if nproc > 1: pool = mp.Pool(nproc) th_knns = list( tqdm(pool.imap(self.filter_by_th, range(tot)), total=tot)) pool.close() else: th_knns = [self.filter_by_th(i) for i in range(tot)] return th_knns def knns2ordered_nbrs(knns, sort=True): if isinstance(knns, list): knns = np.array(knns) nbrs = knns[:, 0, :].astype(np.int32) dists = knns[:, 1, :] if sort: # sort dists from low to high nb_idx = np.argsort(dists, axis=1) idxs = np.arange(nb_idx.shape[0]).reshape(-1, 1) dists = dists[idxs, nb_idx] nbrs = nbrs[idxs, nb_idx] return dists, nbrs # 构造边 def get_links(single, links, nbrs, dists, min_sim): for i in tqdm(range(nbrs.shape[0])): count = 0 for j in range(0, len(nbrs[i])): # 排除本身节点 if i == nbrs[i][j]: pass elif dists[i][j] <= 1 - min_sim: count += 1 links[(i, nbrs[i][j])] = float(1 - dists[i][j]) else: break # 统计孤立点 if count == 0: single.append(i) return single, links def cluster_by_infomap(nbrs, dists, min_sim, cluster_num=2): """ 基于infomap的聚类 :param nbrs: :param dists: :param pred_label_path: :return: """ single = [] links = {} with Timer('get links', verbose=True): single, links = get_links(single=single, links=links, nbrs=nbrs, dists=dists, min_sim=min_sim) infomapWrapper = infomap.Infomap("--two-level --directed") for (i, j), sim in tqdm(links.items()): _ = infomapWrapper.addLink(int(i), int(j), sim) # 聚类运算 infomapWrapper.run() label2idx = {} idx2label = {} # 聚类结果统计 for node in infomapWrapper.iterTree(): # node.physicalId 特征向量的编号 # node.moduleIndex() 聚类的编号 if node.moduleIndex() not in label2idx: label2idx[node.moduleIndex()] = [] label2idx[node.moduleIndex()].append(node.physicalId) node_count = 0 for k, v in label2idx.items(): if k == 0: each_index_list = v[2:] node_count += len(each_index_list) label2idx[k] = each_index_list else: each_index_list = v[1:] node_count += len(each_index_list) label2idx[k] = each_index_list for each_index in each_index_list: idx2label[each_index] = k keys_len = len(list(label2idx.keys())) # 孤立点放入到结果中 for single_node in single: idx2label[single_node] = keys_len label2idx[keys_len] = [single_node] keys_len += 1 node_count += 1 # 孤立点个数 print("孤立点数：{}".format(len(single))) idx_len = len(list(idx2label.keys())) assert idx_len == node_count, 'idx_len not equal node_count!' print("总节点数：{}".format(idx_len)) old_label_container = set() for each_label, each_index_list in label2idx.items(): if len(each_index_list) <= cluster_num: for each_index in each_index_list: idx2label[each_index] = -1 else: old_label_container.add(each_label) old2new = {old_label: new_label for new_label, old_label in enumerate(old_label_container)} for each_index, each_label in idx2label.items(): if each_label == -1: continue idx2label[each_index] = old2new[each_label] pre_labels = intdict2ndarray(idx2label) print("总类别数：{}/{}".format(keys_len, len(set(pre_labels)) - (1 if -1 in pre_labels else 0))) return pre_labels def get_dist_nbr(features, k=80, knn_method='faiss-cpu'): index = knn_faiss(feats=features, k=k, knn_method=knn_method) knns = index.get_knns() dists, nbrs = knns2ordered_nbrs(knns) return dists, nbrs ================================================ FILE: clustercontrast/utils/infomap_utils.py ================================================ import time class TextColors: HEADER = '\033[35m' OKBLUE = '\033[34m' OKGREEN = '\033[32m' WARNING = '\033[33m' FATAL = '\033[31m' ENDC = '\033[0m' BOLD = '\033[1m' UNDERLINE = '\033[4m' class Timer(): def __init__(self, name='task', verbose=True): self.name = name self.verbose = verbose def __enter__(self): self.start = time.time() return self def __exit__(self, exc_type, exc_val, exc_tb): if self.verbose: print('[Time] {} consumes {:.4f} s'.format( self.name, time.time() - self.start)) return exc_type is None ================================================ FILE: clustercontrast/utils/logging.py ================================================ from __future__ import absolute_import import os import sys from .osutils import mkdir_if_missing class Logger(object): def __init__(self, fpath=None): self.console = sys.stdout self.file = None if fpath is not None: mkdir_if_missing(os.path.dirname(fpath)) self.file = open(fpath, 'w') def __del__(self): self.close() def __enter__(self): pass def __exit__(self, *args): self.close() def write(self, msg): self.console.write(msg) if self.file is not None: self.file.write(msg) def flush(self): self.console.flush() if self.file is not None: self.file.flush() os.fsync(self.file.fileno()) def close(self): self.console.close() if self.file is not None: self.file.close() ================================================ FILE: clustercontrast/utils/meters.py ================================================ from __future__ import absolute_import class AverageMeter(object): """Computes and stores the average and current value""" def __init__(self): self.val = 0 self.avg = 0 self.sum = 0 self.count = 0 def reset(self): self.val = 0 self.avg = 0 self.sum = 0 self.count = 0 def update(self, val, n=1): self.val = val self.sum += val * n self.count += n self.avg = self.sum / self.count ================================================ FILE: clustercontrast/utils/osutils.py ================================================ from __future__ import absolute_import import os import errno def mkdir_if_missing(dir_path): try: os.makedirs(dir_path) except OSError as e: if e.errno != errno.EEXIST: raise ================================================ FILE: clustercontrast/utils/rerank.py ================================================ #!/usr/bin/env python2/python3 # -*- coding: utf-8 -*- """ Source: https://github.com/zhunzhong07/person-re-ranking Created on Mon Jun 26 14:46:56 2017 @author: luohao Modified by Houjing Huang, 2017-12-22. - This version accepts distance matrix instead of raw features. - The difference of `/` division between python 2 and 3 is handled. - numpy.float16 is replaced by numpy.float32 for numerical precision. CVPR2017 paper:Zhong Z, Zheng L, Cao D, et al. Re-ranking Person Re-identification with k-reciprocal Encoding[J]. 2017. url:http://openaccess.thecvf.com/content_cvpr_2017/papers/Zhong_Re-Ranking_Person_Re-Identification_CVPR_2017_paper.pdf Matlab version: https://github.com/zhunzhong07/person-re-ranking API q_g_dist: query-gallery distance matrix, numpy array, shape [num_query, num_gallery] q_q_dist: query-query distance matrix, numpy array, shape [num_query, num_query] g_g_dist: gallery-gallery distance matrix, numpy array, shape [num_gallery, num_gallery] k1, k2, lambda_value: parameters, the original paper is (k1=20, k2=6, lambda_value=0.3) Returns: final_dist: re-ranked distance, numpy array, shape [num_query, num_gallery] """ from __future__ import absolute_import from __future__ import print_function from __future__ import division __all__ = ['re_ranking'] import numpy as np def re_ranking(q_g_dist, q_q_dist, g_g_dist, k1=20, k2=6, lambda_value=0.3): # The following naming, e.g. gallery_num, is different from outer scope. # Don't care about it. original_dist = np.concatenate( [np.concatenate([q_q_dist, q_g_dist], axis=1), np.concatenate([q_g_dist.T, g_g_dist], axis=1)], axis=0) original_dist = np.power(original_dist, 2).astype(np.float32) original_dist = np.transpose(1. * original_dist/np.max(original_dist,axis = 0)) V = np.zeros_like(original_dist).astype(np.float32) initial_rank = np.argsort(original_dist).astype(np.int32) query_num = q_g_dist.shape[0] gallery_num = q_g_dist.shape[0] + q_g_dist.shape[1] all_num = gallery_num for i in range(all_num): # k-reciprocal neighbors forward_k_neigh_index = initial_rank[i,:k1+1] backward_k_neigh_index = initial_rank[forward_k_neigh_index,:k1+1] fi = np.where(backward_k_neigh_index==i)[0] k_reciprocal_index = forward_k_neigh_index[fi] k_reciprocal_expansion_index = k_reciprocal_index for j in range(len(k_reciprocal_index)): candidate = k_reciprocal_index[j] candidate_forward_k_neigh_index = initial_rank[candidate,:int(np.around(k1/2.))+1] candidate_backward_k_neigh_index = initial_rank[candidate_forward_k_neigh_index,:int(np.around(k1/2.))+1] fi_candidate = np.where(candidate_backward_k_neigh_index == candidate)[0] candidate_k_reciprocal_index = candidate_forward_k_neigh_index[fi_candidate] if len(np.intersect1d(candidate_k_reciprocal_index,k_reciprocal_index))> 2./3*len(candidate_k_reciprocal_index): k_reciprocal_expansion_index = np.append(k_reciprocal_expansion_index,candidate_k_reciprocal_index) k_reciprocal_expansion_index = np.unique(k_reciprocal_expansion_index) weight = np.exp(-original_dist[i,k_reciprocal_expansion_index]) V[i,k_reciprocal_expansion_index] = 1.*weight/np.sum(weight) original_dist = original_dist[:query_num,] if k2 != 1: V_qe = np.zeros_like(V,dtype=np.float32) for i in range(all_num): V_qe[i,:] = np.mean(V[initial_rank[i,:k2],:],axis=0) V = V_qe del V_qe del initial_rank invIndex = [] for i in range(gallery_num): invIndex.append(np.where(V[:,i] != 0)[0]) jaccard_dist = np.zeros_like(original_dist,dtype = np.float32) for i in range(query_num): temp_min = np.zeros(shape=[1,gallery_num],dtype=np.float32) indNonZero = np.where(V[i,:] != 0)[0] indImages = [] indImages = [invIndex[ind] for ind in indNonZero] for j in range(len(indNonZero)): temp_min[0,indImages[j]] = temp_min[0,indImages[j]]+ np.minimum(V[i,indNonZero[j]],V[indImages[j],indNonZero[j]]) jaccard_dist[i] = 1-temp_min/(2.-temp_min) final_dist = jaccard_dist*(1-lambda_value) + original_dist*lambda_value del original_dist del V del jaccard_dist final_dist = final_dist[:query_num,query_num:] return final_dist ================================================ FILE: clustercontrast/utils/serialization.py ================================================ from __future__ import print_function, absolute_import import json import os.path as osp import shutil import torch from torch.nn import Parameter from .osutils import mkdir_if_missing def read_json(fpath): with open(fpath, 'r') as f: obj = json.load(f) return obj def write_json(obj, fpath): mkdir_if_missing(osp.dirname(fpath)) with open(fpath, 'w') as f: json.dump(obj, f, indent=4, separators=(',', ': ')) def save_checkpoint(state, is_best, fpath='checkpoint.pth.tar'): mkdir_if_missing(osp.dirname(fpath)) torch.save(state, fpath) if is_best: shutil.copy(fpath, osp.join(osp.dirname(fpath), 'model_best.pth.tar')) def load_checkpoint(fpath): if osp.isfile(fpath): # checkpoint = torch.load(fpath) checkpoint = torch.load(fpath, map_location=torch.device('cpu')) print("=> Loaded checkpoint '{}'".format(fpath)) return checkpoint else: raise ValueError("=> No checkpoint found at '{}'".format(fpath)) def copy_state_dict(state_dict, model, strip=None): tgt_state = model.state_dict() copied_names = set() for name, param in state_dict.items(): if strip is not None and name.startswith(strip): name = name[len(strip):] if name not in tgt_state: continue if isinstance(param, Parameter): param = param.data if param.size() != tgt_state[name].size(): print('mismatch:', name, param.size(), tgt_state[name].size()) continue tgt_state[name].copy_(param) copied_names.add(name) missing = set(tgt_state.keys()) - copied_names if len(missing) > 0: print("missing keys in state_dict:", missing) return model ================================================ FILE: examples/cluster_contrast_train_usl.py ================================================ # -*- coding: utf-8 -*- from __future__ import print_function, absolute_import import argparse import os.path as osp import random import numpy as np import sys import collections import time from datetime import timedelta from sklearn.cluster import DBSCAN import torch from torch import nn from torch.backends import cudnn from torch.utils.data import DataLoader import torch.nn.functional as F from clustercontrast import datasets from clustercontrast import models from clustercontrast.models.cm import ClusterMemory from clustercontrast.trainers import ClusterContrastTrainer from clustercontrast.evaluators import Evaluator, extract_features from clustercontrast.utils.data import IterLoader from clustercontrast.utils.data import transforms as T from clustercontrast.utils.data.preprocessor import Preprocessor from clustercontrast.utils.logging import Logger from clustercontrast.utils.serialization import load_checkpoint, save_checkpoint from clustercontrast.utils.faiss_rerank import compute_jaccard_distance from clustercontrast.utils.data.sampler import RandomMultipleGallerySampler, RandomMultipleGallerySamplerNoCam start_epoch = best_mAP = 0 def get_data(name, data_dir): root = osp.join(data_dir, name) dataset = datasets.create(name, root) return dataset def get_train_loader(args, dataset, height, width, batch_size, workers, num_instances, iters, trainset=None, no_cam=False): normalizer = T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]) train_transformer = T.Compose([ T.Resize((height, width), interpolation=3), T.RandomHorizontalFlip(p=0.5), T.Pad(10), T.RandomCrop((height, width)), T.ToTensor(), normalizer, T.RandomErasing(probability=0.5, mean=[0.485, 0.456, 0.406]) ]) train_set = sorted(dataset.train) if trainset is None else sorted(trainset) rmgs_flag = num_instances > 0 if rmgs_flag: if no_cam: sampler = RandomMultipleGallerySamplerNoCam(train_set, num_instances) else: sampler = RandomMultipleGallerySampler(train_set, num_instances) else: sampler = None train_loader = IterLoader( DataLoader(Preprocessor(train_set, root=dataset.images_dir, transform=train_transformer), batch_size=batch_size, num_workers=workers, sampler=sampler, shuffle=not rmgs_flag, pin_memory=True, drop_last=True), length=iters) return train_loader def get_test_loader(dataset, height, width, batch_size, workers, testset=None): normalizer = T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]) test_transformer = T.Compose([ T.Resize((height, width), interpolation=3), T.ToTensor(), normalizer ]) if testset is None: testset = list(set(dataset.query) | set(dataset.gallery)) test_loader = DataLoader( Preprocessor(testset, root=dataset.images_dir, transform=test_transformer), batch_size=batch_size, num_workers=workers, shuffle=False, pin_memory=True) return test_loader def create_model(args): model = models.create(args.arch, num_features=args.features, norm=True, dropout=args.dropout, num_classes=0, pooling_type=args.pooling_type) # use CUDA model.cuda() model = nn.DataParallel(model) return model def main(): args = parser.parse_args() if args.seed is not None: random.seed(args.seed) np.random.seed(args.seed) torch.manual_seed(args.seed) cudnn.deterministic = True main_worker(args) def main_worker(args): global start_epoch, best_mAP start_time = time.monotonic() cudnn.benchmark = True sys.stdout = Logger(osp.join(args.logs_dir, 'log.txt')) print("==========\nArgs:{}\n==========".format(args)) # Create datasets iters = args.iters if (args.iters > 0) else None print("==> Load unlabeled dataset") dataset = get_data(args.dataset, args.data_dir) test_loader = get_test_loader(dataset, args.height, args.width, args.batch_size, args.workers) # Create model model = create_model(args) # Evaluator evaluator = Evaluator(model) # Optimizer params = [{"params": [value]} for _, value in model.named_parameters() if value.requires_grad] optimizer = torch.optim.Adam(params, lr=args.lr, weight_decay=args.weight_decay) lr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=args.step_size, gamma=0.1) # Trainer trainer = ClusterContrastTrainer(model) for epoch in range(args.epochs): with torch.no_grad(): print('==> Create pseudo labels for unlabeled data') cluster_loader = get_test_loader(dataset, args.height, args.width, args.batch_size, args.workers, testset=sorted(dataset.train)) features, _ = extract_features(model, cluster_loader, print_freq=50) features = torch.cat([features[f].unsqueeze(0) for f, _, _ in sorted(dataset.train)], 0) rerank_dist = compute_jaccard_distance(features, k1=args.k1, k2=args.k2) if epoch == 0: # DBSCAN cluster eps = args.eps print('Clustering criterion: eps: {:.3f}'.format(eps)) cluster = DBSCAN(eps=eps, min_samples=4, metric='precomputed', n_jobs=-1) # select & cluster images as training set of this epochs pseudo_labels = cluster.fit_predict(rerank_dist) num_cluster = len(set(pseudo_labels)) - (1 if -1 in pseudo_labels else 0) # print("epoch: {} \n pseudo_labels: {}".format(epoch, pseudo_labels.tolist()[:100])) # generate new dataset and calculate cluster centers @torch.no_grad() def generate_cluster_features(labels, features): centers = collections.defaultdict(list) for i, label in enumerate(labels): if label == -1: continue centers[labels[i]].append(features[i]) centers = [ torch.stack(centers[idx], dim=0).mean(0) for idx in sorted(centers.keys()) ] centers = torch.stack(centers, dim=0) return centers cluster_features = generate_cluster_features(pseudo_labels, features) del cluster_loader, features # Create hybrid memory memory = ClusterMemory(model.module.num_features, num_cluster, temp=args.temp, momentum=args.momentum, use_hard=args.use_hard).cuda() memory.features = F.normalize(cluster_features, dim=1).cuda() trainer.memory = memory pseudo_labeled_dataset = [] for i, ((fname, _, cid), label) in enumerate(zip(sorted(dataset.train), pseudo_labels)): if label != -1: pseudo_labeled_dataset.append((fname, label.item(), cid)) print('==> Statistics for epoch {}: {} clusters'.format(epoch, num_cluster)) train_loader = get_train_loader(args, dataset, args.height, args.width, args.batch_size, args.workers, args.num_instances, iters, trainset=pseudo_labeled_dataset, no_cam=args.no_cam) train_loader.new_epoch() trainer.train(epoch, train_loader, optimizer, print_freq=args.print_freq, train_iters=len(train_loader)) if (epoch + 1) % args.eval_step == 0 or (epoch == args.epochs - 1): mAP = evaluator.evaluate(test_loader, dataset.query, dataset.gallery, cmc_flag=False) is_best = (mAP > best_mAP) best_mAP = max(mAP, best_mAP) save_checkpoint({ 'state_dict': model.state_dict(), 'epoch': epoch + 1, 'best_mAP': best_mAP, }, is_best, fpath=osp.join(args.logs_dir, 'checkpoint.pth.tar')) print('\n * Finished epoch {:3d} model mAP: {:5.1%} best: {:5.1%}{}\n'. format(epoch, mAP, best_mAP, ' *' if is_best else '')) lr_scheduler.step() print('==> Test with the best model:') checkpoint = load_checkpoint(osp.join(args.logs_dir, 'model_best.pth.tar')) model.load_state_dict(checkpoint['state_dict']) evaluator.evaluate(test_loader, dataset.query, dataset.gallery, cmc_flag=True) end_time = time.monotonic() print('Total running time: ', timedelta(seconds=end_time - start_time)) if __name__ == '__main__': parser = argparse.ArgumentParser(description="Self-paced contrastive learning on unsupervised re-ID") # data parser.add_argument('-d', '--dataset', type=str, default='dukemtmcreid', choices=datasets.names()) parser.add_argument('-b', '--batch-size', type=int, default=2) parser.add_argument('-j', '--workers', type=int, default=4) parser.add_argument('--height', type=int, default=256, help="input height") parser.add_argument('--width', type=int, default=128, help="input width") parser.add_argument('--num-instances', type=int, default=4, help="each minibatch consist of " "(batch_size // num_instances) identities, and " "each identity has num_instances instances, " "default: 0 (NOT USE)") # cluster parser.add_argument('--eps', type=float, default=0.6, help="max neighbor distance for DBSCAN") parser.add_argument('--eps-gap', type=float, default=0.02, help="multi-scale criterion for measuring cluster reliability") parser.add_argument('--k1', type=int, default=30, help="hyperparameter for jaccard distance") parser.add_argument('--k2', type=int, default=6, help="hyperparameter for jaccard distance") # model parser.add_argument('-a', '--arch', type=str, default='resnet50', choices=models.names()) parser.add_argument('--features', type=int, default=0) parser.add_argument('--dropout', type=float, default=0) parser.add_argument('--momentum', type=float, default=0.2, help="update momentum for the hybrid memory") # optimizer parser.add_argument('--lr', type=float, default=0.00035, help="learning rate") parser.add_argument('--weight-decay', type=float, default=5e-4) parser.add_argument('--epochs', type=int, default=50) parser.add_argument('--iters', type=int, default=400) parser.add_argument('--step-size', type=int, default=20) # training configs parser.add_argument('--seed', type=int, default=1) parser.add_argument('--print-freq', type=int, default=10) parser.add_argument('--eval-step', type=int, default=10) parser.add_argument('--temp', type=float, default=0.05, help="temperature for scaling contrastive loss") # path working_dir = osp.dirname(osp.abspath(__file__)) parser.add_argument('--data-dir', type=str, metavar='PATH', default=osp.join(working_dir, 'data')) parser.add_argument('--logs-dir', type=str, metavar='PATH', default=osp.join(working_dir, 'logs')) parser.add_argument('--pooling-type', type=str, default='gem') parser.add_argument('--use-hard', action="store_true") parser.add_argument('--no-cam', action="store_true") main() ================================================ FILE: examples/cluster_contrast_train_usl_infomap.py ================================================ # -*- coding: utf-8 -*- from __future__ import print_function, absolute_import import argparse import os.path as osp import random import numpy as np import sys import collections import time from datetime import timedelta import torch from torch import nn from torch.backends import cudnn from torch.utils.data import DataLoader import torch.nn.functional as F from clustercontrast import datasets from clustercontrast import models from clustercontrast.models.cm import ClusterMemory from clustercontrast.trainers import ClusterContrastTrainer from clustercontrast.evaluators import Evaluator, extract_features from clustercontrast.utils.data import IterLoader from clustercontrast.utils.data import transforms as T from clustercontrast.utils.data.sampler import RandomMultipleGallerySampler, RandomMultipleGallerySamplerNoCam from clustercontrast.utils.data.preprocessor import Preprocessor from clustercontrast.utils.logging import Logger from clustercontrast.utils.serialization import load_checkpoint, save_checkpoint from clustercontrast.utils.infomap_cluster import get_dist_nbr, cluster_by_infomap start_epoch = best_mAP = 0 def str2bool(v): if isinstance(v, bool): return v if v.lower() in ('yes', 'true', 't', 'y', '1'): return True elif v.lower() in ('no', 'false', 'f', 'n', '0'): return False else: raise argparse.ArgumentTypeError('Boolean value expected.') def get_data(name, data_dir): root = osp.join(data_dir, name) dataset = datasets.create(name, root) return dataset def get_train_loader(args, dataset, height, width, batch_size, workers, num_instances, iters, trainset=None, no_cam=False): normalizer = T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]) train_transformer = T.Compose([ T.Resize((height, width), interpolation=3), T.RandomHorizontalFlip(p=0.5), T.Pad(10), T.RandomCrop((height, width)), T.ToTensor(), normalizer, T.RandomErasing(probability=0.5, mean=[0.485, 0.456, 0.406]) ]) train_set = sorted(dataset.train) if trainset is None else sorted(trainset) rmgs_flag = num_instances > 0 if rmgs_flag: if no_cam: sampler = RandomMultipleGallerySamplerNoCam(train_set, num_instances) else: sampler = RandomMultipleGallerySampler(train_set, num_instances) else: sampler = None train_loader = IterLoader( DataLoader(Preprocessor(train_set, root=dataset.images_dir, transform=train_transformer), batch_size=batch_size, num_workers=workers, sampler=sampler, shuffle=not rmgs_flag, pin_memory=True, drop_last=True), length=iters) return train_loader def get_test_loader(dataset, height, width, batch_size, workers, testset=None): normalizer = T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]) test_transformer = T.Compose([ T.Resize((height, width), interpolation=3), T.ToTensor(), normalizer ]) if testset is None: testset = list(set(dataset.query) | set(dataset.gallery)) test_loader = DataLoader( Preprocessor(testset, root=dataset.images_dir, transform=test_transformer), batch_size=batch_size, num_workers=workers, shuffle=False, pin_memory=True) return test_loader def create_model(args): model = models.create(args.arch, num_features=args.features, norm=True, dropout=args.dropout, num_classes=0, pooling_type=args.pooling_type) # use CUDA model.cuda() model = nn.DataParallel(model) return model def main(): args = parser.parse_args() if args.seed is not None: random.seed(args.seed) np.random.seed(args.seed) torch.manual_seed(args.seed) cudnn.deterministic = True main_worker(args) def main_worker(args): global start_epoch, best_mAP start_time = time.monotonic() cudnn.benchmark = True sys.stdout = Logger(osp.join(args.logs_dir, 'log.txt')) print("==========\nArgs:{}\n==========".format(args)) # Create datasets iters = args.iters if (args.iters > 0) else None print("==> Load unlabeled dataset") dataset = get_data(args.dataset, args.data_dir) test_loader = get_test_loader(dataset, args.height, args.width, args.batch_size, args.workers) # Create model model = create_model(args) # Evaluator evaluator = Evaluator(model) # Optimizer params = [{"params": [value]} for _, value in model.named_parameters() if value.requires_grad] optimizer = torch.optim.Adam(params, lr=args.lr, weight_decay=args.weight_decay) lr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=args.step_size, gamma=0.1) # Trainer trainer = ClusterContrastTrainer(model) for epoch in range(args.epochs): with torch.no_grad(): print('==> Create pseudo labels for unlabeled data') cluster_loader = get_test_loader(dataset, args.height, args.width, args.batch_size, args.workers, testset=sorted(dataset.train)) features, _ = extract_features(model, cluster_loader, print_freq=50) features = torch.cat([features[f].unsqueeze(0) for f, _, _ in sorted(dataset.train)], 0) features_array = F.normalize(features, dim=1).cpu().numpy() feat_dists, feat_nbrs = get_dist_nbr(features=features_array, k=args.k1, knn_method='faiss-gpu') del features_array s = time.time() pseudo_labels = cluster_by_infomap(feat_nbrs, feat_dists, min_sim=args.eps, cluster_num=args.k2) pseudo_labels = pseudo_labels.astype(np.intp) print('cluster cost time: {}'.format(time.time() - s)) num_cluster = len(set(pseudo_labels)) - (1 if -1 in pseudo_labels else 0) # generate new dataset and calculate cluster centers @torch.no_grad() def generate_cluster_features(labels, features): centers = collections.defaultdict(list) for i, label in enumerate(labels): if label == -1: continue centers[labels[i]].append(features[i]) centers = [ torch.stack(centers[idx], dim=0).mean(0) for idx in sorted(centers.keys()) ] centers = torch.stack(centers, dim=0) return centers cluster_features = generate_cluster_features(pseudo_labels, features) del cluster_loader, features # Create hybrid memory memory = ClusterMemory(model.module.num_features, num_cluster, temp=args.temp, momentum=args.momentum, use_hard=args.use_hard).cuda() memory.features = F.normalize(cluster_features, dim=1).cuda() trainer.memory = memory pseudo_labeled_dataset = [] for i, ((fname, _, cid), label) in enumerate(zip(sorted(dataset.train), pseudo_labels)): if label != -1: pseudo_labeled_dataset.append((fname, label.item(), cid)) print('==> Statistics for epoch {}: {} clusters'.format(epoch, num_cluster)) train_loader = get_train_loader(args, dataset, args.height, args.width, args.batch_size, args.workers, args.num_instances, iters, trainset=pseudo_labeled_dataset, no_cam=args.no_cam) train_loader.new_epoch() trainer.train(epoch, train_loader, optimizer, print_freq=args.print_freq, train_iters=len(train_loader)) if (epoch + 1) % args.eval_step == 0 or (epoch == args.epochs - 1): mAP = evaluator.evaluate(test_loader, dataset.query, dataset.gallery, cmc_flag=False) is_best = (mAP > best_mAP) best_mAP = max(mAP, best_mAP) save_checkpoint({ 'state_dict': model.state_dict(), 'epoch': epoch + 1, 'best_mAP': best_mAP, }, is_best, fpath=osp.join(args.logs_dir, 'checkpoint.pth.tar')) print('\n * Finished epoch {:3d} model mAP: {:5.1%} best: {:5.1%}{}\n'. format(epoch, mAP, best_mAP, ' *' if is_best else '')) lr_scheduler.step() print('==> Test with the best model:') checkpoint = load_checkpoint(osp.join(args.logs_dir, 'model_best.pth.tar')) model.load_state_dict(checkpoint['state_dict']) evaluator.evaluate(test_loader, dataset.query, dataset.gallery, cmc_flag=True) end_time = time.monotonic() print('Total running time: ', timedelta(seconds=end_time - start_time)) if __name__ == '__main__': parser = argparse.ArgumentParser(description="Self-paced contrastive learning on unsupervised re-ID") # data parser.add_argument('-d', '--dataset', type=str, default='dukemtmcreid', choices=datasets.names()) parser.add_argument('-b', '--batch-size', type=int, default=2) parser.add_argument('-j', '--workers', type=int, default=4) parser.add_argument('--height', type=int, default=256, help="input height") parser.add_argument('--width', type=int, default=128, help="input width") parser.add_argument('--num-instances', type=int, default=4, help="each minibatch consist of " "(batch_size // num_instances) identities, and " "each identity has num_instances instances, " "default: 0 (NOT USE)") # cluster parser.add_argument('--eps', type=float, default=0.5, help="max neighbor distance for DBSCAN") parser.add_argument('--eps-gap', type=float, default=0.02, help="multi-scale criterion for measuring cluster reliability") parser.add_argument('--k1', type=int, default=15, help="hyperparameter for KNN") parser.add_argument('--k2', type=int, default=4, help="hyperparameter for outline") # model parser.add_argument('-a', '--arch', type=str, default='resnet50', choices=models.names()) parser.add_argument('--features', type=int, default=0) parser.add_argument('--dropout', type=float, default=0) parser.add_argument('--momentum', type=float, default=0.2, help="update momentum for the hybrid memory") # optimizer parser.add_argument('--lr', type=float, default=0.00035, help="learning rate") parser.add_argument('--weight-decay', type=float, default=5e-4) parser.add_argument('--epochs', type=int, default=50) parser.add_argument('--iters', type=int, default=400) parser.add_argument('--step-size', type=int, default=20) # training configs parser.add_argument('--seed', type=int, default=1) parser.add_argument('--print-freq', type=int, default=10) parser.add_argument('--eval-step', type=int, default=10) parser.add_argument('--temp', type=float, default=0.05, help="temperature for scaling contrastive loss") # path working_dir = osp.dirname(osp.abspath(__file__)) parser.add_argument('--data-dir', type=str, metavar='PATH', default=osp.join(working_dir, 'data')) parser.add_argument('--logs-dir', type=str, metavar='PATH', default=osp.join(working_dir, 'logs')) parser.add_argument('--pooling-type', type=str, default='gem') parser.add_argument('--use-hard', action="store_true") parser.add_argument('--no-cam', action="store_true") main() ================================================ FILE: examples/logs/log.txt ================================================ ========== Args:Namespace(arch='resnet_ibn50a', batch_size=256, data_dir='/data0/developer/cluster-contrast/examples/data', dataset='market1501', dropout=0, epochs=50, eps=0.4, eps_gap=0.02, eval_step=10, features=0, height=256, iters=400, k1=30, k2=6, logs_dir='/data0/developer/cluster-contrast/examples/logs/gem-hard', lr=0.00035, momentum=0.1, num_instances=16, pooling_type='gem', print_freq=10, seed=1, step_size=20, temp=0.05, use_hard=True, weight_decay=0.0005, width=128, workers=4) ========== ==> Load unlabeled dataset => Market1501 loaded Dataset statistics: ---------------------------------------- subset | # ids | # images | # cameras ---------------------------------------- train | 751 | 12936 | 6 query | 750 | 3368 | 6 gallery | 751 | 15913 | 6 ---------------------------------------- pooling_type: gem ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.128 (0.379) Data 0.000 (0.019) Computing jaccard distance... Jaccard distance computing time cost: 23.047871351242065 Clustering criterion: eps: 0.400 ==> Statistics for epoch 0: 80 clusters Epoch: [0][10/400] Time 0.370 (0.775) Data 0.000 (0.211) Loss 4.447 (3.906) Epoch: [0][20/400] Time 0.362 (0.733) Data 0.000 (0.271) Loss 3.082 (3.512) Epoch: [0][30/400] Time 0.361 (0.722) Data 0.000 (0.293) Loss 3.117 (3.283) Epoch: [0][40/400] Time 0.361 (0.714) Data 0.000 (0.301) Loss 3.093 (3.113) Epoch: [0][50/400] Time 0.362 (0.712) Data 0.000 (0.309) Loss 2.184 (2.973) Epoch: [0][60/400] Time 0.361 (0.711) Data 0.000 (0.315) Loss 2.049 (2.829) Epoch: [0][70/400] Time 0.354 (0.711) Data 0.000 (0.319) Loss 2.118 (2.714) Epoch: [0][80/400] Time 0.362 (0.709) Data 0.000 (0.321) Loss 2.413 (2.657) Epoch: [0][90/400] Time 0.360 (0.707) Data 0.000 (0.321) Loss 2.092 (2.576) Epoch: [0][100/400] Time 0.362 (0.706) Data 0.000 (0.322) Loss 1.619 (2.488) Epoch: [0][110/400] Time 0.374 (0.704) Data 0.000 (0.322) Loss 2.173 (2.423) Epoch: [0][120/400] Time 0.351 (0.703) Data 0.000 (0.323) Loss 1.402 (2.359) Epoch: [0][130/400] Time 0.369 (0.703) Data 0.000 (0.324) Loss 2.615 (2.307) Epoch: [0][140/400] Time 0.368 (0.703) Data 0.000 (0.325) Loss 1.323 (2.262) Epoch: [0][150/400] Time 0.369 (0.703) Data 0.000 (0.325) Loss 1.811 (2.209) Epoch: [0][160/400] Time 0.369 (0.704) Data 0.000 (0.326) Loss 1.369 (2.153) Epoch: [0][170/400] Time 0.368 (0.704) Data 0.000 (0.327) Loss 1.734 (2.109) Epoch: [0][180/400] Time 0.369 (0.704) Data 0.000 (0.327) Loss 0.889 (2.057) Epoch: [0][190/400] Time 0.366 (0.704) Data 0.000 (0.328) Loss 1.458 (2.011) Epoch: [0][200/400] Time 0.361 (0.705) Data 0.000 (0.329) Loss 1.066 (1.967) Epoch: [0][210/400] Time 0.363 (0.705) Data 0.000 (0.329) Loss 1.535 (1.928) Epoch: [0][220/400] Time 0.362 (0.705) Data 0.000 (0.330) Loss 1.193 (1.898) Epoch: [0][230/400] Time 0.362 (0.705) Data 0.000 (0.330) Loss 1.210 (1.862) Epoch: [0][240/400] Time 0.360 (0.704) Data 0.000 (0.330) Loss 1.218 (1.829) Epoch: [0][250/400] Time 0.363 (0.704) Data 0.000 (0.330) Loss 0.901 (1.800) Epoch: [0][260/400] Time 0.363 (0.704) Data 0.000 (0.330) Loss 1.281 (1.770) Epoch: [0][270/400] Time 0.360 (0.704) Data 0.000 (0.331) Loss 1.332 (1.740) Epoch: [0][280/400] Time 0.362 (0.704) Data 0.000 (0.331) Loss 1.559 (1.710) Epoch: [0][290/400] Time 0.362 (0.704) Data 0.000 (0.331) Loss 0.889 (1.682) Epoch: [0][300/400] Time 0.362 (0.704) Data 0.000 (0.331) Loss 1.134 (1.657) Epoch: [0][310/400] Time 0.363 (0.703) Data 0.000 (0.331) Loss 1.001 (1.630) Epoch: [0][320/400] Time 0.372 (0.703) Data 0.000 (0.331) Loss 0.619 (1.602) Epoch: [0][330/400] Time 0.361 (0.703) Data 0.000 (0.332) Loss 0.942 (1.576) Epoch: [0][340/400] Time 0.363 (0.703) Data 0.000 (0.332) Loss 0.700 (1.552) Epoch: [0][350/400] Time 0.363 (0.703) Data 0.000 (0.332) Loss 0.434 (1.530) Epoch: [0][360/400] Time 0.362 (0.703) Data 0.000 (0.332) Loss 0.739 (1.507) Epoch: [0][370/400] Time 0.364 (0.703) Data 0.000 (0.332) Loss 0.631 (1.485) Epoch: [0][380/400] Time 0.365 (0.702) Data 0.000 (0.332) Loss 0.529 (1.465) Epoch: [0][390/400] Time 0.364 (0.702) Data 0.000 (0.332) Loss 0.699 (1.445) Epoch: [0][400/400] Time 0.363 (0.702) Data 0.000 (0.332) Loss 0.945 (1.426) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.130 (0.158) Data 0.000 (0.027) Computing jaccard distance... Jaccard distance computing time cost: 20.340964555740356 ==> Statistics for epoch 1: 286 clusters Epoch: [1][10/400] Time 0.352 (0.418) Data 0.001 (0.053) Loss 0.530 (0.732) Epoch: [1][20/400] Time 0.360 (0.480) Data 0.000 (0.117) Loss 3.473 (1.095) Epoch: [1][30/400] Time 0.361 (0.441) Data 0.000 (0.078) Loss 3.468 (1.908) Epoch: [1][40/400] Time 0.363 (0.461) Data 0.000 (0.099) Loss 3.717 (2.348) Epoch: [1][50/400] Time 0.366 (0.442) Data 0.000 (0.079) Loss 3.763 (2.694) Epoch: [1][60/400] Time 0.363 (0.456) Data 0.000 (0.094) Loss 3.590 (2.891) Epoch: [1][70/400] Time 0.345 (0.468) Data 0.000 (0.106) Loss 3.950 (3.029) Epoch: [1][80/400] Time 0.362 (0.455) Data 0.000 (0.093) Loss 3.343 (3.112) Epoch: [1][90/400] Time 0.362 (0.465) Data 0.000 (0.102) Loss 3.418 (3.184) Epoch: [1][100/400] Time 0.363 (0.454) Data 0.000 (0.092) Loss 3.445 (3.229) Epoch: [1][110/400] Time 0.362 (0.462) Data 0.001 (0.099) Loss 3.722 (3.268) Epoch: [1][120/400] Time 2.066 (0.468) Data 1.640 (0.105) Loss 2.855 (3.295) Epoch: [1][130/400] Time 0.362 (0.460) Data 0.000 (0.097) Loss 3.671 (3.306) Epoch: [1][140/400] Time 0.363 (0.465) Data 0.000 (0.102) Loss 2.616 (3.296) Epoch: [1][150/400] Time 0.363 (0.458) Data 0.000 (0.095) Loss 3.351 (3.290) Epoch: [1][160/400] Time 0.363 (0.463) Data 0.000 (0.100) Loss 3.071 (3.285) Epoch: [1][170/400] Time 0.363 (0.458) Data 0.000 (0.094) Loss 3.275 (3.282) Epoch: [1][180/400] Time 0.364 (0.461) Data 0.000 (0.098) Loss 3.301 (3.270) Epoch: [1][190/400] Time 0.381 (0.466) Data 0.001 (0.102) Loss 2.709 (3.262) Epoch: [1][200/400] Time 0.365 (0.460) Data 0.000 (0.097) Loss 3.041 (3.249) Epoch: [1][210/400] Time 0.370 (0.464) Data 0.001 (0.100) Loss 3.036 (3.237) Epoch: [1][220/400] Time 0.369 (0.460) Data 0.000 (0.096) Loss 3.528 (3.245) Epoch: [1][230/400] Time 0.362 (0.463) Data 0.001 (0.099) Loss 2.933 (3.224) Epoch: [1][240/400] Time 0.356 (0.466) Data 0.001 (0.101) Loss 2.484 (3.218) Epoch: [1][250/400] Time 0.362 (0.462) Data 0.000 (0.097) Loss 2.455 (3.207) Epoch: [1][260/400] Time 0.414 (0.464) Data 0.001 (0.100) Loss 2.943 (3.191) Epoch: [1][270/400] Time 0.369 (0.461) Data 0.000 (0.096) Loss 2.580 (3.175) Epoch: [1][280/400] Time 0.352 (0.464) Data 0.000 (0.099) Loss 2.666 (3.156) Epoch: [1][290/400] Time 1.998 (0.466) Data 1.634 (0.101) Loss 2.325 (3.150) Epoch: [1][300/400] Time 0.363 (0.463) Data 0.000 (0.098) Loss 3.091 (3.134) Epoch: [1][310/400] Time 0.361 (0.465) Data 0.000 (0.100) Loss 1.971 (3.123) Epoch: [1][320/400] Time 0.364 (0.462) Data 0.000 (0.097) Loss 3.067 (3.111) Epoch: [1][330/400] Time 0.362 (0.464) Data 0.001 (0.099) Loss 2.665 (3.097) Epoch: [1][340/400] Time 0.363 (0.461) Data 0.000 (0.096) Loss 2.564 (3.083) Epoch: [1][350/400] Time 0.363 (0.463) Data 0.000 (0.098) Loss 2.617 (3.072) Epoch: [1][360/400] Time 0.361 (0.465) Data 0.001 (0.100) Loss 2.341 (3.057) Epoch: [1][370/400] Time 0.365 (0.463) Data 0.000 (0.098) Loss 2.255 (3.041) Epoch: [1][380/400] Time 0.364 (0.464) Data 0.001 (0.099) Loss 2.267 (3.029) Epoch: [1][390/400] Time 0.366 (0.462) Data 0.000 (0.097) Loss 2.701 (3.015) Epoch: [1][400/400] Time 0.376 (0.464) Data 0.001 (0.099) Loss 1.968 (3.001) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.130 (0.160) Data 0.000 (0.031) Computing jaccard distance... Jaccard distance computing time cost: 21.161827564239502 ==> Statistics for epoch 2: 561 clusters Epoch: [2][10/400] Time 0.362 (0.422) Data 0.001 (0.054) Loss 0.609 (0.845) Epoch: [2][20/400] Time 0.361 (0.392) Data 0.001 (0.027) Loss 0.627 (0.767) Epoch: [2][30/400] Time 0.361 (0.382) Data 0.000 (0.018) Loss 0.408 (0.696) Epoch: [2][40/400] Time 0.363 (0.420) Data 0.001 (0.056) Loss 4.156 (1.033) Epoch: [2][50/400] Time 0.362 (0.408) Data 0.001 (0.045) Loss 3.165 (1.537) Epoch: [2][60/400] Time 0.386 (0.401) Data 0.000 (0.038) Loss 4.043 (1.907) Epoch: [2][70/400] Time 0.363 (0.396) Data 0.000 (0.032) Loss 4.042 (2.221) Epoch: [2][80/400] Time 0.364 (0.415) Data 0.001 (0.051) Loss 3.583 (2.379) Epoch: [2][90/400] Time 0.364 (0.409) Data 0.000 (0.046) Loss 3.396 (2.509) Epoch: [2][100/400] Time 0.365 (0.405) Data 0.000 (0.041) Loss 3.422 (2.619) Epoch: [2][110/400] Time 0.363 (0.418) Data 0.001 (0.054) Loss 4.696 (2.711) Epoch: [2][120/400] Time 0.363 (0.414) Data 0.001 (0.050) Loss 3.733 (2.794) Epoch: [2][130/400] Time 0.391 (0.410) Data 0.001 (0.046) Loss 3.354 (2.860) Epoch: [2][140/400] Time 0.365 (0.407) Data 0.000 (0.043) Loss 3.528 (2.919) Epoch: [2][150/400] Time 0.362 (0.416) Data 0.001 (0.051) Loss 4.046 (2.952) Epoch: [2][160/400] Time 0.364 (0.413) Data 0.000 (0.048) Loss 3.927 (2.989) Epoch: [2][170/400] Time 0.364 (0.410) Data 0.000 (0.045) Loss 3.270 (3.019) Epoch: [2][180/400] Time 0.362 (0.416) Data 0.001 (0.052) Loss 3.555 (3.050) Epoch: [2][190/400] Time 0.364 (0.414) Data 0.001 (0.049) Loss 3.980 (3.071) Epoch: [2][200/400] Time 0.363 (0.411) Data 0.001 (0.047) Loss 3.768 (3.093) Epoch: [2][210/400] Time 0.363 (0.409) Data 0.000 (0.045) Loss 3.659 (3.104) Epoch: [2][220/400] Time 0.364 (0.415) Data 0.001 (0.050) Loss 2.920 (3.109) Epoch: [2][230/400] Time 0.365 (0.413) Data 0.001 (0.048) Loss 3.674 (3.115) Epoch: [2][240/400] Time 0.365 (0.411) Data 0.000 (0.046) Loss 2.956 (3.118) Epoch: [2][250/400] Time 0.363 (0.416) Data 0.001 (0.051) Loss 2.949 (3.129) Epoch: [2][260/400] Time 0.363 (0.414) Data 0.000 (0.049) Loss 3.126 (3.132) Epoch: [2][270/400] Time 0.365 (0.412) Data 0.001 (0.047) Loss 2.763 (3.149) Epoch: [2][280/400] Time 0.363 (0.410) Data 0.000 (0.046) Loss 2.874 (3.153) Epoch: [2][290/400] Time 0.350 (0.414) Data 0.001 (0.050) Loss 3.045 (3.151) Epoch: [2][300/400] Time 0.366 (0.412) Data 0.001 (0.048) Loss 2.985 (3.155) Epoch: [2][310/400] Time 0.365 (0.411) Data 0.000 (0.046) Loss 3.784 (3.154) Epoch: [2][320/400] Time 0.364 (0.415) Data 0.000 (0.050) Loss 2.958 (3.151) Epoch: [2][330/400] Time 0.366 (0.413) Data 0.001 (0.049) Loss 2.676 (3.160) Epoch: [2][340/400] Time 0.366 (0.412) Data 0.001 (0.047) Loss 3.050 (3.167) Epoch: [2][350/400] Time 0.363 (0.410) Data 0.000 (0.046) Loss 3.733 (3.167) Epoch: [2][360/400] Time 0.365 (0.414) Data 0.001 (0.049) Loss 3.433 (3.169) Epoch: [2][370/400] Time 0.366 (0.413) Data 0.000 (0.048) Loss 3.044 (3.165) Epoch: [2][380/400] Time 0.364 (0.412) Data 0.000 (0.047) Loss 3.781 (3.168) Epoch: [2][390/400] Time 0.364 (0.415) Data 0.000 (0.050) Loss 3.573 (3.170) Epoch: [2][400/400] Time 0.365 (0.414) Data 0.001 (0.049) Loss 3.600 (3.168) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.158) Data 0.000 (0.030) Computing jaccard distance... Jaccard distance computing time cost: 22.187981843948364 ==> Statistics for epoch 3: 669 clusters Epoch: [3][10/400] Time 0.363 (0.432) Data 0.001 (0.064) Loss 0.745 (0.655) Epoch: [3][20/400] Time 0.362 (0.396) Data 0.000 (0.032) Loss 0.646 (0.626) Epoch: [3][30/400] Time 0.363 (0.384) Data 0.001 (0.022) Loss 0.527 (0.619) Epoch: [3][40/400] Time 0.363 (0.379) Data 0.000 (0.016) Loss 0.470 (0.578) Epoch: [3][50/400] Time 0.361 (0.413) Data 0.000 (0.050) Loss 3.554 (1.092) Epoch: [3][60/400] Time 0.364 (0.405) Data 0.000 (0.042) Loss 3.596 (1.479) Epoch: [3][70/400] Time 0.360 (0.399) Data 0.000 (0.036) Loss 3.852 (1.760) Epoch: [3][80/400] Time 0.364 (0.394) Data 0.000 (0.031) Loss 3.392 (1.988) Epoch: [3][90/400] Time 0.359 (0.410) Data 0.000 (0.047) Loss 3.664 (2.139) Epoch: [3][100/400] Time 0.364 (0.406) Data 0.001 (0.043) Loss 3.258 (2.264) Epoch: [3][110/400] Time 0.351 (0.402) Data 0.000 (0.039) Loss 4.049 (2.381) Epoch: [3][120/400] Time 0.362 (0.398) Data 0.000 (0.036) Loss 3.549 (2.481) Epoch: [3][130/400] Time 0.356 (0.410) Data 0.001 (0.047) Loss 3.922 (2.562) Epoch: [3][140/400] Time 0.356 (0.406) Data 0.001 (0.044) Loss 3.350 (2.643) Epoch: [3][150/400] Time 0.358 (0.403) Data 0.001 (0.041) Loss 3.777 (2.697) Epoch: [3][160/400] Time 0.363 (0.400) Data 0.000 (0.038) Loss 3.202 (2.751) Epoch: [3][170/400] Time 0.363 (0.409) Data 0.000 (0.046) Loss 3.472 (2.796) Epoch: [3][180/400] Time 0.353 (0.406) Data 0.001 (0.044) Loss 4.451 (2.837) Epoch: [3][190/400] Time 0.355 (0.404) Data 0.001 (0.042) Loss 3.997 (2.881) Epoch: [3][200/400] Time 0.364 (0.402) Data 0.000 (0.040) Loss 3.018 (2.906) Epoch: [3][210/400] Time 0.358 (0.408) Data 0.001 (0.046) Loss 4.063 (2.924) Epoch: [3][220/400] Time 0.394 (0.406) Data 0.001 (0.044) Loss 3.438 (2.944) Epoch: [3][230/400] Time 0.365 (0.404) Data 0.000 (0.042) Loss 4.328 (2.966) Epoch: [3][240/400] Time 0.363 (0.403) Data 0.000 (0.040) Loss 3.005 (2.975) Epoch: [3][250/400] Time 0.357 (0.408) Data 0.001 (0.046) Loss 3.439 (2.991) Epoch: [3][260/400] Time 0.366 (0.406) Data 0.001 (0.044) Loss 3.608 (3.002) Epoch: [3][270/400] Time 0.366 (0.405) Data 0.000 (0.042) Loss 3.404 (3.014) Epoch: [3][280/400] Time 0.363 (0.403) Data 0.001 (0.041) Loss 3.531 (3.033) Epoch: [3][290/400] Time 0.355 (0.408) Data 0.000 (0.046) Loss 3.654 (3.046) Epoch: [3][300/400] Time 0.354 (0.407) Data 0.000 (0.044) Loss 3.869 (3.054) Epoch: [3][310/400] Time 0.353 (0.405) Data 0.001 (0.043) Loss 3.166 (3.064) Epoch: [3][320/400] Time 0.365 (0.404) Data 0.000 (0.042) Loss 3.749 (3.077) Epoch: [3][330/400] Time 0.349 (0.408) Data 0.001 (0.046) Loss 3.487 (3.084) Epoch: [3][340/400] Time 0.365 (0.407) Data 0.001 (0.045) Loss 3.068 (3.092) Epoch: [3][350/400] Time 0.366 (0.406) Data 0.001 (0.043) Loss 2.509 (3.098) Epoch: [3][360/400] Time 0.366 (0.404) Data 0.000 (0.042) Loss 3.394 (3.107) Epoch: [3][370/400] Time 2.231 (0.408) Data 1.838 (0.046) Loss 2.484 (3.108) Epoch: [3][380/400] Time 0.404 (0.407) Data 0.001 (0.045) Loss 3.090 (3.111) Epoch: [3][390/400] Time 0.363 (0.406) Data 0.000 (0.044) Loss 2.966 (3.116) Epoch: [3][400/400] Time 0.355 (0.405) Data 0.001 (0.043) Loss 2.738 (3.121) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.159) Data 0.000 (0.029) Computing jaccard distance... Jaccard distance computing time cost: 20.534826517105103 ==> Statistics for epoch 4: 718 clusters Epoch: [4][10/400] Time 0.363 (0.422) Data 0.001 (0.059) Loss 0.710 (0.738) Epoch: [4][20/400] Time 0.363 (0.390) Data 0.001 (0.030) Loss 0.527 (0.702) Epoch: [4][30/400] Time 0.364 (0.380) Data 0.000 (0.020) Loss 0.632 (0.662) Epoch: [4][40/400] Time 0.354 (0.375) Data 0.000 (0.015) Loss 0.507 (0.615) Epoch: [4][50/400] Time 0.361 (0.407) Data 0.001 (0.046) Loss 3.590 (0.947) Epoch: [4][60/400] Time 0.365 (0.400) Data 0.000 (0.038) Loss 3.781 (1.381) Epoch: [4][70/400] Time 0.380 (0.395) Data 0.001 (0.033) Loss 3.560 (1.677) Epoch: [4][80/400] Time 0.353 (0.391) Data 0.000 (0.029) Loss 3.385 (1.940) Epoch: [4][90/400] Time 0.336 (0.408) Data 0.001 (0.046) Loss 2.380 (2.110) Epoch: [4][100/400] Time 0.365 (0.403) Data 0.000 (0.041) Loss 3.410 (2.233) Epoch: [4][110/400] Time 0.356 (0.399) Data 0.001 (0.038) Loss 3.001 (2.355) Epoch: [4][120/400] Time 0.365 (0.396) Data 0.000 (0.034) Loss 3.813 (2.443) Epoch: [4][130/400] Time 0.364 (0.393) Data 0.000 (0.032) Loss 3.336 (2.522) Epoch: [4][140/400] Time 0.353 (0.404) Data 0.001 (0.043) Loss 3.169 (2.593) Epoch: [4][150/400] Time 0.364 (0.402) Data 0.001 (0.040) Loss 3.389 (2.639) Epoch: [4][160/400] Time 0.365 (0.399) Data 0.001 (0.037) Loss 3.830 (2.694) Epoch: [4][170/400] Time 0.364 (0.397) Data 0.000 (0.035) Loss 3.357 (2.738) Epoch: [4][180/400] Time 0.365 (0.405) Data 0.001 (0.043) Loss 2.959 (2.768) Epoch: [4][190/400] Time 0.367 (0.403) Data 0.001 (0.041) Loss 3.563 (2.808) Epoch: [4][200/400] Time 0.365 (0.401) Data 0.000 (0.039) Loss 3.389 (2.831) Epoch: [4][210/400] Time 0.365 (0.399) Data 0.001 (0.037) Loss 3.796 (2.872) Epoch: [4][220/400] Time 0.364 (0.397) Data 0.000 (0.036) Loss 3.907 (2.902) Epoch: [4][230/400] Time 0.360 (0.404) Data 0.001 (0.042) Loss 3.402 (2.915) Epoch: [4][240/400] Time 0.356 (0.402) Data 0.001 (0.040) Loss 3.342 (2.935) Epoch: [4][250/400] Time 0.366 (0.400) Data 0.001 (0.038) Loss 2.512 (2.949) Epoch: [4][260/400] Time 0.366 (0.399) Data 0.000 (0.037) Loss 3.143 (2.970) Epoch: [4][270/400] Time 0.364 (0.404) Data 0.000 (0.042) Loss 2.611 (2.978) Epoch: [4][280/400] Time 0.351 (0.402) Data 0.001 (0.040) Loss 3.454 (2.994) Epoch: [4][290/400] Time 0.364 (0.401) Data 0.001 (0.039) Loss 3.459 (3.013) Epoch: [4][300/400] Time 0.365 (0.400) Data 0.001 (0.038) Loss 4.128 (3.031) Epoch: [4][310/400] Time 0.348 (0.404) Data 0.001 (0.042) Loss 3.815 (3.045) Epoch: [4][320/400] Time 0.353 (0.403) Data 0.001 (0.041) Loss 2.844 (3.051) Epoch: [4][330/400] Time 0.364 (0.402) Data 0.000 (0.040) Loss 2.982 (3.059) Epoch: [4][340/400] Time 0.363 (0.400) Data 0.001 (0.039) Loss 2.965 (3.071) Epoch: [4][350/400] Time 0.364 (0.399) Data 0.000 (0.037) Loss 3.538 (3.081) Epoch: [4][360/400] Time 0.353 (0.403) Data 0.000 (0.041) Loss 2.585 (3.086) Epoch: [4][370/400] Time 0.352 (0.402) Data 0.000 (0.040) Loss 3.671 (3.091) Epoch: [4][380/400] Time 0.353 (0.401) Data 0.000 (0.039) Loss 3.471 (3.093) Epoch: [4][390/400] Time 0.354 (0.400) Data 0.000 (0.038) Loss 3.564 (3.103) Epoch: [4][400/400] Time 0.363 (0.403) Data 0.000 (0.042) Loss 3.386 (3.105) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.159) Data 0.000 (0.029) Computing jaccard distance... Jaccard distance computing time cost: 19.915706157684326 ==> Statistics for epoch 5: 738 clusters Epoch: [5][10/400] Time 0.363 (0.427) Data 0.001 (0.058) Loss 0.871 (0.671) Epoch: [5][20/400] Time 0.363 (0.395) Data 0.000 (0.029) Loss 0.594 (0.658) Epoch: [5][30/400] Time 0.361 (0.384) Data 0.001 (0.020) Loss 0.571 (0.646) Epoch: [5][40/400] Time 0.362 (0.379) Data 0.000 (0.015) Loss 0.434 (0.608) Epoch: [5][50/400] Time 0.363 (0.411) Data 0.001 (0.047) Loss 3.662 (0.826) Epoch: [5][60/400] Time 0.363 (0.404) Data 0.001 (0.039) Loss 3.928 (1.278) Epoch: [5][70/400] Time 0.363 (0.398) Data 0.001 (0.034) Loss 3.518 (1.610) Epoch: [5][80/400] Time 0.363 (0.394) Data 0.001 (0.030) Loss 4.034 (1.891) Epoch: [5][90/400] Time 0.360 (0.391) Data 0.000 (0.026) Loss 3.181 (2.090) Epoch: [5][100/400] Time 0.363 (0.406) Data 0.001 (0.042) Loss 3.906 (2.223) Epoch: [5][110/400] Time 0.363 (0.402) Data 0.001 (0.038) Loss 2.889 (2.327) Epoch: [5][120/400] Time 0.364 (0.399) Data 0.001 (0.035) Loss 3.901 (2.436) Epoch: [5][130/400] Time 0.372 (0.396) Data 0.001 (0.032) Loss 3.618 (2.512) Epoch: [5][140/400] Time 0.364 (0.406) Data 0.001 (0.042) Loss 3.443 (2.572) Epoch: [5][150/400] Time 0.362 (0.403) Data 0.001 (0.039) Loss 3.673 (2.634) Epoch: [5][160/400] Time 0.362 (0.401) Data 0.001 (0.037) Loss 3.939 (2.700) Epoch: [5][170/400] Time 0.363 (0.398) Data 0.001 (0.035) Loss 3.513 (2.750) Epoch: [5][180/400] Time 0.362 (0.396) Data 0.000 (0.033) Loss 3.440 (2.786) Epoch: [5][190/400] Time 0.364 (0.404) Data 0.001 (0.040) Loss 3.323 (2.821) Epoch: [5][200/400] Time 0.379 (0.402) Data 0.001 (0.038) Loss 3.773 (2.852) Epoch: [5][210/400] Time 0.364 (0.400) Data 0.001 (0.037) Loss 3.739 (2.879) Epoch: [5][220/400] Time 0.365 (0.399) Data 0.001 (0.035) Loss 2.810 (2.905) Epoch: [5][230/400] Time 0.363 (0.397) Data 0.000 (0.033) Loss 2.995 (2.926) Epoch: [5][240/400] Time 0.363 (0.403) Data 0.001 (0.039) Loss 3.483 (2.945) Epoch: [5][250/400] Time 0.364 (0.402) Data 0.001 (0.038) Loss 2.913 (2.960) Epoch: [5][260/400] Time 0.364 (0.400) Data 0.001 (0.036) Loss 3.559 (2.984) Epoch: [5][270/400] Time 0.363 (0.399) Data 0.000 (0.035) Loss 2.889 (2.996) Epoch: [5][280/400] Time 0.363 (0.404) Data 0.000 (0.040) Loss 3.172 (3.010) Epoch: [5][290/400] Time 0.397 (0.402) Data 0.001 (0.038) Loss 2.851 (3.031) Epoch: [5][300/400] Time 0.363 (0.401) Data 0.001 (0.037) Loss 3.221 (3.040) Epoch: [5][310/400] Time 0.404 (0.400) Data 0.001 (0.036) Loss 4.441 (3.056) Epoch: [5][320/400] Time 0.362 (0.399) Data 0.000 (0.035) Loss 3.435 (3.065) Epoch: [5][330/400] Time 0.363 (0.403) Data 0.000 (0.039) Loss 2.926 (3.070) Epoch: [5][340/400] Time 0.363 (0.402) Data 0.001 (0.038) Loss 3.199 (3.078) Epoch: [5][350/400] Time 0.363 (0.401) Data 0.000 (0.037) Loss 3.741 (3.090) Epoch: [5][360/400] Time 0.364 (0.400) Data 0.000 (0.036) Loss 3.576 (3.102) Epoch: [5][370/400] Time 0.354 (0.404) Data 0.000 (0.040) Loss 3.012 (3.114) Epoch: [5][380/400] Time 0.362 (0.403) Data 0.001 (0.039) Loss 3.448 (3.122) Epoch: [5][390/400] Time 0.362 (0.402) Data 0.001 (0.038) Loss 3.214 (3.128) Epoch: [5][400/400] Time 0.363 (0.401) Data 0.000 (0.037) Loss 2.759 (3.135) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.156) Data 0.000 (0.028) Computing jaccard distance... Jaccard distance computing time cost: 21.006665468215942 ==> Statistics for epoch 6: 774 clusters Epoch: [6][10/400] Time 0.361 (0.421) Data 0.000 (0.055) Loss 0.654 (0.630) Epoch: [6][20/400] Time 0.361 (0.391) Data 0.000 (0.028) Loss 0.532 (0.637) Epoch: [6][30/400] Time 0.349 (0.381) Data 0.001 (0.019) Loss 0.706 (0.609) Epoch: [6][40/400] Time 0.351 (0.376) Data 0.001 (0.014) Loss 0.388 (0.561) Epoch: [6][50/400] Time 0.351 (0.409) Data 0.001 (0.046) Loss 3.556 (0.656) Epoch: [6][60/400] Time 0.363 (0.401) Data 0.001 (0.039) Loss 3.634 (1.125) Epoch: [6][70/400] Time 0.363 (0.394) Data 0.001 (0.033) Loss 3.379 (1.449) Epoch: [6][80/400] Time 0.363 (0.390) Data 0.001 (0.029) Loss 4.524 (1.719) Epoch: [6][90/400] Time 0.361 (0.386) Data 0.000 (0.026) Loss 4.387 (1.916) Epoch: [6][100/400] Time 0.363 (0.401) Data 0.001 (0.040) Loss 3.475 (2.080) Epoch: [6][110/400] Time 0.362 (0.398) Data 0.001 (0.037) Loss 3.509 (2.204) Epoch: [6][120/400] Time 0.363 (0.395) Data 0.000 (0.034) Loss 4.077 (2.310) Epoch: [6][130/400] Time 0.363 (0.392) Data 0.001 (0.031) Loss 3.712 (2.401) Epoch: [6][140/400] Time 0.363 (0.390) Data 0.000 (0.029) Loss 3.296 (2.459) Epoch: [6][150/400] Time 0.363 (0.400) Data 0.001 (0.038) Loss 2.994 (2.518) Epoch: [6][160/400] Time 0.367 (0.397) Data 0.000 (0.036) Loss 3.683 (2.580) Epoch: [6][170/400] Time 0.363 (0.395) Data 0.001 (0.034) Loss 3.782 (2.641) Epoch: [6][180/400] Time 0.356 (0.393) Data 0.000 (0.032) Loss 3.207 (2.689) Epoch: [6][190/400] Time 0.362 (0.392) Data 0.000 (0.030) Loss 3.582 (2.743) Epoch: [6][200/400] Time 0.354 (0.399) Data 0.001 (0.038) Loss 3.799 (2.784) Epoch: [6][210/400] Time 0.356 (0.397) Data 0.001 (0.036) Loss 4.227 (2.824) Epoch: [6][220/400] Time 0.365 (0.396) Data 0.000 (0.034) Loss 3.427 (2.857) Epoch: [6][230/400] Time 0.365 (0.394) Data 0.001 (0.033) Loss 3.153 (2.878) Epoch: [6][240/400] Time 0.361 (0.393) Data 0.000 (0.032) Loss 2.961 (2.900) Epoch: [6][250/400] Time 0.365 (0.399) Data 0.001 (0.037) Loss 3.756 (2.919) Epoch: [6][260/400] Time 0.365 (0.398) Data 0.000 (0.036) Loss 3.699 (2.935) Epoch: [6][270/400] Time 0.363 (0.397) Data 0.001 (0.035) Loss 3.538 (2.951) Epoch: [6][280/400] Time 0.356 (0.395) Data 0.000 (0.033) Loss 3.931 (2.973) Epoch: [6][290/400] Time 0.397 (0.401) Data 0.000 (0.039) Loss 3.717 (2.991) Epoch: [6][300/400] Time 0.364 (0.399) Data 0.000 (0.037) Loss 3.070 (3.006) Epoch: [6][310/400] Time 0.364 (0.398) Data 0.001 (0.036) Loss 3.666 (3.022) Epoch: [6][320/400] Time 0.364 (0.397) Data 0.000 (0.035) Loss 3.174 (3.034) Epoch: [6][330/400] Time 0.365 (0.396) Data 0.000 (0.034) Loss 3.553 (3.049) Epoch: [6][340/400] Time 0.364 (0.401) Data 0.001 (0.038) Loss 3.590 (3.060) Epoch: [6][350/400] Time 0.360 (0.400) Data 0.001 (0.037) Loss 3.599 (3.066) Epoch: [6][360/400] Time 0.353 (0.399) Data 0.001 (0.036) Loss 4.292 (3.077) Epoch: [6][370/400] Time 0.364 (0.398) Data 0.001 (0.035) Loss 3.490 (3.088) Epoch: [6][380/400] Time 0.365 (0.397) Data 0.000 (0.034) Loss 2.599 (3.097) Epoch: [6][390/400] Time 0.367 (0.400) Data 0.001 (0.038) Loss 2.956 (3.100) Epoch: [6][400/400] Time 0.358 (0.399) Data 0.000 (0.037) Loss 3.397 (3.106) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.128 (0.158) Data 0.000 (0.028) Computing jaccard distance... Jaccard distance computing time cost: 20.961553812026978 ==> Statistics for epoch 7: 803 clusters Epoch: [7][10/400] Time 0.362 (0.417) Data 0.001 (0.049) Loss 0.741 (0.740) Epoch: [7][20/400] Time 0.364 (0.392) Data 0.000 (0.025) Loss 0.576 (0.666) Epoch: [7][30/400] Time 0.363 (0.381) Data 0.001 (0.017) Loss 0.634 (0.641) Epoch: [7][40/400] Time 0.363 (0.376) Data 0.000 (0.013) Loss 0.411 (0.603) Epoch: [7][50/400] Time 0.363 (0.374) Data 0.000 (0.010) Loss 0.468 (0.554) Epoch: [7][60/400] Time 0.364 (0.401) Data 0.001 (0.038) Loss 3.117 (1.051) Epoch: [7][70/400] Time 0.364 (0.396) Data 0.000 (0.033) Loss 3.634 (1.421) Epoch: [7][80/400] Time 0.365 (0.392) Data 0.000 (0.029) Loss 3.693 (1.686) Epoch: [7][90/400] Time 0.365 (0.388) Data 0.000 (0.025) Loss 3.826 (1.910) Epoch: [7][100/400] Time 0.363 (0.386) Data 0.000 (0.023) Loss 3.239 (2.086) Epoch: [7][110/400] Time 0.363 (0.399) Data 0.001 (0.037) Loss 3.253 (2.211) Epoch: [7][120/400] Time 0.359 (0.397) Data 0.001 (0.034) Loss 2.935 (2.302) Epoch: [7][130/400] Time 0.363 (0.394) Data 0.001 (0.031) Loss 3.105 (2.382) Epoch: [7][140/400] Time 0.365 (0.392) Data 0.001 (0.029) Loss 2.918 (2.452) Epoch: [7][150/400] Time 0.364 (0.390) Data 0.000 (0.027) Loss 3.601 (2.498) Epoch: [7][160/400] Time 0.356 (0.399) Data 0.001 (0.036) Loss 3.396 (2.568) Epoch: [7][170/400] Time 0.355 (0.396) Data 0.001 (0.033) Loss 3.363 (2.623) Epoch: [7][180/400] Time 0.356 (0.394) Data 0.000 (0.032) Loss 3.293 (2.673) Epoch: [7][190/400] Time 0.353 (0.393) Data 0.001 (0.030) Loss 3.481 (2.716) Epoch: [7][200/400] Time 0.363 (0.391) Data 0.000 (0.029) Loss 3.512 (2.769) Epoch: [7][210/400] Time 0.407 (0.398) Data 0.001 (0.035) Loss 3.056 (2.812) Epoch: [7][220/400] Time 0.364 (0.397) Data 0.001 (0.034) Loss 3.987 (2.845) Epoch: [7][230/400] Time 0.358 (0.395) Data 0.000 (0.032) Loss 4.364 (2.876) Epoch: [7][240/400] Time 0.364 (0.394) Data 0.000 (0.031) Loss 4.100 (2.909) Epoch: [7][250/400] Time 0.363 (0.393) Data 0.000 (0.030) Loss 3.368 (2.932) Epoch: [7][260/400] Time 0.366 (0.398) Data 0.000 (0.035) Loss 3.815 (2.958) Epoch: [7][270/400] Time 0.364 (0.397) Data 0.001 (0.034) Loss 4.178 (2.977) Epoch: [7][280/400] Time 0.366 (0.395) Data 0.001 (0.033) Loss 3.080 (2.994) Epoch: [7][290/400] Time 0.366 (0.394) Data 0.000 (0.031) Loss 3.532 (3.008) Epoch: [7][300/400] Time 0.364 (0.393) Data 0.000 (0.030) Loss 3.602 (3.028) Epoch: [7][310/400] Time 0.363 (0.398) Data 0.001 (0.035) Loss 2.980 (3.041) Epoch: [7][320/400] Time 0.354 (0.397) Data 0.001 (0.034) Loss 3.400 (3.052) Epoch: [7][330/400] Time 0.362 (0.396) Data 0.001 (0.033) Loss 3.165 (3.062) Epoch: [7][340/400] Time 0.353 (0.395) Data 0.001 (0.032) Loss 3.562 (3.073) Epoch: [7][350/400] Time 0.363 (0.394) Data 0.000 (0.031) Loss 3.479 (3.082) Epoch: [7][360/400] Time 0.363 (0.398) Data 0.001 (0.035) Loss 2.918 (3.089) Epoch: [7][370/400] Time 0.364 (0.397) Data 0.001 (0.034) Loss 3.367 (3.101) Epoch: [7][380/400] Time 0.365 (0.396) Data 0.001 (0.033) Loss 3.145 (3.108) Epoch: [7][390/400] Time 0.393 (0.395) Data 0.001 (0.033) Loss 2.261 (3.115) Epoch: [7][400/400] Time 0.364 (0.394) Data 0.000 (0.032) Loss 3.137 (3.124) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.127 (0.161) Data 0.000 (0.030) Computing jaccard distance... Jaccard distance computing time cost: 20.03676414489746 ==> Statistics for epoch 8: 799 clusters Epoch: [8][10/400] Time 0.365 (0.420) Data 0.001 (0.049) Loss 0.598 (0.707) Epoch: [8][20/400] Time 0.359 (0.392) Data 0.001 (0.025) Loss 0.571 (0.668) Epoch: [8][30/400] Time 0.366 (0.382) Data 0.001 (0.017) Loss 0.589 (0.644) Epoch: [8][40/400] Time 0.364 (0.377) Data 0.001 (0.013) Loss 0.573 (0.621) Epoch: [8][50/400] Time 2.100 (0.409) Data 1.716 (0.044) Loss 3.106 (0.639) Epoch: [8][60/400] Time 0.356 (0.401) Data 0.000 (0.037) Loss 3.788 (1.123) Epoch: [8][70/400] Time 0.352 (0.395) Data 0.001 (0.032) Loss 3.493 (1.449) Epoch: [8][80/400] Time 0.353 (0.390) Data 0.001 (0.028) Loss 2.146 (1.712) Epoch: [8][90/400] Time 0.353 (0.387) Data 0.001 (0.025) Loss 2.986 (1.912) Epoch: [8][100/400] Time 0.352 (0.402) Data 0.001 (0.040) Loss 3.582 (2.088) Epoch: [8][110/400] Time 0.365 (0.399) Data 0.001 (0.036) Loss 3.188 (2.201) Epoch: [8][120/400] Time 0.364 (0.396) Data 0.001 (0.033) Loss 3.841 (2.302) Epoch: [8][130/400] Time 0.366 (0.394) Data 0.001 (0.031) Loss 2.937 (2.373) Epoch: [8][140/400] Time 0.360 (0.392) Data 0.001 (0.029) Loss 3.857 (2.442) Epoch: [8][150/400] Time 0.418 (0.402) Data 0.001 (0.039) Loss 3.660 (2.511) Epoch: [8][160/400] Time 0.364 (0.399) Data 0.000 (0.036) Loss 3.513 (2.573) Epoch: [8][170/400] Time 0.362 (0.397) Data 0.001 (0.034) Loss 3.805 (2.619) Epoch: [8][180/400] Time 0.361 (0.395) Data 0.001 (0.032) Loss 3.434 (2.668) Epoch: [8][190/400] Time 0.363 (0.393) Data 0.000 (0.031) Loss 3.731 (2.715) Epoch: [8][200/400] Time 0.358 (0.401) Data 0.000 (0.038) Loss 2.796 (2.752) Epoch: [8][210/400] Time 0.364 (0.399) Data 0.001 (0.036) Loss 3.727 (2.783) Epoch: [8][220/400] Time 0.363 (0.397) Data 0.000 (0.034) Loss 3.256 (2.807) Epoch: [8][230/400] Time 0.363 (0.395) Data 0.001 (0.033) Loss 3.585 (2.840) Epoch: [8][240/400] Time 0.363 (0.394) Data 0.000 (0.032) Loss 3.574 (2.871) Epoch: [8][250/400] Time 0.363 (0.400) Data 0.000 (0.037) Loss 3.430 (2.897) Epoch: [8][260/400] Time 0.355 (0.398) Data 0.000 (0.036) Loss 3.468 (2.922) Epoch: [8][270/400] Time 0.363 (0.397) Data 0.000 (0.034) Loss 3.489 (2.942) Epoch: [8][280/400] Time 0.353 (0.396) Data 0.001 (0.033) Loss 3.625 (2.956) Epoch: [8][290/400] Time 0.363 (0.394) Data 0.000 (0.032) Loss 3.313 (2.974) Epoch: [8][300/400] Time 0.365 (0.399) Data 0.001 (0.037) Loss 3.589 (2.987) Epoch: [8][310/400] Time 0.365 (0.398) Data 0.001 (0.036) Loss 3.810 (3.009) Epoch: [8][320/400] Time 0.355 (0.397) Data 0.001 (0.035) Loss 3.761 (3.022) Epoch: [8][330/400] Time 0.364 (0.396) Data 0.000 (0.034) Loss 4.115 (3.036) Epoch: [8][340/400] Time 0.361 (0.395) Data 0.000 (0.033) Loss 4.035 (3.048) Epoch: [8][350/400] Time 0.364 (0.399) Data 0.001 (0.037) Loss 3.287 (3.061) Epoch: [8][360/400] Time 0.363 (0.398) Data 0.000 (0.036) Loss 3.489 (3.067) Epoch: [8][370/400] Time 0.364 (0.397) Data 0.001 (0.035) Loss 3.295 (3.074) Epoch: [8][380/400] Time 0.364 (0.396) Data 0.000 (0.034) Loss 4.110 (3.090) Epoch: [8][390/400] Time 0.364 (0.395) Data 0.000 (0.033) Loss 2.863 (3.096) Epoch: [8][400/400] Time 0.354 (0.399) Data 0.001 (0.037) Loss 3.270 (3.097) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.160) Data 0.000 (0.030) Computing jaccard distance... Jaccard distance computing time cost: 19.664186716079712 ==> Statistics for epoch 9: 824 clusters Epoch: [9][10/400] Time 0.363 (0.416) Data 0.001 (0.058) Loss 0.592 (0.635) Epoch: [9][20/400] Time 0.364 (0.389) Data 0.001 (0.029) Loss 0.543 (0.622) Epoch: [9][30/400] Time 0.373 (0.379) Data 0.001 (0.020) Loss 0.464 (0.579) Epoch: [9][40/400] Time 0.355 (0.373) Data 0.001 (0.015) Loss 0.486 (0.557) Epoch: [9][50/400] Time 0.363 (0.370) Data 0.000 (0.012) Loss 0.417 (0.524) Epoch: [9][60/400] Time 0.352 (0.397) Data 0.001 (0.039) Loss 3.998 (0.975) Epoch: [9][70/400] Time 0.363 (0.394) Data 0.001 (0.034) Loss 3.432 (1.335) Epoch: [9][80/400] Time 0.352 (0.389) Data 0.000 (0.030) Loss 2.783 (1.618) Epoch: [9][90/400] Time 0.353 (0.386) Data 0.000 (0.026) Loss 3.003 (1.821) Epoch: [9][100/400] Time 0.354 (0.383) Data 0.000 (0.024) Loss 3.782 (2.005) Epoch: [9][110/400] Time 0.354 (0.398) Data 0.001 (0.038) Loss 3.219 (2.122) Epoch: [9][120/400] Time 0.369 (0.395) Data 0.001 (0.035) Loss 3.513 (2.217) Epoch: [9][130/400] Time 0.356 (0.392) Data 0.001 (0.032) Loss 3.717 (2.314) Epoch: [9][140/400] Time 0.354 (0.389) Data 0.001 (0.030) Loss 3.731 (2.382) Epoch: [9][150/400] Time 0.364 (0.387) Data 0.000 (0.028) Loss 3.522 (2.460) Epoch: [9][160/400] Time 0.364 (0.398) Data 0.000 (0.038) Loss 3.247 (2.517) Epoch: [9][170/400] Time 0.358 (0.395) Data 0.001 (0.036) Loss 3.656 (2.571) Epoch: [9][180/400] Time 0.370 (0.393) Data 0.000 (0.034) Loss 3.259 (2.614) Epoch: [9][190/400] Time 0.360 (0.391) Data 0.000 (0.032) Loss 3.847 (2.663) Epoch: [9][200/400] Time 0.365 (0.390) Data 0.000 (0.030) Loss 2.787 (2.698) Epoch: [9][210/400] Time 0.353 (0.397) Data 0.001 (0.037) Loss 3.997 (2.739) Epoch: [9][220/400] Time 0.353 (0.395) Data 0.001 (0.036) Loss 3.220 (2.773) Epoch: [9][230/400] Time 0.363 (0.393) Data 0.000 (0.034) Loss 2.794 (2.797) Epoch: [9][240/400] Time 0.353 (0.391) Data 0.000 (0.033) Loss 3.842 (2.828) Epoch: [9][250/400] Time 0.364 (0.390) Data 0.000 (0.031) Loss 3.323 (2.856) Epoch: [9][260/400] Time 0.355 (0.396) Data 0.000 (0.037) Loss 2.733 (2.875) Epoch: [9][270/400] Time 0.353 (0.395) Data 0.001 (0.036) Loss 2.985 (2.894) Epoch: [9][280/400] Time 0.352 (0.393) Data 0.001 (0.035) Loss 3.120 (2.913) Epoch: [9][290/400] Time 0.352 (0.392) Data 0.001 (0.034) Loss 2.927 (2.932) Epoch: [9][300/400] Time 0.364 (0.391) Data 0.000 (0.032) Loss 3.177 (2.946) Epoch: [9][310/400] Time 0.353 (0.396) Data 0.001 (0.037) Loss 3.438 (2.960) Epoch: [9][320/400] Time 0.353 (0.395) Data 0.001 (0.036) Loss 3.001 (2.971) Epoch: [9][330/400] Time 0.374 (0.394) Data 0.001 (0.035) Loss 2.812 (2.981) Epoch: [9][340/400] Time 0.355 (0.393) Data 0.001 (0.034) Loss 3.710 (3.000) Epoch: [9][350/400] Time 0.356 (0.392) Data 0.001 (0.033) Loss 3.407 (3.016) Epoch: [9][360/400] Time 0.354 (0.395) Data 0.001 (0.037) Loss 2.849 (3.028) Epoch: [9][370/400] Time 0.352 (0.394) Data 0.001 (0.036) Loss 3.054 (3.037) Epoch: [9][380/400] Time 0.354 (0.394) Data 0.001 (0.035) Loss 3.148 (3.037) Epoch: [9][390/400] Time 0.351 (0.393) Data 0.001 (0.034) Loss 3.203 (3.042) Epoch: [9][400/400] Time 0.354 (0.392) Data 0.001 (0.033) Loss 3.732 (3.048) Extract Features: [50/76] Time 0.133 (0.159) Data 0.000 (0.029) Mean AP: 53.8% * Finished epoch 9 model mAP: 53.8% best: 53.8% * ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.128 (0.161) Data 0.000 (0.029) Computing jaccard distance... Jaccard distance computing time cost: 19.404287099838257 ==> Statistics for epoch 10: 832 clusters Epoch: [10][10/400] Time 0.352 (0.426) Data 0.001 (0.059) Loss 0.724 (0.652) Epoch: [10][20/400] Time 0.352 (0.389) Data 0.001 (0.030) Loss 0.703 (0.629) Epoch: [10][30/400] Time 0.352 (0.377) Data 0.001 (0.020) Loss 0.772 (0.608) Epoch: [10][40/400] Time 0.353 (0.371) Data 0.001 (0.015) Loss 0.289 (0.561) Epoch: [10][50/400] Time 0.363 (0.370) Data 0.000 (0.012) Loss 0.215 (0.538) Epoch: [10][60/400] Time 0.354 (0.396) Data 0.000 (0.039) Loss 3.770 (0.915) Epoch: [10][70/400] Time 0.354 (0.391) Data 0.001 (0.033) Loss 3.789 (1.271) Epoch: [10][80/400] Time 0.354 (0.387) Data 0.000 (0.029) Loss 3.113 (1.542) Epoch: [10][90/400] Time 0.352 (0.383) Data 0.001 (0.026) Loss 3.385 (1.774) Epoch: [10][100/400] Time 0.363 (0.380) Data 0.000 (0.024) Loss 3.141 (1.958) Epoch: [10][110/400] Time 0.353 (0.395) Data 0.001 (0.038) Loss 3.547 (2.085) Epoch: [10][120/400] Time 0.352 (0.392) Data 0.001 (0.035) Loss 3.849 (2.198) Epoch: [10][130/400] Time 0.355 (0.389) Data 0.001 (0.032) Loss 3.358 (2.279) Epoch: [10][140/400] Time 0.357 (0.388) Data 0.000 (0.030) Loss 2.969 (2.351) Epoch: [10][150/400] Time 0.369 (0.386) Data 0.000 (0.028) Loss 3.277 (2.412) Epoch: [10][160/400] Time 0.350 (0.395) Data 0.001 (0.037) Loss 2.731 (2.463) Epoch: [10][170/400] Time 0.369 (0.393) Data 0.001 (0.035) Loss 3.326 (2.526) Epoch: [10][180/400] Time 0.353 (0.391) Data 0.000 (0.033) Loss 3.250 (2.590) Epoch: [10][190/400] Time 0.359 (0.389) Data 0.000 (0.031) Loss 4.185 (2.639) Epoch: [10][200/400] Time 0.354 (0.387) Data 0.000 (0.030) Loss 3.112 (2.672) Epoch: [10][210/400] Time 0.364 (0.395) Data 0.001 (0.037) Loss 3.719 (2.719) Epoch: [10][220/400] Time 0.354 (0.393) Data 0.001 (0.035) Loss 3.780 (2.747) Epoch: [10][230/400] Time 0.353 (0.391) Data 0.001 (0.033) Loss 3.101 (2.770) Epoch: [10][240/400] Time 0.342 (0.390) Data 0.001 (0.032) Loss 3.630 (2.795) Epoch: [10][250/400] Time 0.353 (0.388) Data 0.001 (0.031) Loss 3.657 (2.832) Epoch: [10][260/400] Time 0.365 (0.387) Data 0.000 (0.030) Loss 3.088 (2.845) Epoch: [10][270/400] Time 0.370 (0.393) Data 0.001 (0.035) Loss 3.160 (2.863) Epoch: [10][280/400] Time 0.377 (0.392) Data 0.000 (0.034) Loss 3.240 (2.878) Epoch: [10][290/400] Time 0.362 (0.391) Data 0.001 (0.033) Loss 2.841 (2.893) Epoch: [10][300/400] Time 0.350 (0.390) Data 0.001 (0.032) Loss 3.499 (2.909) Epoch: [10][310/400] Time 0.363 (0.389) Data 0.000 (0.031) Loss 3.529 (2.928) Epoch: [10][320/400] Time 0.352 (0.394) Data 0.001 (0.035) Loss 3.590 (2.945) Epoch: [10][330/400] Time 0.352 (0.392) Data 0.001 (0.034) Loss 3.562 (2.963) Epoch: [10][340/400] Time 0.353 (0.391) Data 0.000 (0.033) Loss 3.800 (2.979) Epoch: [10][350/400] Time 0.352 (0.390) Data 0.001 (0.032) Loss 3.227 (2.986) Epoch: [10][360/400] Time 0.364 (0.389) Data 0.000 (0.032) Loss 4.077 (3.000) Epoch: [10][370/400] Time 0.355 (0.394) Data 0.000 (0.036) Loss 3.087 (3.014) Epoch: [10][380/400] Time 0.361 (0.393) Data 0.000 (0.035) Loss 3.346 (3.021) Epoch: [10][390/400] Time 0.371 (0.392) Data 0.000 (0.034) Loss 3.906 (3.031) Epoch: [10][400/400] Time 0.352 (0.391) Data 0.001 (0.033) Loss 3.951 (3.036) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.161) Data 0.000 (0.028) Computing jaccard distance... Jaccard distance computing time cost: 19.264953136444092 ==> Statistics for epoch 11: 820 clusters Epoch: [11][10/400] Time 0.361 (0.425) Data 0.001 (0.066) Loss 0.612 (0.591) Epoch: [11][20/400] Time 0.364 (0.391) Data 0.000 (0.033) Loss 0.553 (0.592) Epoch: [11][30/400] Time 0.361 (0.379) Data 0.001 (0.022) Loss 0.460 (0.592) Epoch: [11][40/400] Time 0.351 (0.374) Data 0.000 (0.017) Loss 0.447 (0.555) Epoch: [11][50/400] Time 0.362 (0.371) Data 0.000 (0.013) Loss 0.305 (0.512) Epoch: [11][60/400] Time 0.355 (0.399) Data 0.000 (0.041) Loss 3.779 (0.909) Epoch: [11][70/400] Time 0.351 (0.393) Data 0.000 (0.035) Loss 3.464 (1.264) Epoch: [11][80/400] Time 0.354 (0.389) Data 0.000 (0.031) Loss 3.233 (1.546) Epoch: [11][90/400] Time 0.354 (0.385) Data 0.000 (0.027) Loss 3.937 (1.768) Epoch: [11][100/400] Time 0.362 (0.382) Data 0.000 (0.025) Loss 3.272 (1.956) Epoch: [11][110/400] Time 0.351 (0.396) Data 0.000 (0.038) Loss 3.169 (2.079) Epoch: [11][120/400] Time 0.357 (0.393) Data 0.001 (0.035) Loss 3.332 (2.180) Epoch: [11][130/400] Time 0.352 (0.389) Data 0.000 (0.032) Loss 3.293 (2.266) Epoch: [11][140/400] Time 0.351 (0.387) Data 0.001 (0.030) Loss 3.317 (2.349) Epoch: [11][150/400] Time 0.362 (0.385) Data 0.000 (0.028) Loss 3.531 (2.424) Epoch: [11][160/400] Time 0.351 (0.394) Data 0.001 (0.037) Loss 3.524 (2.475) Epoch: [11][170/400] Time 0.351 (0.392) Data 0.001 (0.035) Loss 2.960 (2.529) Epoch: [11][180/400] Time 0.351 (0.390) Data 0.001 (0.033) Loss 3.660 (2.580) Epoch: [11][190/400] Time 0.365 (0.388) Data 0.001 (0.032) Loss 2.995 (2.623) Epoch: [11][200/400] Time 0.363 (0.387) Data 0.000 (0.030) Loss 3.274 (2.664) Epoch: [11][210/400] Time 0.362 (0.395) Data 0.001 (0.038) Loss 3.163 (2.692) Epoch: [11][220/400] Time 0.352 (0.393) Data 0.001 (0.036) Loss 3.408 (2.712) Epoch: [11][230/400] Time 0.363 (0.392) Data 0.001 (0.035) Loss 3.713 (2.756) Epoch: [11][240/400] Time 0.354 (0.390) Data 0.001 (0.033) Loss 2.348 (2.772) Epoch: [11][250/400] Time 0.364 (0.389) Data 0.000 (0.032) Loss 3.499 (2.798) Epoch: [11][260/400] Time 0.352 (0.395) Data 0.000 (0.038) Loss 4.510 (2.814) Epoch: [11][270/400] Time 0.372 (0.394) Data 0.001 (0.036) Loss 3.749 (2.832) Epoch: [11][280/400] Time 0.352 (0.393) Data 0.001 (0.035) Loss 2.962 (2.854) Epoch: [11][290/400] Time 0.362 (0.392) Data 0.001 (0.034) Loss 3.504 (2.875) Epoch: [11][300/400] Time 0.363 (0.390) Data 0.000 (0.033) Loss 3.720 (2.880) Epoch: [11][310/400] Time 0.351 (0.395) Data 0.000 (0.037) Loss 2.789 (2.896) Epoch: [11][320/400] Time 0.363 (0.394) Data 0.001 (0.036) Loss 3.354 (2.910) Epoch: [11][330/400] Time 0.355 (0.393) Data 0.001 (0.035) Loss 2.951 (2.927) Epoch: [11][340/400] Time 0.364 (0.392) Data 0.001 (0.034) Loss 3.100 (2.945) Epoch: [11][350/400] Time 0.354 (0.391) Data 0.001 (0.033) Loss 3.269 (2.954) Epoch: [11][360/400] Time 0.353 (0.395) Data 0.001 (0.037) Loss 3.439 (2.968) Epoch: [11][370/400] Time 0.353 (0.394) Data 0.001 (0.036) Loss 2.336 (2.975) Epoch: [11][380/400] Time 0.353 (0.393) Data 0.001 (0.035) Loss 3.314 (2.985) Epoch: [11][390/400] Time 0.352 (0.392) Data 0.001 (0.034) Loss 3.860 (2.992) Epoch: [11][400/400] Time 0.354 (0.391) Data 0.001 (0.033) Loss 2.886 (3.006) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.161) Data 0.000 (0.028) Computing jaccard distance... Jaccard distance computing time cost: 19.479636669158936 ==> Statistics for epoch 12: 823 clusters Epoch: [12][10/400] Time 0.350 (0.406) Data 0.000 (0.049) Loss 0.580 (0.660) Epoch: [12][20/400] Time 0.352 (0.381) Data 0.000 (0.025) Loss 0.551 (0.625) Epoch: [12][30/400] Time 0.368 (0.373) Data 0.001 (0.017) Loss 0.421 (0.581) Epoch: [12][40/400] Time 0.355 (0.369) Data 0.001 (0.013) Loss 0.313 (0.540) Epoch: [12][50/400] Time 0.364 (0.367) Data 0.000 (0.010) Loss 0.316 (0.492) Epoch: [12][60/400] Time 0.352 (0.394) Data 0.000 (0.037) Loss 3.107 (0.885) Epoch: [12][70/400] Time 0.377 (0.389) Data 0.000 (0.031) Loss 3.758 (1.241) Epoch: [12][80/400] Time 0.352 (0.385) Data 0.000 (0.028) Loss 3.670 (1.538) Epoch: [12][90/400] Time 0.351 (0.381) Data 0.001 (0.025) Loss 2.967 (1.747) Epoch: [12][100/400] Time 0.363 (0.379) Data 0.000 (0.022) Loss 3.561 (1.928) Epoch: [12][110/400] Time 0.353 (0.394) Data 0.001 (0.036) Loss 3.007 (2.034) Epoch: [12][120/400] Time 0.352 (0.390) Data 0.000 (0.033) Loss 3.539 (2.135) Epoch: [12][130/400] Time 0.352 (0.388) Data 0.001 (0.031) Loss 3.415 (2.224) Epoch: [12][140/400] Time 0.375 (0.385) Data 0.001 (0.029) Loss 3.808 (2.296) Epoch: [12][150/400] Time 0.364 (0.384) Data 0.000 (0.027) Loss 3.084 (2.358) Epoch: [12][160/400] Time 0.352 (0.393) Data 0.000 (0.036) Loss 2.886 (2.419) Epoch: [12][170/400] Time 0.354 (0.391) Data 0.000 (0.034) Loss 3.283 (2.467) Epoch: [12][180/400] Time 0.354 (0.389) Data 0.000 (0.032) Loss 3.750 (2.515) Epoch: [12][190/400] Time 0.353 (0.387) Data 0.000 (0.030) Loss 3.949 (2.560) Epoch: [12][200/400] Time 0.363 (0.386) Data 0.000 (0.029) Loss 3.164 (2.605) Epoch: [12][210/400] Time 0.352 (0.394) Data 0.001 (0.037) Loss 3.314 (2.634) Epoch: [12][220/400] Time 0.351 (0.392) Data 0.000 (0.035) Loss 3.724 (2.659) Epoch: [12][230/400] Time 0.353 (0.391) Data 0.001 (0.034) Loss 3.433 (2.687) Epoch: [12][240/400] Time 0.351 (0.389) Data 0.000 (0.032) Loss 3.072 (2.719) Epoch: [12][250/400] Time 0.361 (0.388) Data 0.000 (0.031) Loss 2.990 (2.742) Epoch: [12][260/400] Time 0.354 (0.393) Data 0.000 (0.037) Loss 2.773 (2.763) Epoch: [12][270/400] Time 0.362 (0.392) Data 0.001 (0.035) Loss 3.360 (2.783) Epoch: [12][280/400] Time 0.353 (0.391) Data 0.000 (0.034) Loss 3.166 (2.801) Epoch: [12][290/400] Time 0.362 (0.390) Data 0.000 (0.033) Loss 2.688 (2.824) Epoch: [12][300/400] Time 0.362 (0.389) Data 0.000 (0.032) Loss 2.819 (2.839) Epoch: [12][310/400] Time 0.352 (0.394) Data 0.000 (0.037) Loss 3.205 (2.846) Epoch: [12][320/400] Time 0.394 (0.393) Data 0.001 (0.036) Loss 3.492 (2.863) Epoch: [12][330/400] Time 0.353 (0.392) Data 0.000 (0.035) Loss 3.305 (2.868) Epoch: [12][340/400] Time 0.362 (0.391) Data 0.000 (0.034) Loss 3.045 (2.877) Epoch: [12][350/400] Time 0.354 (0.390) Data 0.000 (0.033) Loss 3.212 (2.886) Epoch: [12][360/400] Time 0.351 (0.394) Data 0.000 (0.037) Loss 2.953 (2.895) Epoch: [12][370/400] Time 0.355 (0.393) Data 0.000 (0.036) Loss 3.228 (2.902) Epoch: [12][380/400] Time 0.353 (0.392) Data 0.001 (0.035) Loss 2.765 (2.906) Epoch: [12][390/400] Time 0.376 (0.392) Data 0.000 (0.034) Loss 3.026 (2.912) Epoch: [12][400/400] Time 0.363 (0.391) Data 0.001 (0.033) Loss 2.972 (2.922) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.160) Data 0.000 (0.029) Computing jaccard distance... Jaccard distance computing time cost: 19.265992641448975 ==> Statistics for epoch 13: 795 clusters Epoch: [13][10/400] Time 0.350 (0.426) Data 0.000 (0.053) Loss 0.824 (0.678) Epoch: [13][20/400] Time 0.352 (0.391) Data 0.001 (0.027) Loss 0.324 (0.580) Epoch: [13][30/400] Time 0.352 (0.378) Data 0.000 (0.018) Loss 0.401 (0.550) Epoch: [13][40/400] Time 0.351 (0.372) Data 0.000 (0.014) Loss 0.377 (0.512) Epoch: [13][50/400] Time 2.274 (0.408) Data 1.869 (0.048) Loss 3.357 (0.535) Epoch: [13][60/400] Time 0.352 (0.399) Data 0.000 (0.040) Loss 3.581 (0.979) Epoch: [13][70/400] Time 0.355 (0.392) Data 0.000 (0.035) Loss 4.060 (1.349) Epoch: [13][80/400] Time 0.352 (0.387) Data 0.000 (0.030) Loss 3.170 (1.605) Epoch: [13][90/400] Time 0.353 (0.384) Data 0.000 (0.027) Loss 3.976 (1.797) Epoch: [13][100/400] Time 0.378 (0.401) Data 0.000 (0.043) Loss 2.443 (1.947) Epoch: [13][110/400] Time 0.362 (0.397) Data 0.000 (0.039) Loss 3.263 (2.076) Epoch: [13][120/400] Time 0.362 (0.394) Data 0.000 (0.036) Loss 3.032 (2.173) Epoch: [13][130/400] Time 0.364 (0.391) Data 0.000 (0.033) Loss 3.170 (2.243) Epoch: [13][140/400] Time 0.362 (0.389) Data 0.001 (0.031) Loss 3.357 (2.306) Epoch: [13][150/400] Time 0.356 (0.399) Data 0.000 (0.040) Loss 3.208 (2.359) Epoch: [13][160/400] Time 0.359 (0.396) Data 0.001 (0.038) Loss 3.320 (2.415) Epoch: [13][170/400] Time 0.356 (0.394) Data 0.000 (0.036) Loss 3.595 (2.476) Epoch: [13][180/400] Time 0.353 (0.392) Data 0.001 (0.034) Loss 3.498 (2.532) Epoch: [13][190/400] Time 0.363 (0.390) Data 0.000 (0.032) Loss 3.206 (2.583) Epoch: [13][200/400] Time 0.354 (0.399) Data 0.000 (0.040) Loss 3.678 (2.616) Epoch: [13][210/400] Time 0.353 (0.397) Data 0.001 (0.038) Loss 3.085 (2.651) Epoch: [13][220/400] Time 0.354 (0.395) Data 0.000 (0.036) Loss 2.938 (2.679) Epoch: [13][230/400] Time 0.352 (0.393) Data 0.001 (0.035) Loss 3.427 (2.707) Epoch: [13][240/400] Time 0.363 (0.392) Data 0.000 (0.033) Loss 3.214 (2.734) Epoch: [13][250/400] Time 0.354 (0.397) Data 0.001 (0.039) Loss 3.185 (2.753) Epoch: [13][260/400] Time 0.351 (0.396) Data 0.001 (0.037) Loss 3.228 (2.780) Epoch: [13][270/400] Time 0.353 (0.394) Data 0.001 (0.036) Loss 3.484 (2.799) Epoch: [13][280/400] Time 0.351 (0.393) Data 0.001 (0.035) Loss 3.392 (2.818) Epoch: [13][290/400] Time 0.364 (0.392) Data 0.000 (0.034) Loss 3.271 (2.830) Epoch: [13][300/400] Time 0.358 (0.397) Data 0.001 (0.039) Loss 3.744 (2.848) Epoch: [13][310/400] Time 0.354 (0.396) Data 0.001 (0.037) Loss 3.242 (2.856) Epoch: [13][320/400] Time 0.351 (0.394) Data 0.001 (0.036) Loss 4.199 (2.868) Epoch: [13][330/400] Time 0.353 (0.393) Data 0.000 (0.035) Loss 3.117 (2.881) Epoch: [13][340/400] Time 0.364 (0.393) Data 0.000 (0.034) Loss 3.464 (2.896) Epoch: [13][350/400] Time 0.376 (0.397) Data 0.000 (0.038) Loss 2.727 (2.909) Epoch: [13][360/400] Time 0.354 (0.395) Data 0.000 (0.037) Loss 3.185 (2.920) Epoch: [13][370/400] Time 0.354 (0.394) Data 0.000 (0.036) Loss 3.677 (2.932) Epoch: [13][380/400] Time 0.354 (0.393) Data 0.001 (0.035) Loss 3.252 (2.945) Epoch: [13][390/400] Time 0.375 (0.393) Data 0.000 (0.034) Loss 2.978 (2.949) Epoch: [13][400/400] Time 0.354 (0.396) Data 0.001 (0.038) Loss 3.420 (2.955) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.159) Data 0.000 (0.027) Computing jaccard distance... Jaccard distance computing time cost: 19.923077821731567 ==> Statistics for epoch 14: 790 clusters Epoch: [14][10/400] Time 0.362 (0.416) Data 0.000 (0.048) Loss 0.610 (0.566) Epoch: [14][20/400] Time 0.351 (0.386) Data 0.001 (0.024) Loss 0.554 (0.538) Epoch: [14][30/400] Time 0.352 (0.376) Data 0.001 (0.016) Loss 0.321 (0.498) Epoch: [14][40/400] Time 0.361 (0.372) Data 0.001 (0.012) Loss 0.381 (0.475) Epoch: [14][50/400] Time 2.216 (0.406) Data 1.846 (0.047) Loss 2.810 (0.496) Epoch: [14][60/400] Time 0.351 (0.399) Data 0.001 (0.039) Loss 3.211 (0.944) Epoch: [14][70/400] Time 0.353 (0.393) Data 0.001 (0.034) Loss 3.647 (1.274) Epoch: [14][80/400] Time 0.351 (0.389) Data 0.001 (0.030) Loss 4.010 (1.545) Epoch: [14][90/400] Time 0.353 (0.385) Data 0.001 (0.026) Loss 3.302 (1.752) Epoch: [14][100/400] Time 0.348 (0.401) Data 0.000 (0.041) Loss 3.138 (1.925) Epoch: [14][110/400] Time 0.352 (0.397) Data 0.000 (0.038) Loss 3.270 (2.021) Epoch: [14][120/400] Time 0.362 (0.394) Data 0.001 (0.035) Loss 3.810 (2.130) Epoch: [14][130/400] Time 0.355 (0.391) Data 0.001 (0.032) Loss 3.018 (2.196) Epoch: [14][140/400] Time 0.352 (0.389) Data 0.001 (0.030) Loss 3.428 (2.267) Epoch: [14][150/400] Time 0.351 (0.399) Data 0.000 (0.040) Loss 2.540 (2.318) Epoch: [14][160/400] Time 0.362 (0.396) Data 0.001 (0.037) Loss 3.774 (2.392) Epoch: [14][170/400] Time 0.352 (0.394) Data 0.000 (0.035) Loss 3.386 (2.439) Epoch: [14][180/400] Time 0.352 (0.392) Data 0.000 (0.033) Loss 3.828 (2.493) Epoch: [14][190/400] Time 0.363 (0.390) Data 0.000 (0.031) Loss 3.470 (2.540) Epoch: [14][200/400] Time 0.353 (0.398) Data 0.000 (0.039) Loss 3.056 (2.566) Epoch: [14][210/400] Time 0.361 (0.396) Data 0.001 (0.038) Loss 3.308 (2.603) Epoch: [14][220/400] Time 0.352 (0.395) Data 0.001 (0.036) Loss 3.197 (2.634) Epoch: [14][230/400] Time 0.406 (0.393) Data 0.001 (0.034) Loss 3.360 (2.657) Epoch: [14][240/400] Time 0.362 (0.392) Data 0.000 (0.033) Loss 3.102 (2.685) Epoch: [14][250/400] Time 0.351 (0.398) Data 0.001 (0.039) Loss 3.292 (2.709) Epoch: [14][260/400] Time 0.355 (0.396) Data 0.001 (0.037) Loss 3.541 (2.733) Epoch: [14][270/400] Time 0.352 (0.395) Data 0.001 (0.036) Loss 3.391 (2.754) Epoch: [14][280/400] Time 0.357 (0.393) Data 0.001 (0.035) Loss 3.009 (2.780) Epoch: [14][290/400] Time 0.364 (0.392) Data 0.000 (0.033) Loss 3.992 (2.795) Epoch: [14][300/400] Time 0.353 (0.398) Data 0.000 (0.039) Loss 3.267 (2.809) Epoch: [14][310/400] Time 0.368 (0.396) Data 0.001 (0.037) Loss 3.523 (2.826) Epoch: [14][320/400] Time 0.363 (0.395) Data 0.001 (0.036) Loss 2.903 (2.833) Epoch: [14][330/400] Time 0.351 (0.394) Data 0.000 (0.035) Loss 2.882 (2.840) Epoch: [14][340/400] Time 0.364 (0.393) Data 0.000 (0.034) Loss 3.980 (2.858) Epoch: [14][350/400] Time 0.351 (0.397) Data 0.001 (0.038) Loss 4.101 (2.872) Epoch: [14][360/400] Time 0.361 (0.396) Data 0.001 (0.037) Loss 3.365 (2.882) Epoch: [14][370/400] Time 0.352 (0.395) Data 0.001 (0.036) Loss 3.678 (2.894) Epoch: [14][380/400] Time 0.351 (0.394) Data 0.001 (0.035) Loss 3.371 (2.905) Epoch: [14][390/400] Time 0.363 (0.393) Data 0.000 (0.034) Loss 3.487 (2.907) Epoch: [14][400/400] Time 0.352 (0.397) Data 0.001 (0.038) Loss 2.707 (2.913) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.159) Data 0.000 (0.029) Computing jaccard distance... Jaccard distance computing time cost: 18.7038893699646 ==> Statistics for epoch 15: 762 clusters Epoch: [15][10/400] Time 0.361 (0.408) Data 0.001 (0.052) Loss 0.427 (0.511) Epoch: [15][20/400] Time 0.344 (0.379) Data 0.001 (0.027) Loss 0.680 (0.525) Epoch: [15][30/400] Time 0.350 (0.369) Data 0.001 (0.018) Loss 0.465 (0.490) Epoch: [15][40/400] Time 0.379 (0.365) Data 0.001 (0.014) Loss 0.393 (0.465) Epoch: [15][50/400] Time 0.353 (0.403) Data 0.001 (0.048) Loss 3.886 (0.625) Epoch: [15][60/400] Time 0.363 (0.395) Data 0.001 (0.041) Loss 4.075 (1.036) Epoch: [15][70/400] Time 0.354 (0.389) Data 0.001 (0.035) Loss 3.462 (1.347) Epoch: [15][80/400] Time 0.351 (0.385) Data 0.001 (0.031) Loss 3.275 (1.604) Epoch: [15][90/400] Time 0.361 (0.382) Data 0.000 (0.027) Loss 3.815 (1.821) Epoch: [15][100/400] Time 0.352 (0.397) Data 0.001 (0.042) Loss 3.366 (1.957) Epoch: [15][110/400] Time 0.352 (0.393) Data 0.001 (0.038) Loss 3.331 (2.063) Epoch: [15][120/400] Time 0.358 (0.391) Data 0.001 (0.035) Loss 2.829 (2.147) Epoch: [15][130/400] Time 0.351 (0.388) Data 0.001 (0.032) Loss 3.148 (2.216) Epoch: [15][140/400] Time 0.362 (0.386) Data 0.000 (0.030) Loss 3.133 (2.283) Epoch: [15][150/400] Time 0.353 (0.395) Data 0.000 (0.039) Loss 3.000 (2.326) Epoch: [15][160/400] Time 0.351 (0.392) Data 0.000 (0.037) Loss 2.966 (2.371) Epoch: [15][170/400] Time 0.352 (0.390) Data 0.001 (0.035) Loss 3.734 (2.429) Epoch: [15][180/400] Time 0.352 (0.388) Data 0.000 (0.033) Loss 3.507 (2.491) Epoch: [15][190/400] Time 0.351 (0.396) Data 0.000 (0.040) Loss 2.590 (2.533) Epoch: [15][200/400] Time 0.352 (0.394) Data 0.001 (0.038) Loss 3.072 (2.566) Epoch: [15][210/400] Time 0.351 (0.392) Data 0.000 (0.037) Loss 3.743 (2.606) Epoch: [15][220/400] Time 0.351 (0.390) Data 0.001 (0.035) Loss 4.051 (2.639) Epoch: [15][230/400] Time 0.363 (0.389) Data 0.000 (0.033) Loss 2.909 (2.665) Epoch: [15][240/400] Time 0.353 (0.395) Data 0.001 (0.039) Loss 3.017 (2.698) Epoch: [15][250/400] Time 0.353 (0.394) Data 0.001 (0.038) Loss 3.426 (2.713) Epoch: [15][260/400] Time 0.374 (0.392) Data 0.001 (0.036) Loss 2.636 (2.730) Epoch: [15][270/400] Time 0.355 (0.391) Data 0.001 (0.035) Loss 2.961 (2.754) Epoch: [15][280/400] Time 0.364 (0.390) Data 0.000 (0.034) Loss 3.096 (2.773) Epoch: [15][290/400] Time 0.355 (0.395) Data 0.001 (0.039) Loss 2.648 (2.790) Epoch: [15][300/400] Time 0.354 (0.394) Data 0.001 (0.038) Loss 3.332 (2.796) Epoch: [15][310/400] Time 0.363 (0.393) Data 0.001 (0.036) Loss 2.959 (2.814) Epoch: [15][320/400] Time 0.352 (0.392) Data 0.001 (0.035) Loss 3.060 (2.828) Epoch: [15][330/400] Time 2.109 (0.396) Data 1.729 (0.039) Loss 3.621 (2.842) Epoch: [15][340/400] Time 0.353 (0.395) Data 0.001 (0.038) Loss 3.697 (2.851) Epoch: [15][350/400] Time 0.354 (0.394) Data 0.001 (0.037) Loss 3.061 (2.856) Epoch: [15][360/400] Time 0.353 (0.393) Data 0.000 (0.036) Loss 3.597 (2.868) Epoch: [15][370/400] Time 0.365 (0.392) Data 0.000 (0.035) Loss 2.889 (2.879) Epoch: [15][380/400] Time 0.353 (0.396) Data 0.000 (0.039) Loss 3.172 (2.887) Epoch: [15][390/400] Time 0.405 (0.395) Data 0.001 (0.038) Loss 3.439 (2.896) Epoch: [15][400/400] Time 0.353 (0.394) Data 0.000 (0.037) Loss 3.548 (2.901) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.162) Data 0.000 (0.028) Computing jaccard distance... Jaccard distance computing time cost: 19.068045377731323 ==> Statistics for epoch 16: 752 clusters Epoch: [16][10/400] Time 0.349 (0.414) Data 0.000 (0.053) Loss 0.500 (0.518) Epoch: [16][20/400] Time 0.351 (0.382) Data 0.000 (0.027) Loss 0.438 (0.509) Epoch: [16][30/400] Time 0.351 (0.374) Data 0.000 (0.018) Loss 0.239 (0.498) Epoch: [16][40/400] Time 0.351 (0.370) Data 0.001 (0.014) Loss 0.273 (0.460) Epoch: [16][50/400] Time 0.351 (0.406) Data 0.000 (0.049) Loss 3.233 (0.605) Epoch: [16][60/400] Time 0.371 (0.399) Data 0.000 (0.041) Loss 2.734 (1.023) Epoch: [16][70/400] Time 0.350 (0.392) Data 0.000 (0.035) Loss 3.455 (1.349) Epoch: [16][80/400] Time 0.354 (0.388) Data 0.001 (0.031) Loss 3.746 (1.599) Epoch: [16][90/400] Time 0.362 (0.384) Data 0.000 (0.027) Loss 3.622 (1.805) Epoch: [16][100/400] Time 0.357 (0.400) Data 0.001 (0.042) Loss 2.684 (1.938) Epoch: [16][110/400] Time 0.351 (0.396) Data 0.001 (0.039) Loss 3.758 (2.046) Epoch: [16][120/400] Time 0.361 (0.393) Data 0.000 (0.035) Loss 2.084 (2.116) Epoch: [16][130/400] Time 0.371 (0.390) Data 0.001 (0.033) Loss 3.511 (2.181) Epoch: [16][140/400] Time 0.363 (0.388) Data 0.000 (0.030) Loss 2.800 (2.250) Epoch: [16][150/400] Time 0.375 (0.399) Data 0.000 (0.041) Loss 2.657 (2.301) Epoch: [16][160/400] Time 0.352 (0.396) Data 0.001 (0.038) Loss 3.241 (2.360) Epoch: [16][170/400] Time 0.350 (0.394) Data 0.001 (0.036) Loss 3.451 (2.422) Epoch: [16][180/400] Time 0.352 (0.391) Data 0.001 (0.034) Loss 3.335 (2.469) Epoch: [16][190/400] Time 0.375 (0.399) Data 0.001 (0.042) Loss 3.002 (2.510) Epoch: [16][200/400] Time 0.355 (0.397) Data 0.000 (0.039) Loss 2.965 (2.534) Epoch: [16][210/400] Time 0.365 (0.395) Data 0.001 (0.038) Loss 3.615 (2.571) Epoch: [16][220/400] Time 0.361 (0.394) Data 0.001 (0.036) Loss 3.552 (2.598) Epoch: [16][230/400] Time 0.362 (0.392) Data 0.000 (0.034) Loss 3.023 (2.618) Epoch: [16][240/400] Time 0.364 (0.398) Data 0.000 (0.040) Loss 3.833 (2.643) Epoch: [16][250/400] Time 0.350 (0.396) Data 0.001 (0.038) Loss 2.741 (2.667) Epoch: [16][260/400] Time 0.352 (0.395) Data 0.000 (0.037) Loss 2.653 (2.688) Epoch: [16][270/400] Time 0.354 (0.393) Data 0.000 (0.036) Loss 3.082 (2.706) Epoch: [16][280/400] Time 0.363 (0.392) Data 0.000 (0.034) Loss 3.040 (2.733) Epoch: [16][290/400] Time 0.364 (0.397) Data 0.000 (0.039) Loss 3.054 (2.752) Epoch: [16][300/400] Time 0.356 (0.396) Data 0.001 (0.038) Loss 3.046 (2.761) Epoch: [16][310/400] Time 0.366 (0.395) Data 0.000 (0.037) Loss 3.188 (2.774) Epoch: [16][320/400] Time 0.364 (0.394) Data 0.001 (0.036) Loss 3.074 (2.783) Epoch: [16][330/400] Time 2.142 (0.398) Data 1.691 (0.040) Loss 3.194 (2.789) Epoch: [16][340/400] Time 0.353 (0.397) Data 0.001 (0.038) Loss 1.998 (2.798) Epoch: [16][350/400] Time 0.354 (0.396) Data 0.000 (0.037) Loss 2.835 (2.806) Epoch: [16][360/400] Time 0.354 (0.394) Data 0.000 (0.036) Loss 3.600 (2.824) Epoch: [16][370/400] Time 0.363 (0.393) Data 0.000 (0.035) Loss 3.361 (2.837) Epoch: [16][380/400] Time 0.350 (0.397) Data 0.000 (0.039) Loss 3.627 (2.846) Epoch: [16][390/400] Time 0.351 (0.396) Data 0.001 (0.038) Loss 3.628 (2.857) Epoch: [16][400/400] Time 0.354 (0.395) Data 0.001 (0.037) Loss 2.607 (2.863) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.158) Data 0.000 (0.027) Computing jaccard distance... Jaccard distance computing time cost: 18.778992891311646 ==> Statistics for epoch 17: 744 clusters Epoch: [17][10/400] Time 0.350 (0.404) Data 0.001 (0.046) Loss 0.413 (0.525) Epoch: [17][20/400] Time 0.351 (0.382) Data 0.001 (0.023) Loss 0.344 (0.506) Epoch: [17][30/400] Time 0.362 (0.374) Data 0.001 (0.016) Loss 0.320 (0.475) Epoch: [17][40/400] Time 0.363 (0.369) Data 0.000 (0.012) Loss 0.243 (0.429) Epoch: [17][50/400] Time 0.352 (0.405) Data 0.000 (0.046) Loss 2.625 (0.619) Epoch: [17][60/400] Time 0.363 (0.397) Data 0.001 (0.039) Loss 3.538 (1.028) Epoch: [17][70/400] Time 0.356 (0.391) Data 0.000 (0.033) Loss 3.579 (1.337) Epoch: [17][80/400] Time 0.364 (0.387) Data 0.001 (0.029) Loss 2.884 (1.575) Epoch: [17][90/400] Time 0.363 (0.384) Data 0.000 (0.026) Loss 3.305 (1.786) Epoch: [17][100/400] Time 0.352 (0.399) Data 0.000 (0.041) Loss 2.545 (1.909) Epoch: [17][110/400] Time 0.350 (0.395) Data 0.001 (0.038) Loss 3.551 (2.010) Epoch: [17][120/400] Time 0.379 (0.392) Data 0.001 (0.035) Loss 2.456 (2.070) Epoch: [17][130/400] Time 0.358 (0.389) Data 0.001 (0.032) Loss 2.775 (2.147) Epoch: [17][140/400] Time 0.351 (0.400) Data 0.000 (0.042) Loss 2.695 (2.210) Epoch: [17][150/400] Time 0.351 (0.397) Data 0.000 (0.039) Loss 2.791 (2.272) Epoch: [17][160/400] Time 0.354 (0.395) Data 0.001 (0.037) Loss 3.777 (2.328) Epoch: [17][170/400] Time 0.353 (0.392) Data 0.001 (0.035) Loss 3.818 (2.384) Epoch: [17][180/400] Time 0.363 (0.390) Data 0.000 (0.033) Loss 3.937 (2.442) Epoch: [17][190/400] Time 0.354 (0.398) Data 0.001 (0.040) Loss 3.735 (2.481) Epoch: [17][200/400] Time 0.350 (0.396) Data 0.001 (0.038) Loss 3.109 (2.507) Epoch: [17][210/400] Time 0.362 (0.394) Data 0.001 (0.036) Loss 2.810 (2.538) Epoch: [17][220/400] Time 0.352 (0.393) Data 0.001 (0.035) Loss 2.615 (2.563) Epoch: [17][230/400] Time 0.362 (0.391) Data 0.000 (0.033) Loss 3.619 (2.587) Epoch: [17][240/400] Time 0.362 (0.397) Data 0.001 (0.039) Loss 2.995 (2.613) Epoch: [17][250/400] Time 0.352 (0.395) Data 0.001 (0.038) Loss 3.345 (2.634) Epoch: [17][260/400] Time 0.354 (0.394) Data 0.001 (0.036) Loss 2.918 (2.650) Epoch: [17][270/400] Time 0.364 (0.392) Data 0.000 (0.035) Loss 2.958 (2.669) Epoch: [17][280/400] Time 0.405 (0.398) Data 0.000 (0.040) Loss 2.801 (2.687) Epoch: [17][290/400] Time 0.349 (0.396) Data 0.001 (0.038) Loss 3.234 (2.700) Epoch: [17][300/400] Time 0.362 (0.395) Data 0.001 (0.037) Loss 2.987 (2.710) Epoch: [17][310/400] Time 0.353 (0.394) Data 0.001 (0.036) Loss 3.327 (2.726) Epoch: [17][320/400] Time 0.363 (0.393) Data 0.000 (0.035) Loss 3.132 (2.733) Epoch: [17][330/400] Time 0.352 (0.397) Data 0.001 (0.039) Loss 2.924 (2.742) Epoch: [17][340/400] Time 0.364 (0.396) Data 0.001 (0.038) Loss 3.477 (2.755) Epoch: [17][350/400] Time 0.353 (0.395) Data 0.000 (0.037) Loss 4.079 (2.770) Epoch: [17][360/400] Time 0.364 (0.394) Data 0.000 (0.036) Loss 3.004 (2.781) Epoch: [17][370/400] Time 0.351 (0.397) Data 0.001 (0.039) Loss 2.665 (2.791) Epoch: [17][380/400] Time 0.353 (0.396) Data 0.001 (0.038) Loss 2.900 (2.793) Epoch: [17][390/400] Time 0.351 (0.395) Data 0.001 (0.037) Loss 2.701 (2.798) Epoch: [17][400/400] Time 0.352 (0.394) Data 0.001 (0.037) Loss 2.122 (2.806) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.124 (0.159) Data 0.000 (0.029) Computing jaccard distance... Jaccard distance computing time cost: 18.61119508743286 ==> Statistics for epoch 18: 725 clusters Epoch: [18][10/400] Time 0.360 (0.419) Data 0.001 (0.058) Loss 0.515 (0.526) Epoch: [18][20/400] Time 0.366 (0.389) Data 0.001 (0.029) Loss 0.368 (0.455) Epoch: [18][30/400] Time 0.369 (0.379) Data 0.000 (0.020) Loss 0.421 (0.432) Epoch: [18][40/400] Time 0.362 (0.374) Data 0.000 (0.015) Loss 0.357 (0.407) Epoch: [18][50/400] Time 0.353 (0.405) Data 0.000 (0.046) Loss 2.700 (0.637) Epoch: [18][60/400] Time 0.371 (0.398) Data 0.001 (0.038) Loss 2.934 (1.057) Epoch: [18][70/400] Time 0.350 (0.392) Data 0.001 (0.033) Loss 3.014 (1.374) Epoch: [18][80/400] Time 0.364 (0.388) Data 0.001 (0.029) Loss 2.810 (1.612) Epoch: [18][90/400] Time 0.362 (0.385) Data 0.000 (0.026) Loss 3.201 (1.804) Epoch: [18][100/400] Time 0.354 (0.401) Data 0.000 (0.041) Loss 2.767 (1.907) Epoch: [18][110/400] Time 0.351 (0.397) Data 0.001 (0.037) Loss 3.115 (1.999) Epoch: [18][120/400] Time 0.353 (0.393) Data 0.000 (0.034) Loss 2.943 (2.105) Epoch: [18][130/400] Time 0.364 (0.390) Data 0.000 (0.032) Loss 2.687 (2.179) Epoch: [18][140/400] Time 0.353 (0.401) Data 0.000 (0.042) Loss 2.881 (2.230) Epoch: [18][150/400] Time 0.362 (0.398) Data 0.001 (0.039) Loss 3.405 (2.303) Epoch: [18][160/400] Time 0.352 (0.395) Data 0.001 (0.037) Loss 3.115 (2.353) Epoch: [18][170/400] Time 0.369 (0.393) Data 0.001 (0.035) Loss 3.282 (2.411) Epoch: [18][180/400] Time 0.362 (0.391) Data 0.000 (0.033) Loss 3.578 (2.455) Epoch: [18][190/400] Time 0.352 (0.400) Data 0.001 (0.041) Loss 2.819 (2.486) Epoch: [18][200/400] Time 0.351 (0.397) Data 0.000 (0.039) Loss 3.273 (2.520) Epoch: [18][210/400] Time 0.352 (0.395) Data 0.001 (0.037) Loss 3.079 (2.542) Epoch: [18][220/400] Time 0.362 (0.394) Data 0.000 (0.036) Loss 3.234 (2.572) Epoch: [18][230/400] Time 0.354 (0.400) Data 0.000 (0.042) Loss 2.924 (2.595) Epoch: [18][240/400] Time 0.349 (0.398) Data 0.000 (0.040) Loss 3.282 (2.626) Epoch: [18][250/400] Time 0.353 (0.397) Data 0.001 (0.038) Loss 3.278 (2.646) Epoch: [18][260/400] Time 0.352 (0.395) Data 0.000 (0.037) Loss 3.142 (2.666) Epoch: [18][270/400] Time 0.362 (0.394) Data 0.000 (0.035) Loss 3.349 (2.683) Epoch: [18][280/400] Time 0.363 (0.399) Data 0.001 (0.041) Loss 2.549 (2.701) Epoch: [18][290/400] Time 0.353 (0.398) Data 0.000 (0.039) Loss 2.947 (2.714) Epoch: [18][300/400] Time 0.356 (0.397) Data 0.001 (0.038) Loss 3.413 (2.728) Epoch: [18][310/400] Time 0.361 (0.396) Data 0.000 (0.037) Loss 3.380 (2.737) Epoch: [18][320/400] Time 0.352 (0.400) Data 0.001 (0.041) Loss 3.836 (2.751) Epoch: [18][330/400] Time 0.353 (0.399) Data 0.001 (0.040) Loss 2.782 (2.762) Epoch: [18][340/400] Time 0.352 (0.397) Data 0.001 (0.039) Loss 3.140 (2.770) Epoch: [18][350/400] Time 0.353 (0.396) Data 0.001 (0.038) Loss 3.102 (2.777) Epoch: [18][360/400] Time 0.364 (0.395) Data 0.000 (0.037) Loss 3.474 (2.785) Epoch: [18][370/400] Time 0.364 (0.400) Data 0.001 (0.041) Loss 3.068 (2.792) Epoch: [18][380/400] Time 0.398 (0.399) Data 0.000 (0.040) Loss 2.818 (2.802) Epoch: [18][390/400] Time 0.364 (0.397) Data 0.001 (0.039) Loss 2.980 (2.805) Epoch: [18][400/400] Time 0.363 (0.396) Data 0.000 (0.038) Loss 2.303 (2.812) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.164) Data 0.000 (0.030) Computing jaccard distance... Jaccard distance computing time cost: 18.65268301963806 ==> Statistics for epoch 19: 718 clusters Epoch: [19][10/400] Time 0.354 (0.409) Data 0.001 (0.048) Loss 0.564 (0.489) Epoch: [19][20/400] Time 0.351 (0.383) Data 0.001 (0.024) Loss 0.433 (0.468) Epoch: [19][30/400] Time 0.350 (0.372) Data 0.001 (0.016) Loss 0.333 (0.432) Epoch: [19][40/400] Time 0.362 (0.368) Data 0.000 (0.012) Loss 0.277 (0.401) Epoch: [19][50/400] Time 0.362 (0.401) Data 0.001 (0.044) Loss 3.021 (0.724) Epoch: [19][60/400] Time 0.353 (0.393) Data 0.000 (0.036) Loss 3.214 (1.109) Epoch: [19][70/400] Time 0.352 (0.388) Data 0.001 (0.031) Loss 3.490 (1.362) Epoch: [19][80/400] Time 0.352 (0.383) Data 0.000 (0.027) Loss 2.828 (1.590) Epoch: [19][90/400] Time 0.349 (0.400) Data 0.001 (0.044) Loss 3.026 (1.755) Epoch: [19][100/400] Time 0.363 (0.396) Data 0.001 (0.040) Loss 2.432 (1.860) Epoch: [19][110/400] Time 0.353 (0.393) Data 0.000 (0.036) Loss 2.974 (1.940) Epoch: [19][120/400] Time 0.361 (0.390) Data 0.001 (0.033) Loss 2.866 (2.025) Epoch: [19][130/400] Time 0.363 (0.387) Data 0.000 (0.031) Loss 2.962 (2.082) Epoch: [19][140/400] Time 0.351 (0.399) Data 0.001 (0.041) Loss 2.721 (2.155) Epoch: [19][150/400] Time 0.350 (0.396) Data 0.001 (0.039) Loss 3.033 (2.206) Epoch: [19][160/400] Time 0.351 (0.393) Data 0.001 (0.036) Loss 3.136 (2.266) Epoch: [19][170/400] Time 0.364 (0.391) Data 0.000 (0.034) Loss 2.575 (2.304) Epoch: [19][180/400] Time 0.351 (0.400) Data 0.001 (0.042) Loss 3.043 (2.341) Epoch: [19][190/400] Time 0.353 (0.398) Data 0.001 (0.040) Loss 2.589 (2.358) Epoch: [19][200/400] Time 0.354 (0.395) Data 0.001 (0.038) Loss 2.823 (2.389) Epoch: [19][210/400] Time 0.354 (0.393) Data 0.001 (0.036) Loss 2.407 (2.422) Epoch: [19][220/400] Time 0.355 (0.392) Data 0.000 (0.035) Loss 3.007 (2.451) Epoch: [19][230/400] Time 0.355 (0.398) Data 0.001 (0.041) Loss 3.365 (2.478) Epoch: [19][240/400] Time 0.354 (0.397) Data 0.000 (0.039) Loss 2.966 (2.501) Epoch: [19][250/400] Time 0.354 (0.395) Data 0.001 (0.038) Loss 3.292 (2.523) Epoch: [19][260/400] Time 0.374 (0.394) Data 0.000 (0.036) Loss 2.445 (2.539) Epoch: [19][270/400] Time 0.363 (0.399) Data 0.001 (0.042) Loss 3.306 (2.552) Epoch: [19][280/400] Time 0.354 (0.397) Data 0.001 (0.040) Loss 3.214 (2.571) Epoch: [19][290/400] Time 0.375 (0.396) Data 0.001 (0.039) Loss 3.514 (2.592) Epoch: [19][300/400] Time 0.351 (0.395) Data 0.001 (0.037) Loss 2.527 (2.602) Epoch: [19][310/400] Time 0.351 (0.400) Data 0.001 (0.042) Loss 3.175 (2.617) Epoch: [19][320/400] Time 0.353 (0.398) Data 0.001 (0.041) Loss 3.120 (2.630) Epoch: [19][330/400] Time 0.362 (0.397) Data 0.001 (0.039) Loss 2.933 (2.646) Epoch: [19][340/400] Time 0.354 (0.396) Data 0.000 (0.038) Loss 2.819 (2.662) Epoch: [19][350/400] Time 0.365 (0.395) Data 0.000 (0.037) Loss 3.588 (2.676) Epoch: [19][360/400] Time 0.353 (0.399) Data 0.000 (0.041) Loss 2.967 (2.684) Epoch: [19][370/400] Time 0.352 (0.398) Data 0.001 (0.040) Loss 2.823 (2.688) Epoch: [19][380/400] Time 0.375 (0.397) Data 0.000 (0.039) Loss 3.236 (2.693) Epoch: [19][390/400] Time 0.364 (0.396) Data 0.000 (0.038) Loss 2.652 (2.697) Epoch: [19][400/400] Time 0.354 (0.400) Data 0.001 (0.042) Loss 2.646 (2.707) Extract Features: [50/76] Time 0.133 (0.163) Data 0.000 (0.029) Mean AP: 76.9% * Finished epoch 19 model mAP: 76.9% best: 76.9% * ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.136 (0.161) Data 0.000 (0.029) Computing jaccard distance... Jaccard distance computing time cost: 19.028404235839844 ==> Statistics for epoch 20: 710 clusters Epoch: [20][10/400] Time 0.361 (0.421) Data 0.001 (0.052) Loss 0.460 (0.458) Epoch: [20][20/400] Time 0.349 (0.387) Data 0.001 (0.026) Loss 0.355 (0.444) Epoch: [20][30/400] Time 0.362 (0.378) Data 0.001 (0.018) Loss 0.334 (0.431) Epoch: [20][40/400] Time 0.363 (0.373) Data 0.000 (0.014) Loss 0.351 (0.403) Epoch: [20][50/400] Time 0.349 (0.407) Data 0.001 (0.047) Loss 2.913 (0.716) Epoch: [20][60/400] Time 0.351 (0.399) Data 0.001 (0.040) Loss 3.591 (1.113) Epoch: [20][70/400] Time 0.351 (0.392) Data 0.001 (0.034) Loss 2.805 (1.384) Epoch: [20][80/400] Time 0.350 (0.387) Data 0.001 (0.030) Loss 2.635 (1.594) Epoch: [20][90/400] Time 0.351 (0.405) Data 0.001 (0.047) Loss 1.877 (1.735) Epoch: [20][100/400] Time 0.354 (0.401) Data 0.001 (0.042) Loss 2.308 (1.833) Epoch: [20][110/400] Time 0.381 (0.397) Data 0.001 (0.038) Loss 2.102 (1.879) Epoch: [20][120/400] Time 0.352 (0.393) Data 0.001 (0.035) Loss 2.467 (1.931) Epoch: [20][130/400] Time 0.363 (0.390) Data 0.000 (0.033) Loss 2.248 (1.984) Epoch: [20][140/400] Time 0.353 (0.400) Data 0.000 (0.043) Loss 2.529 (2.044) Epoch: [20][150/400] Time 0.364 (0.398) Data 0.001 (0.040) Loss 2.474 (2.089) Epoch: [20][160/400] Time 0.351 (0.395) Data 0.001 (0.037) Loss 2.442 (2.117) Epoch: [20][170/400] Time 0.363 (0.393) Data 0.000 (0.035) Loss 2.734 (2.156) Epoch: [20][180/400] Time 0.408 (0.402) Data 0.000 (0.044) Loss 2.851 (2.183) Epoch: [20][190/400] Time 0.351 (0.400) Data 0.001 (0.041) Loss 2.188 (2.217) Epoch: [20][200/400] Time 0.353 (0.397) Data 0.001 (0.039) Loss 1.985 (2.231) Epoch: [20][210/400] Time 0.352 (0.395) Data 0.000 (0.037) Loss 2.666 (2.240) Epoch: [20][220/400] Time 0.364 (0.394) Data 0.000 (0.036) Loss 2.769 (2.245) Epoch: [20][230/400] Time 0.363 (0.400) Data 0.001 (0.042) Loss 2.979 (2.259) Epoch: [20][240/400] Time 0.353 (0.399) Data 0.001 (0.041) Loss 3.191 (2.271) Epoch: [20][250/400] Time 0.350 (0.397) Data 0.001 (0.039) Loss 2.494 (2.283) Epoch: [20][260/400] Time 0.364 (0.395) Data 0.000 (0.037) Loss 2.453 (2.293) Epoch: [20][270/400] Time 0.354 (0.401) Data 0.001 (0.043) Loss 2.789 (2.306) Epoch: [20][280/400] Time 0.363 (0.400) Data 0.000 (0.041) Loss 1.968 (2.313) Epoch: [20][290/400] Time 0.360 (0.398) Data 0.001 (0.040) Loss 1.779 (2.317) Epoch: [20][300/400] Time 0.364 (0.397) Data 0.001 (0.039) Loss 2.203 (2.328) Epoch: [20][310/400] Time 0.352 (0.402) Data 0.001 (0.043) Loss 3.181 (2.337) Epoch: [20][320/400] Time 0.352 (0.400) Data 0.000 (0.042) Loss 2.559 (2.335) Epoch: [20][330/400] Time 0.363 (0.399) Data 0.000 (0.040) Loss 2.705 (2.342) Epoch: [20][340/400] Time 0.375 (0.398) Data 0.000 (0.039) Loss 2.751 (2.342) Epoch: [20][350/400] Time 0.364 (0.397) Data 0.000 (0.038) Loss 2.336 (2.344) Epoch: [20][360/400] Time 0.354 (0.401) Data 0.000 (0.042) Loss 2.553 (2.348) Epoch: [20][370/400] Time 0.362 (0.400) Data 0.000 (0.041) Loss 2.264 (2.346) Epoch: [20][380/400] Time 0.352 (0.399) Data 0.000 (0.040) Loss 2.168 (2.349) Epoch: [20][390/400] Time 0.363 (0.397) Data 0.000 (0.039) Loss 1.949 (2.354) Epoch: [20][400/400] Time 0.351 (0.401) Data 0.000 (0.042) Loss 2.349 (2.359) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.128 (0.159) Data 0.000 (0.028) Computing jaccard distance... Jaccard distance computing time cost: 18.549808740615845 ==> Statistics for epoch 21: 712 clusters Epoch: [21][10/400] Time 0.361 (0.412) Data 0.001 (0.046) Loss 0.307 (0.372) Epoch: [21][20/400] Time 0.351 (0.382) Data 0.001 (0.023) Loss 0.255 (0.347) Epoch: [21][30/400] Time 0.361 (0.374) Data 0.001 (0.016) Loss 0.242 (0.333) Epoch: [21][40/400] Time 0.361 (0.369) Data 0.000 (0.012) Loss 0.209 (0.311) Epoch: [21][50/400] Time 0.361 (0.405) Data 0.001 (0.047) Loss 2.810 (0.615) Epoch: [21][60/400] Time 0.369 (0.398) Data 0.000 (0.039) Loss 2.503 (0.985) Epoch: [21][70/400] Time 0.363 (0.392) Data 0.001 (0.033) Loss 2.820 (1.208) Epoch: [21][80/400] Time 0.375 (0.388) Data 0.000 (0.029) Loss 3.185 (1.413) Epoch: [21][90/400] Time 0.350 (0.406) Data 0.001 (0.046) Loss 2.023 (1.545) Epoch: [21][100/400] Time 0.352 (0.400) Data 0.000 (0.042) Loss 2.388 (1.639) Epoch: [21][110/400] Time 0.353 (0.396) Data 0.000 (0.038) Loss 2.053 (1.702) Epoch: [21][120/400] Time 0.353 (0.393) Data 0.000 (0.035) Loss 1.929 (1.761) Epoch: [21][130/400] Time 0.364 (0.390) Data 0.000 (0.032) Loss 2.019 (1.798) Epoch: [21][140/400] Time 0.351 (0.402) Data 0.001 (0.043) Loss 2.398 (1.841) Epoch: [21][150/400] Time 0.362 (0.399) Data 0.001 (0.041) Loss 2.625 (1.885) Epoch: [21][160/400] Time 0.353 (0.396) Data 0.000 (0.038) Loss 2.128 (1.921) Epoch: [21][170/400] Time 0.370 (0.394) Data 0.000 (0.036) Loss 2.737 (1.962) Epoch: [21][180/400] Time 0.364 (0.402) Data 0.000 (0.044) Loss 2.979 (1.998) Epoch: [21][190/400] Time 0.354 (0.400) Data 0.000 (0.042) Loss 2.289 (2.016) Epoch: [21][200/400] Time 0.353 (0.398) Data 0.000 (0.039) Loss 2.543 (2.038) Epoch: [21][210/400] Time 0.352 (0.396) Data 0.001 (0.038) Loss 1.918 (2.045) Epoch: [21][220/400] Time 0.363 (0.394) Data 0.000 (0.036) Loss 2.924 (2.060) Epoch: [21][230/400] Time 0.353 (0.400) Data 0.001 (0.042) Loss 2.544 (2.073) Epoch: [21][240/400] Time 0.362 (0.398) Data 0.000 (0.040) Loss 2.803 (2.096) Epoch: [21][250/400] Time 0.355 (0.397) Data 0.001 (0.039) Loss 2.281 (2.103) Epoch: [21][260/400] Time 0.361 (0.395) Data 0.000 (0.037) Loss 2.552 (2.116) Epoch: [21][270/400] Time 0.353 (0.401) Data 0.000 (0.043) Loss 1.931 (2.124) Epoch: [21][280/400] Time 0.353 (0.399) Data 0.001 (0.041) Loss 2.650 (2.132) Epoch: [21][290/400] Time 0.370 (0.398) Data 0.000 (0.040) Loss 1.741 (2.139) Epoch: [21][300/400] Time 0.353 (0.396) Data 0.001 (0.038) Loss 2.356 (2.148) Epoch: [21][310/400] Time 0.352 (0.401) Data 0.001 (0.043) Loss 2.661 (2.163) Epoch: [21][320/400] Time 0.363 (0.400) Data 0.001 (0.042) Loss 2.844 (2.170) Epoch: [21][330/400] Time 0.350 (0.398) Data 0.001 (0.040) Loss 2.405 (2.173) Epoch: [21][340/400] Time 0.373 (0.397) Data 0.001 (0.039) Loss 2.405 (2.181) Epoch: [21][350/400] Time 0.363 (0.396) Data 0.000 (0.038) Loss 2.124 (2.188) Epoch: [21][360/400] Time 0.367 (0.400) Data 0.001 (0.042) Loss 2.459 (2.204) Epoch: [21][370/400] Time 0.362 (0.400) Data 0.001 (0.041) Loss 2.598 (2.205) Epoch: [21][380/400] Time 0.351 (0.399) Data 0.000 (0.040) Loss 2.395 (2.213) Epoch: [21][390/400] Time 0.358 (0.398) Data 0.000 (0.039) Loss 2.266 (2.217) Epoch: [21][400/400] Time 0.362 (0.401) Data 0.001 (0.043) Loss 2.810 (2.222) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.161) Data 0.000 (0.029) Computing jaccard distance... Jaccard distance computing time cost: 18.53739333152771 ==> Statistics for epoch 22: 710 clusters Epoch: [22][10/400] Time 0.361 (0.414) Data 0.001 (0.046) Loss 0.377 (0.346) Epoch: [22][20/400] Time 0.407 (0.386) Data 0.001 (0.023) Loss 0.420 (0.336) Epoch: [22][30/400] Time 0.359 (0.375) Data 0.001 (0.016) Loss 0.235 (0.312) Epoch: [22][40/400] Time 0.361 (0.369) Data 0.000 (0.012) Loss 0.290 (0.296) Epoch: [22][50/400] Time 0.350 (0.402) Data 0.001 (0.044) Loss 2.791 (0.583) Epoch: [22][60/400] Time 0.363 (0.395) Data 0.001 (0.037) Loss 3.026 (0.926) Epoch: [22][70/400] Time 0.352 (0.390) Data 0.001 (0.031) Loss 2.184 (1.168) Epoch: [22][80/400] Time 0.354 (0.386) Data 0.001 (0.028) Loss 3.398 (1.365) Epoch: [22][90/400] Time 0.372 (0.404) Data 0.001 (0.045) Loss 2.459 (1.491) Epoch: [22][100/400] Time 0.370 (0.400) Data 0.001 (0.041) Loss 1.967 (1.560) Epoch: [22][110/400] Time 0.351 (0.395) Data 0.001 (0.037) Loss 1.832 (1.622) Epoch: [22][120/400] Time 0.350 (0.392) Data 0.001 (0.034) Loss 2.662 (1.681) Epoch: [22][130/400] Time 0.362 (0.389) Data 0.000 (0.032) Loss 2.741 (1.731) Epoch: [22][140/400] Time 0.352 (0.400) Data 0.001 (0.042) Loss 1.949 (1.783) Epoch: [22][150/400] Time 0.361 (0.397) Data 0.001 (0.039) Loss 2.409 (1.825) Epoch: [22][160/400] Time 0.350 (0.395) Data 0.001 (0.037) Loss 2.350 (1.855) Epoch: [22][170/400] Time 0.364 (0.393) Data 0.000 (0.034) Loss 2.622 (1.886) Epoch: [22][180/400] Time 0.351 (0.401) Data 0.000 (0.043) Loss 2.235 (1.911) Epoch: [22][190/400] Time 0.369 (0.399) Data 0.001 (0.041) Loss 2.155 (1.941) Epoch: [22][200/400] Time 0.365 (0.397) Data 0.001 (0.039) Loss 2.705 (1.960) Epoch: [22][210/400] Time 0.370 (0.395) Data 0.001 (0.037) Loss 1.725 (1.977) Epoch: [22][220/400] Time 0.362 (0.394) Data 0.000 (0.035) Loss 2.362 (1.992) Epoch: [22][230/400] Time 0.354 (0.400) Data 0.001 (0.041) Loss 1.859 (2.004) Epoch: [22][240/400] Time 0.351 (0.398) Data 0.001 (0.040) Loss 2.818 (2.028) Epoch: [22][250/400] Time 0.352 (0.397) Data 0.001 (0.038) Loss 2.263 (2.040) Epoch: [22][260/400] Time 0.362 (0.395) Data 0.000 (0.037) Loss 1.736 (2.044) Epoch: [22][270/400] Time 0.357 (0.401) Data 0.000 (0.042) Loss 1.936 (2.057) Epoch: [22][280/400] Time 0.352 (0.399) Data 0.001 (0.041) Loss 1.676 (2.064) Epoch: [22][290/400] Time 0.352 (0.398) Data 0.000 (0.039) Loss 2.071 (2.073) Epoch: [22][300/400] Time 0.355 (0.396) Data 0.001 (0.038) Loss 2.014 (2.073) Epoch: [22][310/400] Time 0.376 (0.401) Data 0.000 (0.043) Loss 2.728 (2.084) Epoch: [22][320/400] Time 0.354 (0.400) Data 0.001 (0.041) Loss 2.114 (2.087) Epoch: [22][330/400] Time 0.354 (0.399) Data 0.001 (0.040) Loss 2.277 (2.094) Epoch: [22][340/400] Time 0.352 (0.397) Data 0.001 (0.039) Loss 2.347 (2.098) Epoch: [22][350/400] Time 0.364 (0.396) Data 0.000 (0.038) Loss 2.331 (2.106) Epoch: [22][360/400] Time 0.352 (0.400) Data 0.000 (0.042) Loss 2.734 (2.110) Epoch: [22][370/400] Time 0.354 (0.399) Data 0.001 (0.041) Loss 2.916 (2.124) Epoch: [22][380/400] Time 0.376 (0.398) Data 0.000 (0.039) Loss 2.088 (2.131) Epoch: [22][390/400] Time 0.364 (0.397) Data 0.000 (0.038) Loss 2.388 (2.131) Epoch: [22][400/400] Time 0.354 (0.401) Data 0.000 (0.042) Loss 2.091 (2.132) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.163) Data 0.000 (0.030) Computing jaccard distance... Jaccard distance computing time cost: 18.42928433418274 ==> Statistics for epoch 23: 713 clusters Epoch: [23][10/400] Time 0.361 (0.406) Data 0.001 (0.046) Loss 0.402 (0.346) Epoch: [23][20/400] Time 0.352 (0.383) Data 0.000 (0.023) Loss 0.363 (0.328) Epoch: [23][30/400] Time 0.362 (0.373) Data 0.000 (0.016) Loss 0.231 (0.313) Epoch: [23][40/400] Time 0.363 (0.370) Data 0.000 (0.012) Loss 0.100 (0.279) Epoch: [23][50/400] Time 0.352 (0.403) Data 0.001 (0.045) Loss 2.306 (0.563) Epoch: [23][60/400] Time 0.353 (0.396) Data 0.000 (0.038) Loss 2.223 (0.922) Epoch: [23][70/400] Time 0.375 (0.390) Data 0.001 (0.032) Loss 2.137 (1.124) Epoch: [23][80/400] Time 0.363 (0.385) Data 0.001 (0.028) Loss 2.805 (1.312) Epoch: [23][90/400] Time 0.358 (0.402) Data 0.001 (0.044) Loss 1.662 (1.431) Epoch: [23][100/400] Time 0.356 (0.397) Data 0.001 (0.040) Loss 1.970 (1.493) Epoch: [23][110/400] Time 0.353 (0.393) Data 0.000 (0.036) Loss 2.019 (1.547) Epoch: [23][120/400] Time 0.351 (0.390) Data 0.001 (0.033) Loss 2.887 (1.613) Epoch: [23][130/400] Time 0.364 (0.388) Data 0.000 (0.031) Loss 1.536 (1.656) Epoch: [23][140/400] Time 0.361 (0.398) Data 0.001 (0.041) Loss 2.075 (1.701) Epoch: [23][150/400] Time 0.356 (0.396) Data 0.001 (0.038) Loss 2.588 (1.753) Epoch: [23][160/400] Time 0.364 (0.393) Data 0.001 (0.036) Loss 1.957 (1.778) Epoch: [23][170/400] Time 0.362 (0.391) Data 0.000 (0.034) Loss 2.318 (1.802) Epoch: [23][180/400] Time 0.353 (0.400) Data 0.001 (0.042) Loss 2.423 (1.826) Epoch: [23][190/400] Time 0.353 (0.397) Data 0.001 (0.040) Loss 1.775 (1.850) Epoch: [23][200/400] Time 0.372 (0.395) Data 0.001 (0.038) Loss 2.113 (1.869) Epoch: [23][210/400] Time 0.353 (0.393) Data 0.001 (0.036) Loss 1.826 (1.892) Epoch: [23][220/400] Time 0.363 (0.392) Data 0.000 (0.035) Loss 2.857 (1.915) Epoch: [23][230/400] Time 0.353 (0.398) Data 0.001 (0.041) Loss 1.979 (1.938) Epoch: [23][240/400] Time 0.373 (0.396) Data 0.001 (0.039) Loss 2.044 (1.949) Epoch: [23][250/400] Time 0.354 (0.395) Data 0.001 (0.038) Loss 1.894 (1.966) Epoch: [23][260/400] Time 0.364 (0.394) Data 0.000 (0.036) Loss 2.139 (1.984) Epoch: [23][270/400] Time 0.363 (0.399) Data 0.001 (0.042) Loss 3.073 (1.994) Epoch: [23][280/400] Time 0.353 (0.397) Data 0.000 (0.040) Loss 2.604 (2.007) Epoch: [23][290/400] Time 0.364 (0.396) Data 0.001 (0.039) Loss 2.114 (2.018) Epoch: [23][300/400] Time 0.351 (0.395) Data 0.001 (0.038) Loss 2.095 (2.021) Epoch: [23][310/400] Time 0.342 (0.399) Data 0.001 (0.042) Loss 2.046 (2.030) Epoch: [23][320/400] Time 0.354 (0.398) Data 0.001 (0.041) Loss 2.289 (2.042) Epoch: [23][330/400] Time 0.396 (0.397) Data 0.001 (0.039) Loss 2.013 (2.046) Epoch: [23][340/400] Time 0.354 (0.395) Data 0.001 (0.038) Loss 1.960 (2.052) Epoch: [23][350/400] Time 0.364 (0.394) Data 0.000 (0.037) Loss 2.948 (2.056) Epoch: [23][360/400] Time 0.381 (0.398) Data 0.001 (0.041) Loss 1.699 (2.059) Epoch: [23][370/400] Time 0.352 (0.397) Data 0.001 (0.040) Loss 1.855 (2.069) Epoch: [23][380/400] Time 0.351 (0.396) Data 0.001 (0.039) Loss 1.582 (2.073) Epoch: [23][390/400] Time 0.364 (0.395) Data 0.000 (0.038) Loss 1.826 (2.077) Epoch: [23][400/400] Time 0.350 (0.399) Data 0.001 (0.041) Loss 2.326 (2.080) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.128 (0.163) Data 0.000 (0.030) Computing jaccard distance... Jaccard distance computing time cost: 18.305078983306885 ==> Statistics for epoch 24: 716 clusters Epoch: [24][10/400] Time 0.352 (0.406) Data 0.001 (0.046) Loss 0.243 (0.322) Epoch: [24][20/400] Time 0.351 (0.380) Data 0.001 (0.023) Loss 0.231 (0.310) Epoch: [24][30/400] Time 0.361 (0.372) Data 0.001 (0.016) Loss 0.266 (0.292) Epoch: [24][40/400] Time 0.363 (0.368) Data 0.000 (0.012) Loss 0.116 (0.275) Epoch: [24][50/400] Time 0.352 (0.403) Data 0.001 (0.046) Loss 1.981 (0.547) Epoch: [24][60/400] Time 0.349 (0.395) Data 0.000 (0.039) Loss 2.227 (0.871) Epoch: [24][70/400] Time 0.361 (0.389) Data 0.001 (0.033) Loss 2.061 (1.091) Epoch: [24][80/400] Time 0.352 (0.385) Data 0.001 (0.029) Loss 2.861 (1.256) Epoch: [24][90/400] Time 0.362 (0.403) Data 0.000 (0.047) Loss 2.304 (1.391) Epoch: [24][100/400] Time 0.352 (0.398) Data 0.000 (0.042) Loss 2.466 (1.471) Epoch: [24][110/400] Time 0.355 (0.394) Data 0.001 (0.038) Loss 2.271 (1.526) Epoch: [24][120/400] Time 0.351 (0.391) Data 0.001 (0.035) Loss 1.862 (1.586) Epoch: [24][130/400] Time 0.361 (0.388) Data 0.000 (0.032) Loss 2.511 (1.648) Epoch: [24][140/400] Time 0.352 (0.399) Data 0.000 (0.042) Loss 2.394 (1.700) Epoch: [24][150/400] Time 0.350 (0.396) Data 0.001 (0.039) Loss 2.440 (1.737) Epoch: [24][160/400] Time 0.352 (0.393) Data 0.001 (0.037) Loss 2.214 (1.770) Epoch: [24][170/400] Time 0.363 (0.391) Data 0.000 (0.035) Loss 2.031 (1.803) Epoch: [24][180/400] Time 0.351 (0.399) Data 0.001 (0.042) Loss 1.688 (1.825) Epoch: [24][190/400] Time 0.351 (0.396) Data 0.001 (0.040) Loss 1.648 (1.840) Epoch: [24][200/400] Time 0.351 (0.394) Data 0.001 (0.038) Loss 1.322 (1.849) Epoch: [24][210/400] Time 0.351 (0.392) Data 0.001 (0.036) Loss 1.882 (1.859) Epoch: [24][220/400] Time 0.362 (0.390) Data 0.000 (0.035) Loss 1.609 (1.877) Epoch: [24][230/400] Time 0.362 (0.396) Data 0.001 (0.041) Loss 2.267 (1.887) Epoch: [24][240/400] Time 0.352 (0.395) Data 0.000 (0.039) Loss 2.272 (1.894) Epoch: [24][250/400] Time 0.363 (0.393) Data 0.001 (0.038) Loss 2.677 (1.914) Epoch: [24][260/400] Time 0.362 (0.392) Data 0.000 (0.036) Loss 2.411 (1.924) Epoch: [24][270/400] Time 0.353 (0.398) Data 0.000 (0.042) Loss 1.875 (1.945) Epoch: [24][280/400] Time 0.359 (0.396) Data 0.000 (0.040) Loss 2.719 (1.952) Epoch: [24][290/400] Time 0.354 (0.395) Data 0.001 (0.039) Loss 2.635 (1.964) Epoch: [24][300/400] Time 0.352 (0.394) Data 0.000 (0.038) Loss 2.588 (1.974) Epoch: [24][310/400] Time 0.371 (0.398) Data 0.000 (0.042) Loss 1.678 (1.985) Epoch: [24][320/400] Time 0.354 (0.397) Data 0.000 (0.041) Loss 1.551 (1.994) Epoch: [24][330/400] Time 0.355 (0.396) Data 0.001 (0.040) Loss 2.005 (2.000) Epoch: [24][340/400] Time 0.355 (0.395) Data 0.000 (0.038) Loss 2.369 (2.002) Epoch: [24][350/400] Time 0.364 (0.394) Data 0.000 (0.037) Loss 2.188 (2.013) Epoch: [24][360/400] Time 0.356 (0.398) Data 0.001 (0.041) Loss 2.251 (2.017) Epoch: [24][370/400] Time 0.354 (0.397) Data 0.001 (0.040) Loss 2.289 (2.024) Epoch: [24][380/400] Time 0.352 (0.396) Data 0.001 (0.039) Loss 1.841 (2.033) Epoch: [24][390/400] Time 0.363 (0.395) Data 0.000 (0.038) Loss 1.745 (2.043) Epoch: [24][400/400] Time 0.355 (0.399) Data 0.001 (0.042) Loss 2.466 (2.047) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.128 (0.159) Data 0.000 (0.028) Computing jaccard distance... Jaccard distance computing time cost: 18.23905038833618 ==> Statistics for epoch 25: 711 clusters Epoch: [25][10/400] Time 0.361 (0.423) Data 0.001 (0.062) Loss 0.292 (0.276) Epoch: [25][20/400] Time 0.351 (0.391) Data 0.001 (0.031) Loss 0.212 (0.295) Epoch: [25][30/400] Time 0.361 (0.379) Data 0.001 (0.021) Loss 0.247 (0.272) Epoch: [25][40/400] Time 0.360 (0.373) Data 0.000 (0.016) Loss 0.179 (0.253) Epoch: [25][50/400] Time 0.376 (0.405) Data 0.001 (0.047) Loss 2.275 (0.484) Epoch: [25][60/400] Time 0.362 (0.398) Data 0.000 (0.039) Loss 1.952 (0.807) Epoch: [25][70/400] Time 0.363 (0.393) Data 0.001 (0.033) Loss 2.215 (1.028) Epoch: [25][80/400] Time 0.353 (0.388) Data 0.001 (0.029) Loss 2.909 (1.224) Epoch: [25][90/400] Time 0.350 (0.405) Data 0.001 (0.045) Loss 1.771 (1.355) Epoch: [25][100/400] Time 0.352 (0.400) Data 0.000 (0.041) Loss 2.343 (1.442) Epoch: [25][110/400] Time 0.351 (0.396) Data 0.001 (0.037) Loss 1.742 (1.506) Epoch: [25][120/400] Time 0.354 (0.393) Data 0.000 (0.034) Loss 2.541 (1.553) Epoch: [25][130/400] Time 0.364 (0.390) Data 0.000 (0.032) Loss 1.868 (1.594) Epoch: [25][140/400] Time 0.350 (0.401) Data 0.001 (0.042) Loss 1.714 (1.630) Epoch: [25][150/400] Time 0.352 (0.398) Data 0.000 (0.040) Loss 2.128 (1.679) Epoch: [25][160/400] Time 0.343 (0.395) Data 0.001 (0.037) Loss 2.646 (1.721) Epoch: [25][170/400] Time 0.363 (0.393) Data 0.000 (0.035) Loss 2.862 (1.749) Epoch: [25][180/400] Time 0.352 (0.401) Data 0.000 (0.043) Loss 2.219 (1.776) Epoch: [25][190/400] Time 0.365 (0.399) Data 0.001 (0.041) Loss 1.603 (1.796) Epoch: [25][200/400] Time 0.353 (0.397) Data 0.001 (0.039) Loss 2.012 (1.815) Epoch: [25][210/400] Time 0.354 (0.395) Data 0.001 (0.037) Loss 1.765 (1.829) Epoch: [25][220/400] Time 0.356 (0.393) Data 0.000 (0.035) Loss 2.505 (1.852) Epoch: [25][230/400] Time 0.351 (0.400) Data 0.001 (0.042) Loss 1.637 (1.862) Epoch: [25][240/400] Time 0.354 (0.398) Data 0.001 (0.040) Loss 2.129 (1.877) Epoch: [25][250/400] Time 0.352 (0.396) Data 0.001 (0.038) Loss 2.527 (1.880) Epoch: [25][260/400] Time 0.360 (0.395) Data 0.000 (0.037) Loss 2.561 (1.896) Epoch: [25][270/400] Time 0.352 (0.400) Data 0.000 (0.042) Loss 2.506 (1.904) Epoch: [25][280/400] Time 0.355 (0.399) Data 0.000 (0.041) Loss 2.811 (1.924) Epoch: [25][290/400] Time 0.353 (0.397) Data 0.000 (0.039) Loss 2.267 (1.933) Epoch: [25][300/400] Time 0.353 (0.396) Data 0.000 (0.038) Loss 2.235 (1.938) Epoch: [25][310/400] Time 0.354 (0.400) Data 0.001 (0.042) Loss 2.424 (1.953) Epoch: [25][320/400] Time 0.353 (0.399) Data 0.000 (0.041) Loss 1.801 (1.958) Epoch: [25][330/400] Time 0.350 (0.397) Data 0.001 (0.040) Loss 2.150 (1.963) Epoch: [25][340/400] Time 0.350 (0.396) Data 0.001 (0.039) Loss 2.056 (1.966) Epoch: [25][350/400] Time 0.363 (0.395) Data 0.000 (0.038) Loss 2.046 (1.977) Epoch: [25][360/400] Time 0.350 (0.399) Data 0.001 (0.042) Loss 2.717 (1.983) Epoch: [25][370/400] Time 0.350 (0.398) Data 0.001 (0.040) Loss 2.238 (1.985) Epoch: [25][380/400] Time 0.357 (0.397) Data 0.001 (0.039) Loss 2.029 (1.988) Epoch: [25][390/400] Time 0.363 (0.396) Data 0.000 (0.038) Loss 2.094 (1.990) Epoch: [25][400/400] Time 0.352 (0.400) Data 0.000 (0.042) Loss 1.520 (1.993) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.160) Data 0.000 (0.027) Computing jaccard distance... Jaccard distance computing time cost: 18.398639917373657 ==> Statistics for epoch 26: 717 clusters Epoch: [26][10/400] Time 0.361 (0.422) Data 0.001 (0.057) Loss 0.278 (0.339) Epoch: [26][20/400] Time 0.367 (0.389) Data 0.001 (0.029) Loss 0.315 (0.314) Epoch: [26][30/400] Time 0.371 (0.380) Data 0.000 (0.019) Loss 0.206 (0.292) Epoch: [26][40/400] Time 0.362 (0.376) Data 0.000 (0.015) Loss 0.139 (0.264) Epoch: [26][50/400] Time 0.351 (0.410) Data 0.000 (0.049) Loss 2.654 (0.509) Epoch: [26][60/400] Time 0.354 (0.402) Data 0.001 (0.041) Loss 2.286 (0.826) Epoch: [26][70/400] Time 0.397 (0.396) Data 0.001 (0.035) Loss 2.926 (1.071) Epoch: [26][80/400] Time 0.350 (0.390) Data 0.001 (0.031) Loss 2.400 (1.231) Epoch: [26][90/400] Time 0.349 (0.405) Data 0.001 (0.046) Loss 1.698 (1.342) Epoch: [26][100/400] Time 0.352 (0.400) Data 0.001 (0.041) Loss 1.819 (1.411) Epoch: [26][110/400] Time 0.362 (0.397) Data 0.000 (0.038) Loss 1.967 (1.474) Epoch: [26][120/400] Time 0.368 (0.394) Data 0.000 (0.034) Loss 2.246 (1.522) Epoch: [26][130/400] Time 0.363 (0.391) Data 0.000 (0.032) Loss 2.159 (1.582) Epoch: [26][140/400] Time 0.372 (0.401) Data 0.001 (0.042) Loss 1.633 (1.622) Epoch: [26][150/400] Time 0.354 (0.398) Data 0.001 (0.039) Loss 1.507 (1.668) Epoch: [26][160/400] Time 0.351 (0.395) Data 0.001 (0.037) Loss 1.784 (1.707) Epoch: [26][170/400] Time 0.363 (0.393) Data 0.000 (0.035) Loss 2.864 (1.736) Epoch: [26][180/400] Time 0.350 (0.401) Data 0.000 (0.042) Loss 2.127 (1.770) Epoch: [26][190/400] Time 0.355 (0.398) Data 0.000 (0.040) Loss 2.116 (1.794) Epoch: [26][200/400] Time 0.353 (0.396) Data 0.001 (0.038) Loss 1.829 (1.818) Epoch: [26][210/400] Time 0.386 (0.395) Data 0.001 (0.036) Loss 1.824 (1.837) Epoch: [26][220/400] Time 0.371 (0.393) Data 0.000 (0.035) Loss 1.653 (1.845) Epoch: [26][230/400] Time 0.353 (0.399) Data 0.001 (0.040) Loss 2.259 (1.859) Epoch: [26][240/400] Time 0.354 (0.397) Data 0.001 (0.039) Loss 2.017 (1.872) Epoch: [26][250/400] Time 0.363 (0.396) Data 0.001 (0.037) Loss 2.357 (1.876) Epoch: [26][260/400] Time 0.364 (0.394) Data 0.000 (0.036) Loss 2.528 (1.889) Epoch: [26][270/400] Time 0.354 (0.400) Data 0.001 (0.041) Loss 2.455 (1.903) Epoch: [26][280/400] Time 0.382 (0.398) Data 0.001 (0.040) Loss 1.738 (1.918) Epoch: [26][290/400] Time 0.354 (0.397) Data 0.001 (0.038) Loss 2.629 (1.925) Epoch: [26][300/400] Time 0.355 (0.395) Data 0.001 (0.037) Loss 2.672 (1.935) Epoch: [26][310/400] Time 0.354 (0.400) Data 0.001 (0.041) Loss 2.295 (1.945) Epoch: [26][320/400] Time 0.362 (0.398) Data 0.001 (0.040) Loss 2.442 (1.954) Epoch: [26][330/400] Time 0.353 (0.397) Data 0.001 (0.039) Loss 1.867 (1.958) Epoch: [26][340/400] Time 0.353 (0.396) Data 0.001 (0.038) Loss 2.522 (1.963) Epoch: [26][350/400] Time 0.362 (0.395) Data 0.000 (0.037) Loss 2.413 (1.967) Epoch: [26][360/400] Time 0.362 (0.399) Data 0.000 (0.041) Loss 2.700 (1.974) Epoch: [26][370/400] Time 0.351 (0.398) Data 0.001 (0.040) Loss 1.509 (1.979) Epoch: [26][380/400] Time 0.362 (0.397) Data 0.001 (0.039) Loss 1.781 (1.984) Epoch: [26][390/400] Time 0.364 (0.396) Data 0.000 (0.038) Loss 2.561 (1.988) Epoch: [26][400/400] Time 0.352 (0.399) Data 0.000 (0.041) Loss 2.522 (1.992) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.158) Data 0.000 (0.026) Computing jaccard distance... Jaccard distance computing time cost: 18.381394863128662 ==> Statistics for epoch 27: 715 clusters Epoch: [27][10/400] Time 0.349 (0.426) Data 0.000 (0.064) Loss 0.274 (0.282) Epoch: [27][20/400] Time 0.351 (0.392) Data 0.000 (0.032) Loss 0.183 (0.274) Epoch: [27][30/400] Time 0.350 (0.379) Data 0.001 (0.022) Loss 0.302 (0.267) Epoch: [27][40/400] Time 0.361 (0.373) Data 0.000 (0.016) Loss 0.184 (0.259) Epoch: [27][50/400] Time 0.351 (0.405) Data 0.001 (0.047) Loss 1.973 (0.478) Epoch: [27][60/400] Time 0.389 (0.397) Data 0.000 (0.040) Loss 2.429 (0.773) Epoch: [27][70/400] Time 0.349 (0.390) Data 0.001 (0.034) Loss 2.763 (1.013) Epoch: [27][80/400] Time 0.350 (0.386) Data 0.001 (0.030) Loss 1.946 (1.165) Epoch: [27][90/400] Time 0.347 (0.403) Data 0.000 (0.047) Loss 1.607 (1.281) Epoch: [27][100/400] Time 0.352 (0.399) Data 0.000 (0.042) Loss 1.960 (1.346) Epoch: [27][110/400] Time 0.354 (0.395) Data 0.001 (0.038) Loss 2.283 (1.411) Epoch: [27][120/400] Time 0.352 (0.391) Data 0.000 (0.035) Loss 2.405 (1.471) Epoch: [27][130/400] Time 0.361 (0.389) Data 0.000 (0.033) Loss 1.659 (1.502) Epoch: [27][140/400] Time 0.363 (0.399) Data 0.001 (0.043) Loss 2.130 (1.539) Epoch: [27][150/400] Time 0.352 (0.397) Data 0.001 (0.040) Loss 2.817 (1.596) Epoch: [27][160/400] Time 0.360 (0.394) Data 0.001 (0.038) Loss 1.796 (1.631) Epoch: [27][170/400] Time 0.363 (0.392) Data 0.000 (0.035) Loss 2.564 (1.662) Epoch: [27][180/400] Time 0.388 (0.400) Data 0.000 (0.043) Loss 1.992 (1.681) Epoch: [27][190/400] Time 0.351 (0.398) Data 0.001 (0.041) Loss 2.945 (1.707) Epoch: [27][200/400] Time 0.361 (0.396) Data 0.001 (0.039) Loss 2.035 (1.733) Epoch: [27][210/400] Time 0.353 (0.394) Data 0.001 (0.037) Loss 1.779 (1.751) Epoch: [27][220/400] Time 0.362 (0.392) Data 0.000 (0.035) Loss 2.641 (1.764) Epoch: [27][230/400] Time 0.358 (0.399) Data 0.001 (0.041) Loss 1.857 (1.778) Epoch: [27][240/400] Time 0.363 (0.397) Data 0.001 (0.040) Loss 2.062 (1.798) Epoch: [27][250/400] Time 0.364 (0.396) Data 0.001 (0.038) Loss 1.770 (1.816) Epoch: [27][260/400] Time 0.364 (0.394) Data 0.000 (0.037) Loss 2.249 (1.826) Epoch: [27][270/400] Time 0.353 (0.400) Data 0.001 (0.042) Loss 1.956 (1.842) Epoch: [27][280/400] Time 0.354 (0.398) Data 0.001 (0.041) Loss 2.069 (1.848) Epoch: [27][290/400] Time 0.365 (0.397) Data 0.000 (0.039) Loss 2.086 (1.858) Epoch: [27][300/400] Time 0.353 (0.395) Data 0.001 (0.038) Loss 2.112 (1.876) Epoch: [27][310/400] Time 0.373 (0.400) Data 0.001 (0.043) Loss 2.207 (1.885) Epoch: [27][320/400] Time 0.353 (0.399) Data 0.001 (0.041) Loss 2.382 (1.893) Epoch: [27][330/400] Time 0.363 (0.398) Data 0.001 (0.040) Loss 2.760 (1.903) Epoch: [27][340/400] Time 0.363 (0.397) Data 0.001 (0.039) Loss 1.652 (1.907) Epoch: [27][350/400] Time 0.362 (0.396) Data 0.000 (0.038) Loss 1.640 (1.910) Epoch: [27][360/400] Time 0.353 (0.400) Data 0.000 (0.042) Loss 1.722 (1.917) Epoch: [27][370/400] Time 0.362 (0.399) Data 0.001 (0.041) Loss 2.274 (1.923) Epoch: [27][380/400] Time 0.352 (0.398) Data 0.000 (0.039) Loss 1.876 (1.924) Epoch: [27][390/400] Time 0.364 (0.397) Data 0.000 (0.038) Loss 1.737 (1.925) Epoch: [27][400/400] Time 0.363 (0.401) Data 0.000 (0.042) Loss 1.736 (1.924) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.163) Data 0.000 (0.030) Computing jaccard distance... Jaccard distance computing time cost: 18.326934814453125 ==> Statistics for epoch 28: 716 clusters Epoch: [28][10/400] Time 0.351 (0.419) Data 0.001 (0.062) Loss 0.284 (0.315) Epoch: [28][20/400] Time 0.351 (0.388) Data 0.000 (0.031) Loss 0.369 (0.285) Epoch: [28][30/400] Time 0.407 (0.378) Data 0.001 (0.021) Loss 0.280 (0.267) Epoch: [28][40/400] Time 0.364 (0.374) Data 0.000 (0.016) Loss 0.228 (0.246) Epoch: [28][50/400] Time 0.362 (0.407) Data 0.001 (0.048) Loss 2.438 (0.495) Epoch: [28][60/400] Time 0.351 (0.398) Data 0.001 (0.040) Loss 2.782 (0.831) Epoch: [28][70/400] Time 0.351 (0.392) Data 0.000 (0.034) Loss 1.735 (1.024) Epoch: [28][80/400] Time 0.351 (0.387) Data 0.000 (0.030) Loss 2.260 (1.188) Epoch: [28][90/400] Time 0.349 (0.403) Data 0.001 (0.046) Loss 2.026 (1.285) Epoch: [28][100/400] Time 0.387 (0.399) Data 0.000 (0.042) Loss 1.900 (1.355) Epoch: [28][110/400] Time 0.350 (0.395) Data 0.001 (0.038) Loss 2.403 (1.432) Epoch: [28][120/400] Time 0.363 (0.392) Data 0.000 (0.035) Loss 2.064 (1.485) Epoch: [28][130/400] Time 0.361 (0.389) Data 0.000 (0.032) Loss 2.202 (1.525) Epoch: [28][140/400] Time 0.350 (0.399) Data 0.001 (0.042) Loss 2.244 (1.571) Epoch: [28][150/400] Time 0.356 (0.396) Data 0.001 (0.039) Loss 1.948 (1.615) Epoch: [28][160/400] Time 0.361 (0.394) Data 0.001 (0.037) Loss 2.378 (1.651) Epoch: [28][170/400] Time 0.362 (0.391) Data 0.000 (0.035) Loss 2.072 (1.680) Epoch: [28][180/400] Time 0.354 (0.400) Data 0.000 (0.043) Loss 1.663 (1.698) Epoch: [28][190/400] Time 0.361 (0.397) Data 0.000 (0.041) Loss 1.754 (1.698) Epoch: [28][200/400] Time 0.350 (0.395) Data 0.000 (0.039) Loss 1.721 (1.711) Epoch: [28][210/400] Time 0.353 (0.393) Data 0.000 (0.037) Loss 2.144 (1.737) Epoch: [28][220/400] Time 0.363 (0.392) Data 0.000 (0.035) Loss 2.250 (1.753) Epoch: [28][230/400] Time 0.352 (0.398) Data 0.001 (0.041) Loss 2.690 (1.768) Epoch: [28][240/400] Time 0.352 (0.396) Data 0.001 (0.040) Loss 1.952 (1.783) Epoch: [28][250/400] Time 0.353 (0.395) Data 0.001 (0.038) Loss 1.914 (1.796) Epoch: [28][260/400] Time 0.363 (0.393) Data 0.000 (0.037) Loss 2.043 (1.809) Epoch: [28][270/400] Time 0.352 (0.399) Data 0.001 (0.042) Loss 2.725 (1.821) Epoch: [28][280/400] Time 0.353 (0.397) Data 0.001 (0.041) Loss 2.246 (1.827) Epoch: [28][290/400] Time 0.353 (0.396) Data 0.001 (0.039) Loss 2.356 (1.835) Epoch: [28][300/400] Time 0.352 (0.395) Data 0.001 (0.038) Loss 2.861 (1.846) Epoch: [28][310/400] Time 0.377 (0.400) Data 0.000 (0.043) Loss 1.423 (1.850) Epoch: [28][320/400] Time 0.353 (0.399) Data 0.000 (0.041) Loss 3.093 (1.859) Epoch: [28][330/400] Time 0.352 (0.397) Data 0.000 (0.040) Loss 2.111 (1.865) Epoch: [28][340/400] Time 0.355 (0.396) Data 0.001 (0.039) Loss 2.292 (1.875) Epoch: [28][350/400] Time 0.354 (0.395) Data 0.000 (0.038) Loss 2.111 (1.882) Epoch: [28][360/400] Time 0.354 (0.399) Data 0.000 (0.042) Loss 2.014 (1.888) Epoch: [28][370/400] Time 0.364 (0.398) Data 0.001 (0.041) Loss 2.212 (1.896) Epoch: [28][380/400] Time 0.353 (0.397) Data 0.001 (0.040) Loss 1.227 (1.898) Epoch: [28][390/400] Time 0.364 (0.396) Data 0.000 (0.039) Loss 1.393 (1.893) Epoch: [28][400/400] Time 0.352 (0.400) Data 0.000 (0.042) Loss 1.566 (1.893) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.161) Data 0.000 (0.030) Computing jaccard distance... Jaccard distance computing time cost: 18.5582857131958 ==> Statistics for epoch 29: 716 clusters Epoch: [29][10/400] Time 0.351 (0.406) Data 0.001 (0.046) Loss 0.328 (0.281) Epoch: [29][20/400] Time 0.350 (0.381) Data 0.001 (0.023) Loss 0.215 (0.274) Epoch: [29][30/400] Time 0.352 (0.371) Data 0.001 (0.016) Loss 0.280 (0.259) Epoch: [29][40/400] Time 0.361 (0.369) Data 0.000 (0.012) Loss 0.286 (0.245) Epoch: [29][50/400] Time 0.350 (0.401) Data 0.001 (0.045) Loss 2.234 (0.490) Epoch: [29][60/400] Time 0.351 (0.394) Data 0.001 (0.037) Loss 1.989 (0.813) Epoch: [29][70/400] Time 0.348 (0.388) Data 0.001 (0.032) Loss 1.999 (0.983) Epoch: [29][80/400] Time 0.352 (0.383) Data 0.001 (0.028) Loss 2.434 (1.169) Epoch: [29][90/400] Time 0.350 (0.401) Data 0.001 (0.045) Loss 1.446 (1.269) Epoch: [29][100/400] Time 0.351 (0.396) Data 0.000 (0.040) Loss 2.144 (1.329) Epoch: [29][110/400] Time 0.362 (0.393) Data 0.001 (0.037) Loss 1.831 (1.385) Epoch: [29][120/400] Time 0.351 (0.389) Data 0.001 (0.034) Loss 2.147 (1.436) Epoch: [29][130/400] Time 0.361 (0.387) Data 0.000 (0.031) Loss 2.310 (1.479) Epoch: [29][140/400] Time 0.351 (0.399) Data 0.000 (0.042) Loss 2.493 (1.509) Epoch: [29][150/400] Time 0.360 (0.396) Data 0.001 (0.040) Loss 2.518 (1.551) Epoch: [29][160/400] Time 0.351 (0.393) Data 0.000 (0.037) Loss 2.251 (1.587) Epoch: [29][170/400] Time 0.361 (0.391) Data 0.000 (0.035) Loss 1.932 (1.617) Epoch: [29][180/400] Time 0.352 (0.400) Data 0.001 (0.043) Loss 2.165 (1.648) Epoch: [29][190/400] Time 0.359 (0.397) Data 0.001 (0.041) Loss 1.469 (1.657) Epoch: [29][200/400] Time 0.363 (0.395) Data 0.001 (0.039) Loss 2.036 (1.675) Epoch: [29][210/400] Time 0.352 (0.393) Data 0.001 (0.037) Loss 2.396 (1.698) Epoch: [29][220/400] Time 0.361 (0.392) Data 0.000 (0.036) Loss 1.767 (1.726) Epoch: [29][230/400] Time 0.362 (0.398) Data 0.001 (0.042) Loss 1.820 (1.743) Epoch: [29][240/400] Time 0.363 (0.397) Data 0.001 (0.040) Loss 2.084 (1.754) Epoch: [29][250/400] Time 0.362 (0.395) Data 0.000 (0.038) Loss 2.524 (1.769) Epoch: [29][260/400] Time 0.362 (0.394) Data 0.000 (0.037) Loss 1.901 (1.776) Epoch: [29][270/400] Time 0.361 (0.399) Data 0.001 (0.042) Loss 1.845 (1.784) Epoch: [29][280/400] Time 0.351 (0.398) Data 0.001 (0.041) Loss 2.507 (1.793) Epoch: [29][290/400] Time 0.392 (0.396) Data 0.000 (0.039) Loss 2.691 (1.804) Epoch: [29][300/400] Time 0.352 (0.395) Data 0.001 (0.038) Loss 2.069 (1.808) Epoch: [29][310/400] Time 0.349 (0.400) Data 0.001 (0.042) Loss 2.542 (1.815) Epoch: [29][320/400] Time 0.353 (0.398) Data 0.000 (0.041) Loss 1.929 (1.828) Epoch: [29][330/400] Time 0.352 (0.397) Data 0.001 (0.040) Loss 2.480 (1.829) Epoch: [29][340/400] Time 0.351 (0.395) Data 0.000 (0.039) Loss 2.125 (1.837) Epoch: [29][350/400] Time 0.363 (0.394) Data 0.000 (0.038) Loss 2.146 (1.839) Epoch: [29][360/400] Time 0.362 (0.398) Data 0.000 (0.041) Loss 2.462 (1.841) Epoch: [29][370/400] Time 0.353 (0.397) Data 0.000 (0.040) Loss 1.931 (1.845) Epoch: [29][380/400] Time 0.364 (0.396) Data 0.000 (0.039) Loss 1.925 (1.857) Epoch: [29][390/400] Time 0.361 (0.395) Data 0.000 (0.038) Loss 3.028 (1.863) Epoch: [29][400/400] Time 0.351 (0.399) Data 0.000 (0.042) Loss 2.177 (1.871) Extract Features: [50/76] Time 0.133 (0.161) Data 0.000 (0.028) Mean AP: 85.5% * Finished epoch 29 model mAP: 85.5% best: 85.5% * ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.128 (0.163) Data 0.000 (0.031) Computing jaccard distance... Jaccard distance computing time cost: 18.52395009994507 ==> Statistics for epoch 30: 710 clusters Epoch: [30][10/400] Time 0.361 (0.418) Data 0.001 (0.059) Loss 0.307 (0.259) Epoch: [30][20/400] Time 0.347 (0.388) Data 0.001 (0.030) Loss 0.396 (0.269) Epoch: [30][30/400] Time 0.360 (0.376) Data 0.001 (0.020) Loss 0.201 (0.252) Epoch: [30][40/400] Time 0.363 (0.371) Data 0.000 (0.015) Loss 0.162 (0.230) Epoch: [30][50/400] Time 0.377 (0.405) Data 0.001 (0.048) Loss 2.497 (0.468) Epoch: [30][60/400] Time 0.353 (0.397) Data 0.001 (0.040) Loss 2.282 (0.765) Epoch: [30][70/400] Time 0.351 (0.391) Data 0.001 (0.034) Loss 2.432 (0.965) Epoch: [30][80/400] Time 0.352 (0.387) Data 0.001 (0.030) Loss 1.586 (1.096) Epoch: [30][90/400] Time 0.350 (0.403) Data 0.001 (0.046) Loss 1.397 (1.226) Epoch: [30][100/400] Time 0.352 (0.399) Data 0.001 (0.041) Loss 2.194 (1.313) Epoch: [30][110/400] Time 0.352 (0.395) Data 0.001 (0.038) Loss 2.427 (1.394) Epoch: [30][120/400] Time 0.352 (0.391) Data 0.001 (0.035) Loss 2.159 (1.432) Epoch: [30][130/400] Time 0.363 (0.389) Data 0.000 (0.032) Loss 1.832 (1.465) Epoch: [30][140/400] Time 0.354 (0.399) Data 0.001 (0.043) Loss 1.944 (1.514) Epoch: [30][150/400] Time 0.378 (0.397) Data 0.001 (0.040) Loss 2.359 (1.553) Epoch: [30][160/400] Time 0.353 (0.394) Data 0.001 (0.037) Loss 2.040 (1.589) Epoch: [30][170/400] Time 0.363 (0.392) Data 0.000 (0.035) Loss 1.772 (1.607) Epoch: [30][180/400] Time 0.349 (0.401) Data 0.001 (0.043) Loss 2.023 (1.634) Epoch: [30][190/400] Time 0.361 (0.399) Data 0.001 (0.041) Loss 2.247 (1.667) Epoch: [30][200/400] Time 0.359 (0.397) Data 0.000 (0.039) Loss 2.367 (1.683) Epoch: [30][210/400] Time 0.353 (0.394) Data 0.001 (0.037) Loss 1.764 (1.690) Epoch: [30][220/400] Time 0.363 (0.393) Data 0.000 (0.036) Loss 1.734 (1.707) Epoch: [30][230/400] Time 0.352 (0.399) Data 0.001 (0.042) Loss 1.559 (1.719) Epoch: [30][240/400] Time 0.371 (0.397) Data 0.000 (0.040) Loss 1.811 (1.729) Epoch: [30][250/400] Time 0.351 (0.396) Data 0.001 (0.039) Loss 1.912 (1.738) Epoch: [30][260/400] Time 0.363 (0.394) Data 0.000 (0.037) Loss 2.044 (1.754) Epoch: [30][270/400] Time 0.363 (0.400) Data 0.001 (0.042) Loss 1.696 (1.759) Epoch: [30][280/400] Time 0.362 (0.398) Data 0.001 (0.041) Loss 2.116 (1.765) Epoch: [30][290/400] Time 0.387 (0.397) Data 0.001 (0.039) Loss 2.248 (1.775) Epoch: [30][300/400] Time 0.353 (0.396) Data 0.000 (0.038) Loss 1.655 (1.780) Epoch: [30][310/400] Time 0.348 (0.400) Data 0.001 (0.043) Loss 2.580 (1.782) Epoch: [30][320/400] Time 0.351 (0.399) Data 0.001 (0.041) Loss 2.295 (1.795) Epoch: [30][330/400] Time 0.352 (0.398) Data 0.001 (0.040) Loss 1.989 (1.806) Epoch: [30][340/400] Time 0.350 (0.396) Data 0.001 (0.039) Loss 2.025 (1.815) Epoch: [30][350/400] Time 0.363 (0.395) Data 0.000 (0.038) Loss 2.185 (1.818) Epoch: [30][360/400] Time 0.354 (0.400) Data 0.001 (0.042) Loss 1.556 (1.825) Epoch: [30][370/400] Time 0.356 (0.398) Data 0.001 (0.041) Loss 2.031 (1.831) Epoch: [30][380/400] Time 0.358 (0.397) Data 0.001 (0.040) Loss 2.201 (1.838) Epoch: [30][390/400] Time 0.362 (0.396) Data 0.000 (0.039) Loss 2.389 (1.841) Epoch: [30][400/400] Time 0.362 (0.400) Data 0.001 (0.042) Loss 1.868 (1.843) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.128 (0.160) Data 0.000 (0.027) Computing jaccard distance... Jaccard distance computing time cost: 18.330976486206055 ==> Statistics for epoch 31: 711 clusters Epoch: [31][10/400] Time 0.362 (0.414) Data 0.001 (0.048) Loss 0.196 (0.252) Epoch: [31][20/400] Time 0.352 (0.384) Data 0.000 (0.024) Loss 0.212 (0.261) Epoch: [31][30/400] Time 0.362 (0.374) Data 0.001 (0.016) Loss 0.156 (0.238) Epoch: [31][40/400] Time 0.362 (0.369) Data 0.000 (0.012) Loss 0.294 (0.225) Epoch: [31][50/400] Time 0.362 (0.405) Data 0.001 (0.047) Loss 2.292 (0.480) Epoch: [31][60/400] Time 0.350 (0.397) Data 0.000 (0.040) Loss 1.765 (0.751) Epoch: [31][70/400] Time 0.361 (0.391) Data 0.001 (0.034) Loss 2.759 (0.987) Epoch: [31][80/400] Time 0.349 (0.387) Data 0.001 (0.030) Loss 2.231 (1.135) Epoch: [31][90/400] Time 0.371 (0.404) Data 0.001 (0.046) Loss 2.243 (1.232) Epoch: [31][100/400] Time 0.353 (0.400) Data 0.001 (0.042) Loss 2.106 (1.305) Epoch: [31][110/400] Time 0.354 (0.396) Data 0.001 (0.038) Loss 1.471 (1.351) Epoch: [31][120/400] Time 0.354 (0.393) Data 0.001 (0.035) Loss 2.216 (1.404) Epoch: [31][130/400] Time 0.361 (0.390) Data 0.000 (0.032) Loss 1.866 (1.435) Epoch: [31][140/400] Time 0.361 (0.402) Data 0.001 (0.043) Loss 1.570 (1.471) Epoch: [31][150/400] Time 0.355 (0.399) Data 0.001 (0.040) Loss 2.506 (1.502) Epoch: [31][160/400] Time 0.351 (0.396) Data 0.001 (0.038) Loss 2.229 (1.547) Epoch: [31][170/400] Time 0.362 (0.394) Data 0.000 (0.036) Loss 1.987 (1.569) Epoch: [31][180/400] Time 0.361 (0.403) Data 0.000 (0.044) Loss 1.843 (1.591) Epoch: [31][190/400] Time 0.355 (0.400) Data 0.000 (0.042) Loss 2.522 (1.617) Epoch: [31][200/400] Time 0.399 (0.398) Data 0.000 (0.040) Loss 2.548 (1.639) Epoch: [31][210/400] Time 0.353 (0.396) Data 0.001 (0.038) Loss 2.205 (1.655) Epoch: [31][220/400] Time 0.360 (0.395) Data 0.000 (0.036) Loss 1.400 (1.673) Epoch: [31][230/400] Time 0.353 (0.401) Data 0.001 (0.043) Loss 2.368 (1.699) Epoch: [31][240/400] Time 0.351 (0.399) Data 0.001 (0.041) Loss 1.774 (1.715) Epoch: [31][250/400] Time 0.351 (0.397) Data 0.001 (0.039) Loss 1.778 (1.731) Epoch: [31][260/400] Time 0.363 (0.396) Data 0.000 (0.038) Loss 2.269 (1.746) Epoch: [31][270/400] Time 0.363 (0.401) Data 0.001 (0.043) Loss 1.831 (1.754) Epoch: [31][280/400] Time 0.353 (0.400) Data 0.001 (0.041) Loss 2.091 (1.762) Epoch: [31][290/400] Time 0.363 (0.398) Data 0.001 (0.040) Loss 1.713 (1.762) Epoch: [31][300/400] Time 0.351 (0.397) Data 0.001 (0.039) Loss 2.192 (1.768) Epoch: [31][310/400] Time 0.367 (0.402) Data 0.001 (0.043) Loss 2.388 (1.783) Epoch: [31][320/400] Time 0.351 (0.400) Data 0.001 (0.042) Loss 1.956 (1.789) Epoch: [31][330/400] Time 0.371 (0.399) Data 0.001 (0.041) Loss 2.460 (1.797) Epoch: [31][340/400] Time 0.351 (0.398) Data 0.001 (0.040) Loss 1.812 (1.803) Epoch: [31][350/400] Time 0.361 (0.396) Data 0.000 (0.038) Loss 2.318 (1.803) Epoch: [31][360/400] Time 0.349 (0.400) Data 0.000 (0.042) Loss 2.678 (1.812) Epoch: [31][370/400] Time 0.362 (0.399) Data 0.000 (0.041) Loss 1.420 (1.813) Epoch: [31][380/400] Time 0.363 (0.398) Data 0.001 (0.040) Loss 1.399 (1.816) Epoch: [31][390/400] Time 0.363 (0.397) Data 0.000 (0.039) Loss 2.059 (1.822) Epoch: [31][400/400] Time 0.361 (0.401) Data 0.000 (0.042) Loss 1.773 (1.827) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.127 (0.161) Data 0.000 (0.028) Computing jaccard distance... Jaccard distance computing time cost: 18.22784423828125 ==> Statistics for epoch 32: 710 clusters Epoch: [32][10/400] Time 0.359 (0.426) Data 0.001 (0.063) Loss 0.265 (0.267) Epoch: [32][20/400] Time 0.350 (0.391) Data 0.001 (0.032) Loss 0.357 (0.278) Epoch: [32][30/400] Time 0.362 (0.379) Data 0.001 (0.021) Loss 0.174 (0.252) Epoch: [32][40/400] Time 0.360 (0.374) Data 0.000 (0.016) Loss 0.213 (0.232) Epoch: [32][50/400] Time 0.350 (0.406) Data 0.000 (0.046) Loss 2.206 (0.470) Epoch: [32][60/400] Time 0.349 (0.396) Data 0.000 (0.039) Loss 2.402 (0.775) Epoch: [32][70/400] Time 0.350 (0.390) Data 0.000 (0.033) Loss 1.651 (0.977) Epoch: [32][80/400] Time 0.351 (0.385) Data 0.000 (0.029) Loss 1.936 (1.143) Epoch: [32][90/400] Time 0.364 (0.401) Data 0.000 (0.045) Loss 2.383 (1.264) Epoch: [32][100/400] Time 0.352 (0.396) Data 0.000 (0.040) Loss 2.375 (1.334) Epoch: [32][110/400] Time 0.352 (0.393) Data 0.000 (0.037) Loss 1.922 (1.394) Epoch: [32][120/400] Time 0.375 (0.390) Data 0.000 (0.034) Loss 2.081 (1.440) Epoch: [32][130/400] Time 0.360 (0.387) Data 0.000 (0.031) Loss 1.508 (1.473) Epoch: [32][140/400] Time 0.351 (0.397) Data 0.001 (0.041) Loss 2.418 (1.515) Epoch: [32][150/400] Time 0.351 (0.394) Data 0.001 (0.038) Loss 1.810 (1.544) Epoch: [32][160/400] Time 0.351 (0.391) Data 0.000 (0.036) Loss 1.789 (1.564) Epoch: [32][170/400] Time 0.363 (0.389) Data 0.000 (0.034) Loss 1.595 (1.586) Epoch: [32][180/400] Time 0.354 (0.397) Data 0.000 (0.041) Loss 2.328 (1.606) Epoch: [32][190/400] Time 0.362 (0.395) Data 0.001 (0.039) Loss 1.934 (1.626) Epoch: [32][200/400] Time 0.353 (0.393) Data 0.000 (0.037) Loss 1.853 (1.642) Epoch: [32][210/400] Time 0.354 (0.391) Data 0.000 (0.035) Loss 2.073 (1.664) Epoch: [32][220/400] Time 0.362 (0.390) Data 0.000 (0.034) Loss 1.993 (1.673) Epoch: [32][230/400] Time 0.389 (0.397) Data 0.000 (0.040) Loss 2.494 (1.689) Epoch: [32][240/400] Time 0.352 (0.395) Data 0.000 (0.038) Loss 1.904 (1.701) Epoch: [32][250/400] Time 0.358 (0.393) Data 0.001 (0.037) Loss 2.270 (1.713) Epoch: [32][260/400] Time 0.363 (0.392) Data 0.000 (0.035) Loss 1.670 (1.720) Epoch: [32][270/400] Time 0.352 (0.397) Data 0.001 (0.041) Loss 1.345 (1.724) Epoch: [32][280/400] Time 0.361 (0.396) Data 0.000 (0.039) Loss 2.200 (1.732) Epoch: [32][290/400] Time 0.350 (0.394) Data 0.001 (0.038) Loss 1.404 (1.735) Epoch: [32][300/400] Time 0.374 (0.393) Data 0.000 (0.037) Loss 1.726 (1.743) Epoch: [32][310/400] Time 0.364 (0.398) Data 0.000 (0.041) Loss 1.977 (1.748) Epoch: [32][320/400] Time 0.368 (0.397) Data 0.000 (0.040) Loss 1.587 (1.754) Epoch: [32][330/400] Time 0.352 (0.395) Data 0.000 (0.039) Loss 1.916 (1.757) Epoch: [32][340/400] Time 0.352 (0.394) Data 0.000 (0.038) Loss 2.608 (1.764) Epoch: [32][350/400] Time 0.363 (0.393) Data 0.000 (0.037) Loss 2.680 (1.771) Epoch: [32][360/400] Time 0.352 (0.397) Data 0.000 (0.041) Loss 2.394 (1.779) Epoch: [32][370/400] Time 0.362 (0.396) Data 0.001 (0.039) Loss 3.304 (1.787) Epoch: [32][380/400] Time 0.352 (0.395) Data 0.000 (0.038) Loss 1.731 (1.789) Epoch: [32][390/400] Time 0.363 (0.394) Data 0.000 (0.037) Loss 1.859 (1.793) Epoch: [32][400/400] Time 0.353 (0.398) Data 0.000 (0.041) Loss 2.488 (1.799) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.130 (0.156) Data 0.000 (0.028) Computing jaccard distance... Jaccard distance computing time cost: 18.366796493530273 ==> Statistics for epoch 33: 709 clusters Epoch: [33][10/400] Time 0.361 (0.426) Data 0.001 (0.063) Loss 0.194 (0.253) Epoch: [33][20/400] Time 0.375 (0.393) Data 0.001 (0.032) Loss 0.195 (0.239) Epoch: [33][30/400] Time 0.359 (0.380) Data 0.001 (0.021) Loss 0.146 (0.227) Epoch: [33][40/400] Time 0.362 (0.374) Data 0.000 (0.016) Loss 0.162 (0.217) Epoch: [33][50/400] Time 0.361 (0.406) Data 0.000 (0.047) Loss 2.018 (0.450) Epoch: [33][60/400] Time 0.350 (0.398) Data 0.001 (0.040) Loss 2.206 (0.727) Epoch: [33][70/400] Time 0.351 (0.392) Data 0.001 (0.034) Loss 2.221 (0.939) Epoch: [33][80/400] Time 0.353 (0.388) Data 0.001 (0.030) Loss 2.421 (1.089) Epoch: [33][90/400] Time 0.372 (0.404) Data 0.000 (0.046) Loss 2.164 (1.189) Epoch: [33][100/400] Time 0.349 (0.399) Data 0.000 (0.042) Loss 1.647 (1.266) Epoch: [33][110/400] Time 0.367 (0.396) Data 0.000 (0.038) Loss 1.675 (1.326) Epoch: [33][120/400] Time 0.350 (0.392) Data 0.001 (0.035) Loss 2.078 (1.372) Epoch: [33][130/400] Time 0.362 (0.389) Data 0.000 (0.032) Loss 2.040 (1.409) Epoch: [33][140/400] Time 0.355 (0.400) Data 0.001 (0.042) Loss 2.060 (1.459) Epoch: [33][150/400] Time 0.403 (0.397) Data 0.000 (0.039) Loss 2.235 (1.484) Epoch: [33][160/400] Time 0.354 (0.394) Data 0.000 (0.037) Loss 2.016 (1.525) Epoch: [33][170/400] Time 0.362 (0.392) Data 0.000 (0.035) Loss 1.760 (1.552) Epoch: [33][180/400] Time 0.352 (0.401) Data 0.001 (0.043) Loss 2.339 (1.577) Epoch: [33][190/400] Time 0.353 (0.398) Data 0.001 (0.041) Loss 1.591 (1.586) Epoch: [33][200/400] Time 0.354 (0.396) Data 0.001 (0.039) Loss 2.400 (1.611) Epoch: [33][210/400] Time 0.354 (0.395) Data 0.001 (0.037) Loss 2.234 (1.627) Epoch: [33][220/400] Time 0.364 (0.393) Data 0.000 (0.035) Loss 2.739 (1.644) Epoch: [33][230/400] Time 0.364 (0.400) Data 0.001 (0.042) Loss 3.010 (1.659) Epoch: [33][240/400] Time 0.352 (0.398) Data 0.001 (0.040) Loss 2.128 (1.670) Epoch: [33][250/400] Time 0.362 (0.396) Data 0.001 (0.038) Loss 1.944 (1.685) Epoch: [33][260/400] Time 0.364 (0.394) Data 0.000 (0.037) Loss 1.506 (1.695) Epoch: [33][270/400] Time 0.349 (0.400) Data 0.001 (0.042) Loss 1.854 (1.707) Epoch: [33][280/400] Time 0.402 (0.398) Data 0.001 (0.041) Loss 1.700 (1.716) Epoch: [33][290/400] Time 0.351 (0.397) Data 0.001 (0.039) Loss 2.549 (1.728) Epoch: [33][300/400] Time 0.352 (0.395) Data 0.001 (0.038) Loss 1.660 (1.737) Epoch: [33][310/400] Time 0.363 (0.400) Data 0.001 (0.043) Loss 2.257 (1.740) Epoch: [33][320/400] Time 0.352 (0.399) Data 0.001 (0.041) Loss 1.842 (1.742) Epoch: [33][330/400] Time 0.353 (0.397) Data 0.001 (0.040) Loss 2.136 (1.749) Epoch: [33][340/400] Time 0.352 (0.396) Data 0.001 (0.039) Loss 1.497 (1.756) Epoch: [33][350/400] Time 0.363 (0.395) Data 0.000 (0.038) Loss 1.399 (1.760) Epoch: [33][360/400] Time 0.352 (0.399) Data 0.001 (0.042) Loss 1.820 (1.764) Epoch: [33][370/400] Time 0.364 (0.398) Data 0.001 (0.041) Loss 1.795 (1.765) Epoch: [33][380/400] Time 0.353 (0.397) Data 0.001 (0.040) Loss 1.812 (1.769) Epoch: [33][390/400] Time 0.364 (0.396) Data 0.000 (0.039) Loss 1.959 (1.773) Epoch: [33][400/400] Time 0.371 (0.400) Data 0.000 (0.042) Loss 2.447 (1.777) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.163) Data 0.000 (0.029) Computing jaccard distance... Jaccard distance computing time cost: 18.287232637405396 ==> Statistics for epoch 34: 711 clusters Epoch: [34][10/400] Time 0.362 (0.422) Data 0.001 (0.052) Loss 0.150 (0.241) Epoch: [34][20/400] Time 0.372 (0.391) Data 0.001 (0.026) Loss 0.269 (0.236) Epoch: [34][30/400] Time 0.363 (0.379) Data 0.001 (0.018) Loss 0.237 (0.217) Epoch: [34][40/400] Time 0.363 (0.374) Data 0.000 (0.013) Loss 0.092 (0.207) Epoch: [34][50/400] Time 0.362 (0.406) Data 0.001 (0.045) Loss 2.076 (0.432) Epoch: [34][60/400] Time 0.350 (0.399) Data 0.001 (0.038) Loss 2.680 (0.712) Epoch: [34][70/400] Time 0.352 (0.392) Data 0.001 (0.033) Loss 1.997 (0.898) Epoch: [34][80/400] Time 0.361 (0.388) Data 0.001 (0.029) Loss 1.830 (1.044) Epoch: [34][90/400] Time 0.363 (0.406) Data 0.001 (0.046) Loss 1.457 (1.154) Epoch: [34][100/400] Time 0.352 (0.401) Data 0.000 (0.041) Loss 1.947 (1.216) Epoch: [34][110/400] Time 0.354 (0.397) Data 0.001 (0.038) Loss 1.752 (1.268) Epoch: [34][120/400] Time 0.353 (0.394) Data 0.001 (0.035) Loss 2.355 (1.320) Epoch: [34][130/400] Time 0.363 (0.391) Data 0.000 (0.032) Loss 1.995 (1.356) Epoch: [34][140/400] Time 0.354 (0.401) Data 0.001 (0.042) Loss 1.517 (1.387) Epoch: [34][150/400] Time 0.374 (0.398) Data 0.001 (0.039) Loss 2.148 (1.429) Epoch: [34][160/400] Time 0.353 (0.395) Data 0.001 (0.036) Loss 2.262 (1.462) Epoch: [34][170/400] Time 0.363 (0.393) Data 0.000 (0.034) Loss 1.645 (1.477) Epoch: [34][180/400] Time 0.361 (0.401) Data 0.001 (0.042) Loss 1.650 (1.507) Epoch: [34][190/400] Time 0.358 (0.399) Data 0.001 (0.040) Loss 1.566 (1.532) Epoch: [34][200/400] Time 0.358 (0.396) Data 0.001 (0.038) Loss 2.212 (1.550) Epoch: [34][210/400] Time 0.355 (0.395) Data 0.001 (0.036) Loss 1.762 (1.563) Epoch: [34][220/400] Time 0.363 (0.393) Data 0.000 (0.034) Loss 1.768 (1.576) Epoch: [34][230/400] Time 0.356 (0.400) Data 0.001 (0.041) Loss 1.811 (1.591) Epoch: [34][240/400] Time 0.362 (0.398) Data 0.001 (0.039) Loss 1.862 (1.601) Epoch: [34][250/400] Time 0.362 (0.396) Data 0.001 (0.038) Loss 1.504 (1.617) Epoch: [34][260/400] Time 0.362 (0.395) Data 0.000 (0.036) Loss 1.400 (1.625) Epoch: [34][270/400] Time 0.352 (0.400) Data 0.001 (0.041) Loss 2.275 (1.635) Epoch: [34][280/400] Time 0.363 (0.398) Data 0.001 (0.040) Loss 1.721 (1.641) Epoch: [34][290/400] Time 0.351 (0.397) Data 0.001 (0.038) Loss 1.197 (1.651) Epoch: [34][300/400] Time 0.354 (0.395) Data 0.001 (0.037) Loss 1.492 (1.653) Epoch: [34][310/400] Time 0.352 (0.400) Data 0.001 (0.041) Loss 1.241 (1.652) Epoch: [34][320/400] Time 0.355 (0.398) Data 0.001 (0.040) Loss 2.021 (1.663) Epoch: [34][330/400] Time 0.353 (0.397) Data 0.001 (0.039) Loss 1.802 (1.672) Epoch: [34][340/400] Time 0.354 (0.396) Data 0.001 (0.038) Loss 2.133 (1.679) Epoch: [34][350/400] Time 0.360 (0.395) Data 0.000 (0.037) Loss 2.327 (1.691) Epoch: [34][360/400] Time 0.353 (0.399) Data 0.000 (0.040) Loss 1.637 (1.695) Epoch: [34][370/400] Time 0.351 (0.398) Data 0.001 (0.039) Loss 1.787 (1.695) Epoch: [34][380/400] Time 0.353 (0.396) Data 0.001 (0.038) Loss 1.898 (1.702) Epoch: [34][390/400] Time 0.361 (0.395) Data 0.000 (0.037) Loss 2.211 (1.710) Epoch: [34][400/400] Time 0.350 (0.399) Data 0.001 (0.041) Loss 2.018 (1.718) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.128 (0.158) Data 0.000 (0.027) Computing jaccard distance... Jaccard distance computing time cost: 18.268587589263916 ==> Statistics for epoch 35: 709 clusters Epoch: [35][10/400] Time 0.350 (0.401) Data 0.001 (0.046) Loss 0.238 (0.250) Epoch: [35][20/400] Time 0.351 (0.376) Data 0.001 (0.023) Loss 0.157 (0.250) Epoch: [35][30/400] Time 0.350 (0.367) Data 0.001 (0.016) Loss 0.120 (0.232) Epoch: [35][40/400] Time 0.361 (0.366) Data 0.000 (0.012) Loss 0.129 (0.214) Epoch: [35][50/400] Time 0.351 (0.401) Data 0.001 (0.045) Loss 2.182 (0.447) Epoch: [35][60/400] Time 0.351 (0.392) Data 0.001 (0.038) Loss 2.259 (0.728) Epoch: [35][70/400] Time 0.350 (0.387) Data 0.000 (0.032) Loss 2.739 (0.919) Epoch: [35][80/400] Time 0.352 (0.382) Data 0.000 (0.028) Loss 1.888 (1.064) Epoch: [35][90/400] Time 0.351 (0.399) Data 0.000 (0.045) Loss 1.659 (1.142) Epoch: [35][100/400] Time 0.351 (0.394) Data 0.001 (0.040) Loss 1.626 (1.206) Epoch: [35][110/400] Time 0.352 (0.391) Data 0.000 (0.037) Loss 2.215 (1.257) Epoch: [35][120/400] Time 0.372 (0.388) Data 0.001 (0.034) Loss 1.815 (1.308) Epoch: [35][130/400] Time 0.362 (0.386) Data 0.000 (0.031) Loss 1.783 (1.346) Epoch: [35][140/400] Time 0.352 (0.396) Data 0.000 (0.041) Loss 2.988 (1.371) Epoch: [35][150/400] Time 0.351 (0.393) Data 0.001 (0.038) Loss 1.978 (1.408) Epoch: [35][160/400] Time 0.361 (0.391) Data 0.000 (0.036) Loss 1.353 (1.438) Epoch: [35][170/400] Time 0.362 (0.389) Data 0.000 (0.034) Loss 1.955 (1.464) Epoch: [35][180/400] Time 0.352 (0.398) Data 0.000 (0.043) Loss 1.737 (1.473) Epoch: [35][190/400] Time 0.397 (0.396) Data 0.001 (0.040) Loss 2.531 (1.491) Epoch: [35][200/400] Time 0.349 (0.394) Data 0.001 (0.038) Loss 2.015 (1.501) Epoch: [35][210/400] Time 0.386 (0.392) Data 0.001 (0.037) Loss 1.617 (1.523) Epoch: [35][220/400] Time 0.362 (0.391) Data 0.000 (0.035) Loss 2.169 (1.543) Epoch: [35][230/400] Time 0.362 (0.397) Data 0.001 (0.041) Loss 1.787 (1.551) Epoch: [35][240/400] Time 0.358 (0.395) Data 0.000 (0.039) Loss 2.130 (1.562) Epoch: [35][250/400] Time 0.361 (0.393) Data 0.001 (0.038) Loss 1.558 (1.568) Epoch: [35][260/400] Time 0.361 (0.392) Data 0.000 (0.036) Loss 2.154 (1.582) Epoch: [35][270/400] Time 0.352 (0.398) Data 0.001 (0.042) Loss 1.527 (1.596) Epoch: [35][280/400] Time 0.352 (0.396) Data 0.000 (0.040) Loss 1.892 (1.609) Epoch: [35][290/400] Time 0.352 (0.395) Data 0.001 (0.039) Loss 1.607 (1.617) Epoch: [35][300/400] Time 0.351 (0.393) Data 0.000 (0.038) Loss 2.073 (1.623) Epoch: [35][310/400] Time 0.362 (0.398) Data 0.000 (0.043) Loss 2.095 (1.630) Epoch: [35][320/400] Time 0.344 (0.397) Data 0.001 (0.041) Loss 1.743 (1.635) Epoch: [35][330/400] Time 0.363 (0.396) Data 0.001 (0.040) Loss 2.024 (1.641) Epoch: [35][340/400] Time 0.362 (0.395) Data 0.001 (0.039) Loss 2.016 (1.646) Epoch: [35][350/400] Time 0.363 (0.394) Data 0.000 (0.038) Loss 1.809 (1.647) Epoch: [35][360/400] Time 0.353 (0.398) Data 0.000 (0.042) Loss 1.855 (1.654) Epoch: [35][370/400] Time 0.349 (0.397) Data 0.001 (0.041) Loss 2.347 (1.658) Epoch: [35][380/400] Time 0.361 (0.396) Data 0.001 (0.039) Loss 1.610 (1.660) Epoch: [35][390/400] Time 0.362 (0.395) Data 0.000 (0.038) Loss 1.695 (1.663) Epoch: [35][400/400] Time 0.350 (0.398) Data 0.000 (0.042) Loss 1.929 (1.668) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.162) Data 0.000 (0.031) Computing jaccard distance... Jaccard distance computing time cost: 18.193820238113403 ==> Statistics for epoch 36: 711 clusters Epoch: [36][10/400] Time 0.360 (0.404) Data 0.001 (0.046) Loss 0.284 (0.219) Epoch: [36][20/400] Time 0.349 (0.381) Data 0.000 (0.023) Loss 0.230 (0.232) Epoch: [36][30/400] Time 0.360 (0.371) Data 0.001 (0.016) Loss 0.212 (0.217) Epoch: [36][40/400] Time 0.360 (0.367) Data 0.000 (0.012) Loss 0.105 (0.200) Epoch: [36][50/400] Time 0.360 (0.403) Data 0.001 (0.046) Loss 2.046 (0.427) Epoch: [36][60/400] Time 0.351 (0.395) Data 0.001 (0.039) Loss 2.402 (0.695) Epoch: [36][70/400] Time 0.353 (0.390) Data 0.000 (0.033) Loss 2.158 (0.898) Epoch: [36][80/400] Time 0.351 (0.385) Data 0.001 (0.029) Loss 1.923 (1.028) Epoch: [36][90/400] Time 0.373 (0.404) Data 0.001 (0.047) Loss 1.971 (1.129) Epoch: [36][100/400] Time 0.354 (0.399) Data 0.001 (0.042) Loss 1.879 (1.199) Epoch: [36][110/400] Time 0.350 (0.395) Data 0.000 (0.038) Loss 2.084 (1.264) Epoch: [36][120/400] Time 0.350 (0.391) Data 0.000 (0.035) Loss 1.254 (1.306) Epoch: [36][130/400] Time 0.361 (0.389) Data 0.000 (0.032) Loss 1.993 (1.343) Epoch: [36][140/400] Time 0.352 (0.399) Data 0.000 (0.043) Loss 1.589 (1.374) Epoch: [36][150/400] Time 0.350 (0.396) Data 0.001 (0.040) Loss 2.109 (1.404) Epoch: [36][160/400] Time 0.351 (0.393) Data 0.001 (0.038) Loss 1.775 (1.431) Epoch: [36][170/400] Time 0.360 (0.391) Data 0.000 (0.035) Loss 2.344 (1.454) Epoch: [36][180/400] Time 0.388 (0.400) Data 0.001 (0.044) Loss 2.050 (1.472) Epoch: [36][190/400] Time 0.350 (0.398) Data 0.001 (0.042) Loss 1.485 (1.480) Epoch: [36][200/400] Time 0.352 (0.395) Data 0.001 (0.040) Loss 1.883 (1.493) Epoch: [36][210/400] Time 0.350 (0.393) Data 0.001 (0.038) Loss 2.398 (1.511) Epoch: [36][220/400] Time 0.361 (0.392) Data 0.000 (0.036) Loss 1.606 (1.526) Epoch: [36][230/400] Time 0.351 (0.399) Data 0.000 (0.043) Loss 2.663 (1.542) Epoch: [36][240/400] Time 0.352 (0.397) Data 0.000 (0.041) Loss 2.496 (1.558) Epoch: [36][250/400] Time 0.350 (0.395) Data 0.000 (0.039) Loss 1.472 (1.566) Epoch: [36][260/400] Time 0.361 (0.393) Data 0.000 (0.038) Loss 2.040 (1.581) Epoch: [36][270/400] Time 0.352 (0.399) Data 0.000 (0.043) Loss 1.965 (1.594) Epoch: [36][280/400] Time 0.352 (0.397) Data 0.001 (0.042) Loss 1.581 (1.601) Epoch: [36][290/400] Time 0.354 (0.396) Data 0.000 (0.040) Loss 1.575 (1.613) Epoch: [36][300/400] Time 0.353 (0.395) Data 0.001 (0.039) Loss 1.625 (1.618) Epoch: [36][310/400] Time 0.351 (0.399) Data 0.001 (0.043) Loss 1.476 (1.623) Epoch: [36][320/400] Time 0.351 (0.398) Data 0.000 (0.042) Loss 2.165 (1.634) Epoch: [36][330/400] Time 0.352 (0.397) Data 0.001 (0.041) Loss 2.060 (1.636) Epoch: [36][340/400] Time 0.353 (0.395) Data 0.000 (0.040) Loss 2.750 (1.642) Epoch: [36][350/400] Time 0.362 (0.394) Data 0.000 (0.038) Loss 1.950 (1.643) Epoch: [36][360/400] Time 0.386 (0.398) Data 0.000 (0.042) Loss 1.257 (1.644) Epoch: [36][370/400] Time 0.362 (0.397) Data 0.001 (0.041) Loss 1.395 (1.646) Epoch: [36][380/400] Time 0.354 (0.396) Data 0.000 (0.040) Loss 1.661 (1.654) Epoch: [36][390/400] Time 0.362 (0.395) Data 0.000 (0.039) Loss 1.568 (1.654) Epoch: [36][400/400] Time 0.348 (0.399) Data 0.000 (0.043) Loss 2.014 (1.657) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.161) Data 0.000 (0.029) Computing jaccard distance... Jaccard distance computing time cost: 18.401859998703003 ==> Statistics for epoch 37: 709 clusters Epoch: [37][10/400] Time 0.347 (0.415) Data 0.001 (0.052) Loss 0.302 (0.264) Epoch: [37][20/400] Time 0.347 (0.383) Data 0.000 (0.026) Loss 0.157 (0.230) Epoch: [37][30/400] Time 0.351 (0.373) Data 0.001 (0.018) Loss 0.200 (0.218) Epoch: [37][40/400] Time 0.361 (0.368) Data 0.000 (0.013) Loss 0.108 (0.199) Epoch: [37][50/400] Time 0.361 (0.405) Data 0.001 (0.049) Loss 2.320 (0.430) Epoch: [37][60/400] Time 0.406 (0.397) Data 0.001 (0.041) Loss 2.069 (0.683) Epoch: [37][70/400] Time 0.361 (0.391) Data 0.001 (0.035) Loss 1.630 (0.885) Epoch: [37][80/400] Time 0.351 (0.387) Data 0.001 (0.031) Loss 1.943 (1.008) Epoch: [37][90/400] Time 0.354 (0.404) Data 0.000 (0.046) Loss 1.611 (1.125) Epoch: [37][100/400] Time 0.355 (0.399) Data 0.001 (0.041) Loss 1.927 (1.182) Epoch: [37][110/400] Time 0.352 (0.395) Data 0.001 (0.038) Loss 1.416 (1.230) Epoch: [37][120/400] Time 0.352 (0.391) Data 0.001 (0.035) Loss 1.431 (1.276) Epoch: [37][130/400] Time 0.361 (0.388) Data 0.000 (0.032) Loss 2.194 (1.325) Epoch: [37][140/400] Time 0.354 (0.399) Data 0.000 (0.043) Loss 1.044 (1.348) Epoch: [37][150/400] Time 0.351 (0.396) Data 0.000 (0.040) Loss 1.801 (1.390) Epoch: [37][160/400] Time 0.353 (0.393) Data 0.000 (0.037) Loss 2.198 (1.422) Epoch: [37][170/400] Time 0.362 (0.391) Data 0.000 (0.035) Loss 2.183 (1.454) Epoch: [37][180/400] Time 0.352 (0.399) Data 0.001 (0.043) Loss 1.823 (1.475) Epoch: [37][190/400] Time 0.351 (0.396) Data 0.001 (0.041) Loss 1.610 (1.476) Epoch: [37][200/400] Time 0.364 (0.394) Data 0.001 (0.039) Loss 1.835 (1.494) Epoch: [37][210/400] Time 0.351 (0.392) Data 0.001 (0.037) Loss 2.643 (1.510) Epoch: [37][220/400] Time 0.363 (0.391) Data 0.000 (0.035) Loss 2.271 (1.525) Epoch: [37][230/400] Time 0.360 (0.397) Data 0.001 (0.041) Loss 2.186 (1.544) Epoch: [37][240/400] Time 0.348 (0.395) Data 0.001 (0.040) Loss 2.245 (1.555) Epoch: [37][250/400] Time 0.350 (0.393) Data 0.001 (0.038) Loss 1.419 (1.566) Epoch: [37][260/400] Time 0.362 (0.392) Data 0.000 (0.037) Loss 1.575 (1.572) Epoch: [37][270/400] Time 0.363 (0.398) Data 0.001 (0.042) Loss 2.652 (1.588) Epoch: [37][280/400] Time 0.350 (0.396) Data 0.001 (0.040) Loss 2.592 (1.600) Epoch: [37][290/400] Time 0.361 (0.395) Data 0.001 (0.039) Loss 1.409 (1.602) Epoch: [37][300/400] Time 0.350 (0.393) Data 0.001 (0.038) Loss 1.215 (1.606) Epoch: [37][310/400] Time 0.349 (0.398) Data 0.001 (0.042) Loss 1.869 (1.617) Epoch: [37][320/400] Time 0.352 (0.396) Data 0.001 (0.041) Loss 1.683 (1.621) Epoch: [37][330/400] Time 0.352 (0.395) Data 0.001 (0.040) Loss 1.971 (1.629) Epoch: [37][340/400] Time 0.352 (0.394) Data 0.001 (0.038) Loss 1.028 (1.634) Epoch: [37][350/400] Time 0.362 (0.393) Data 0.000 (0.037) Loss 2.136 (1.637) Epoch: [37][360/400] Time 0.351 (0.397) Data 0.000 (0.041) Loss 1.058 (1.635) Epoch: [37][370/400] Time 0.362 (0.396) Data 0.001 (0.040) Loss 1.410 (1.635) Epoch: [37][380/400] Time 0.351 (0.395) Data 0.001 (0.039) Loss 1.461 (1.637) Epoch: [37][390/400] Time 0.362 (0.394) Data 0.000 (0.038) Loss 1.372 (1.639) Epoch: [37][400/400] Time 0.341 (0.397) Data 0.001 (0.042) Loss 2.433 (1.647) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.128 (0.159) Data 0.000 (0.028) Computing jaccard distance... Jaccard distance computing time cost: 18.123547554016113 ==> Statistics for epoch 38: 708 clusters Epoch: [38][10/400] Time 0.350 (0.422) Data 0.001 (0.063) Loss 0.191 (0.205) Epoch: [38][20/400] Time 0.352 (0.390) Data 0.001 (0.032) Loss 0.253 (0.202) Epoch: [38][30/400] Time 0.378 (0.377) Data 0.001 (0.021) Loss 0.201 (0.203) Epoch: [38][40/400] Time 0.361 (0.372) Data 0.000 (0.016) Loss 0.196 (0.191) Epoch: [38][50/400] Time 0.361 (0.407) Data 0.001 (0.050) Loss 2.190 (0.410) Epoch: [38][60/400] Time 0.352 (0.398) Data 0.001 (0.042) Loss 1.711 (0.637) Epoch: [38][70/400] Time 0.352 (0.392) Data 0.001 (0.036) Loss 1.449 (0.845) Epoch: [38][80/400] Time 0.347 (0.387) Data 0.001 (0.031) Loss 1.806 (0.953) Epoch: [38][90/400] Time 0.349 (0.404) Data 0.001 (0.047) Loss 1.974 (1.062) Epoch: [38][100/400] Time 0.407 (0.399) Data 0.001 (0.042) Loss 0.921 (1.100) Epoch: [38][110/400] Time 0.353 (0.395) Data 0.001 (0.039) Loss 1.614 (1.159) Epoch: [38][120/400] Time 0.376 (0.392) Data 0.001 (0.035) Loss 1.641 (1.208) Epoch: [38][130/400] Time 0.360 (0.390) Data 0.000 (0.033) Loss 1.477 (1.228) Epoch: [38][140/400] Time 0.352 (0.399) Data 0.000 (0.043) Loss 1.564 (1.272) Epoch: [38][150/400] Time 0.351 (0.396) Data 0.000 (0.040) Loss 2.342 (1.311) Epoch: [38][160/400] Time 0.361 (0.394) Data 0.000 (0.037) Loss 1.523 (1.338) Epoch: [38][170/400] Time 0.362 (0.391) Data 0.000 (0.035) Loss 1.103 (1.360) Epoch: [38][180/400] Time 0.366 (0.400) Data 0.000 (0.043) Loss 2.203 (1.374) Epoch: [38][190/400] Time 0.352 (0.397) Data 0.000 (0.041) Loss 1.910 (1.395) Epoch: [38][200/400] Time 0.352 (0.395) Data 0.001 (0.039) Loss 1.841 (1.408) Epoch: [38][210/400] Time 0.351 (0.393) Data 0.000 (0.037) Loss 1.734 (1.427) Epoch: [38][220/400] Time 0.362 (0.391) Data 0.000 (0.035) Loss 1.831 (1.447) Epoch: [38][230/400] Time 0.352 (0.398) Data 0.001 (0.041) Loss 1.554 (1.458) Epoch: [38][240/400] Time 0.351 (0.396) Data 0.000 (0.040) Loss 2.432 (1.468) Epoch: [38][250/400] Time 0.378 (0.394) Data 0.001 (0.038) Loss 1.196 (1.474) Epoch: [38][260/400] Time 0.362 (0.393) Data 0.000 (0.037) Loss 1.838 (1.484) Epoch: [38][270/400] Time 0.350 (0.398) Data 0.000 (0.042) Loss 1.457 (1.493) Epoch: [38][280/400] Time 0.361 (0.397) Data 0.001 (0.041) Loss 1.560 (1.504) Epoch: [38][290/400] Time 0.352 (0.395) Data 0.001 (0.039) Loss 1.937 (1.517) Epoch: [38][300/400] Time 0.361 (0.394) Data 0.000 (0.038) Loss 2.140 (1.529) Epoch: [38][310/400] Time 0.349 (0.398) Data 0.001 (0.042) Loss 1.848 (1.535) Epoch: [38][320/400] Time 0.361 (0.397) Data 0.001 (0.041) Loss 1.352 (1.543) Epoch: [38][330/400] Time 0.361 (0.396) Data 0.001 (0.040) Loss 1.742 (1.550) Epoch: [38][340/400] Time 0.360 (0.395) Data 0.000 (0.038) Loss 1.578 (1.554) Epoch: [38][350/400] Time 0.362 (0.394) Data 0.000 (0.037) Loss 2.461 (1.568) Epoch: [38][360/400] Time 0.350 (0.398) Data 0.000 (0.042) Loss 1.616 (1.570) Epoch: [38][370/400] Time 0.352 (0.397) Data 0.001 (0.040) Loss 2.210 (1.578) Epoch: [38][380/400] Time 0.359 (0.396) Data 0.001 (0.039) Loss 2.130 (1.583) Epoch: [38][390/400] Time 0.361 (0.395) Data 0.000 (0.038) Loss 1.376 (1.589) Epoch: [38][400/400] Time 0.360 (0.398) Data 0.000 (0.042) Loss 1.360 (1.592) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.161) Data 0.000 (0.028) Computing jaccard distance... Jaccard distance computing time cost: 21.01050329208374 ==> Statistics for epoch 39: 711 clusters Epoch: [39][10/400] Time 0.359 (0.423) Data 0.001 (0.057) Loss 0.258 (0.204) Epoch: [39][20/400] Time 0.350 (0.387) Data 0.000 (0.029) Loss 0.162 (0.201) Epoch: [39][30/400] Time 0.352 (0.375) Data 0.000 (0.019) Loss 0.173 (0.196) Epoch: [39][40/400] Time 0.361 (0.370) Data 0.000 (0.015) Loss 0.104 (0.179) Epoch: [39][50/400] Time 0.362 (0.406) Data 0.001 (0.048) Loss 1.335 (0.366) Epoch: [39][60/400] Time 0.351 (0.398) Data 0.001 (0.040) Loss 1.526 (0.599) Epoch: [39][70/400] Time 0.362 (0.392) Data 0.001 (0.035) Loss 1.863 (0.807) Epoch: [39][80/400] Time 0.349 (0.387) Data 0.001 (0.030) Loss 1.671 (0.937) Epoch: [39][90/400] Time 0.361 (0.405) Data 0.000 (0.048) Loss 1.887 (1.042) Epoch: [39][100/400] Time 0.351 (0.401) Data 0.001 (0.043) Loss 2.026 (1.105) Epoch: [39][110/400] Time 0.352 (0.396) Data 0.001 (0.039) Loss 2.422 (1.172) Epoch: [39][120/400] Time 0.353 (0.393) Data 0.000 (0.036) Loss 1.835 (1.202) Epoch: [39][130/400] Time 0.362 (0.390) Data 0.000 (0.033) Loss 2.023 (1.246) Epoch: [39][140/400] Time 0.352 (0.402) Data 0.000 (0.044) Loss 1.504 (1.281) Epoch: [39][150/400] Time 0.353 (0.399) Data 0.000 (0.041) Loss 1.647 (1.314) Epoch: [39][160/400] Time 0.350 (0.396) Data 0.001 (0.039) Loss 2.659 (1.358) Epoch: [39][170/400] Time 0.361 (0.394) Data 0.000 (0.036) Loss 1.229 (1.385) Epoch: [39][180/400] Time 0.360 (0.401) Data 0.000 (0.044) Loss 2.290 (1.402) Epoch: [39][190/400] Time 0.360 (0.399) Data 0.001 (0.041) Loss 1.390 (1.428) Epoch: [39][200/400] Time 0.411 (0.397) Data 0.001 (0.039) Loss 1.557 (1.442) Epoch: [39][210/400] Time 0.362 (0.395) Data 0.001 (0.038) Loss 1.926 (1.458) Epoch: [39][220/400] Time 0.361 (0.394) Data 0.000 (0.036) Loss 2.026 (1.467) Epoch: [39][230/400] Time 0.352 (0.400) Data 0.001 (0.043) Loss 1.330 (1.483) Epoch: [39][240/400] Time 0.338 (0.398) Data 0.001 (0.041) Loss 1.656 (1.496) Epoch: [39][250/400] Time 0.355 (0.397) Data 0.001 (0.039) Loss 1.567 (1.501) Epoch: [39][260/400] Time 0.348 (0.395) Data 0.000 (0.038) Loss 1.718 (1.520) Epoch: [39][270/400] Time 0.350 (0.400) Data 0.000 (0.043) Loss 1.690 (1.536) Epoch: [39][280/400] Time 0.351 (0.398) Data 0.001 (0.041) Loss 1.352 (1.539) Epoch: [39][290/400] Time 0.360 (0.397) Data 0.001 (0.040) Loss 1.725 (1.548) Epoch: [39][300/400] Time 0.352 (0.396) Data 0.001 (0.039) Loss 1.780 (1.555) Epoch: [39][310/400] Time 0.350 (0.400) Data 0.001 (0.043) Loss 2.326 (1.564) Epoch: [39][320/400] Time 0.351 (0.399) Data 0.000 (0.042) Loss 1.868 (1.567) Epoch: [39][330/400] Time 0.351 (0.398) Data 0.001 (0.041) Loss 1.967 (1.577) Epoch: [39][340/400] Time 0.351 (0.397) Data 0.001 (0.039) Loss 2.266 (1.582) Epoch: [39][350/400] Time 0.362 (0.396) Data 0.000 (0.038) Loss 1.948 (1.585) Epoch: [39][360/400] Time 0.353 (0.399) Data 0.000 (0.042) Loss 1.781 (1.584) Epoch: [39][370/400] Time 0.351 (0.398) Data 0.001 (0.041) Loss 2.013 (1.590) Epoch: [39][380/400] Time 0.352 (0.397) Data 0.001 (0.040) Loss 1.505 (1.597) Epoch: [39][390/400] Time 0.364 (0.396) Data 0.000 (0.039) Loss 1.749 (1.606) Epoch: [39][400/400] Time 0.361 (0.400) Data 0.000 (0.043) Loss 1.780 (1.611) Extract Features: [50/76] Time 0.134 (0.159) Data 0.000 (0.026) Mean AP: 86.8% * Finished epoch 39 model mAP: 86.8% best: 86.8% * ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.165) Data 0.000 (0.032) Computing jaccard distance... Jaccard distance computing time cost: 18.143017053604126 ==> Statistics for epoch 40: 707 clusters Epoch: [40][10/400] Time 0.348 (0.417) Data 0.001 (0.057) Loss 0.124 (0.206) Epoch: [40][20/400] Time 0.350 (0.387) Data 0.000 (0.029) Loss 0.184 (0.197) Epoch: [40][30/400] Time 0.350 (0.375) Data 0.000 (0.019) Loss 0.083 (0.185) Epoch: [40][40/400] Time 0.360 (0.370) Data 0.000 (0.015) Loss 0.100 (0.172) Epoch: [40][50/400] Time 0.380 (0.405) Data 0.001 (0.047) Loss 1.776 (0.368) Epoch: [40][60/400] Time 0.351 (0.396) Data 0.001 (0.039) Loss 1.513 (0.628) Epoch: [40][70/400] Time 0.351 (0.390) Data 0.001 (0.033) Loss 2.163 (0.824) Epoch: [40][80/400] Time 0.349 (0.386) Data 0.001 (0.029) Loss 1.906 (0.953) Epoch: [40][90/400] Time 0.351 (0.403) Data 0.001 (0.046) Loss 1.463 (1.035) Epoch: [40][100/400] Time 0.362 (0.398) Data 0.001 (0.041) Loss 1.973 (1.095) Epoch: [40][110/400] Time 0.352 (0.395) Data 0.001 (0.038) Loss 1.685 (1.145) Epoch: [40][120/400] Time 0.352 (0.392) Data 0.001 (0.034) Loss 1.609 (1.174) Epoch: [40][130/400] Time 0.362 (0.390) Data 0.000 (0.032) Loss 0.893 (1.207) Epoch: [40][140/400] Time 0.352 (0.400) Data 0.001 (0.042) Loss 1.766 (1.238) Epoch: [40][150/400] Time 0.363 (0.397) Data 0.000 (0.039) Loss 1.848 (1.271) Epoch: [40][160/400] Time 0.351 (0.394) Data 0.001 (0.037) Loss 1.326 (1.304) Epoch: [40][170/400] Time 0.362 (0.392) Data 0.000 (0.035) Loss 1.707 (1.325) Epoch: [40][180/400] Time 0.361 (0.400) Data 0.000 (0.042) Loss 2.184 (1.337) Epoch: [40][190/400] Time 0.363 (0.398) Data 0.001 (0.040) Loss 1.782 (1.355) Epoch: [40][200/400] Time 0.354 (0.396) Data 0.001 (0.038) Loss 1.454 (1.370) Epoch: [40][210/400] Time 0.364 (0.394) Data 0.001 (0.037) Loss 1.549 (1.378) Epoch: [40][220/400] Time 0.361 (0.393) Data 0.000 (0.035) Loss 1.658 (1.394) Epoch: [40][230/400] Time 0.362 (0.399) Data 0.001 (0.041) Loss 1.838 (1.413) Epoch: [40][240/400] Time 0.350 (0.397) Data 0.001 (0.039) Loss 1.166 (1.419) Epoch: [40][250/400] Time 0.363 (0.395) Data 0.001 (0.038) Loss 0.982 (1.425) Epoch: [40][260/400] Time 0.364 (0.394) Data 0.000 (0.036) Loss 1.744 (1.432) Epoch: [40][270/400] Time 0.353 (0.400) Data 0.001 (0.042) Loss 2.232 (1.445) Epoch: [40][280/400] Time 0.361 (0.398) Data 0.001 (0.040) Loss 1.813 (1.449) Epoch: [40][290/400] Time 0.363 (0.397) Data 0.001 (0.039) Loss 1.564 (1.454) Epoch: [40][300/400] Time 0.374 (0.396) Data 0.001 (0.038) Loss 1.757 (1.464) Epoch: [40][310/400] Time 0.347 (0.401) Data 0.001 (0.042) Loss 1.468 (1.465) Epoch: [40][320/400] Time 0.354 (0.399) Data 0.001 (0.041) Loss 1.230 (1.465) Epoch: [40][330/400] Time 0.353 (0.398) Data 0.001 (0.040) Loss 2.178 (1.471) Epoch: [40][340/400] Time 0.362 (0.397) Data 0.001 (0.038) Loss 1.000 (1.472) Epoch: [40][350/400] Time 0.363 (0.396) Data 0.000 (0.037) Loss 2.050 (1.479) Epoch: [40][360/400] Time 0.353 (0.399) Data 0.000 (0.041) Loss 1.524 (1.482) Epoch: [40][370/400] Time 0.362 (0.398) Data 0.001 (0.040) Loss 1.736 (1.485) Epoch: [40][380/400] Time 0.353 (0.397) Data 0.000 (0.039) Loss 1.418 (1.496) Epoch: [40][390/400] Time 0.363 (0.396) Data 0.000 (0.038) Loss 1.328 (1.498) Epoch: [40][400/400] Time 0.351 (0.400) Data 0.000 (0.042) Loss 1.822 (1.500) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.159) Data 0.000 (0.026) Computing jaccard distance... Jaccard distance computing time cost: 18.482816457748413 ==> Statistics for epoch 41: 710 clusters Epoch: [41][10/400] Time 0.350 (0.405) Data 0.000 (0.046) Loss 0.172 (0.208) Epoch: [41][20/400] Time 0.351 (0.378) Data 0.000 (0.023) Loss 0.214 (0.187) Epoch: [41][30/400] Time 0.350 (0.371) Data 0.000 (0.016) Loss 0.135 (0.180) Epoch: [41][40/400] Time 0.361 (0.366) Data 0.000 (0.012) Loss 0.119 (0.171) Epoch: [41][50/400] Time 0.355 (0.400) Data 0.000 (0.044) Loss 1.536 (0.341) Epoch: [41][60/400] Time 0.350 (0.392) Data 0.000 (0.037) Loss 1.437 (0.581) Epoch: [41][70/400] Time 0.362 (0.386) Data 0.000 (0.032) Loss 1.328 (0.758) Epoch: [41][80/400] Time 0.350 (0.383) Data 0.001 (0.028) Loss 1.718 (0.879) Epoch: [41][90/400] Time 0.374 (0.401) Data 0.001 (0.045) Loss 2.029 (0.992) Epoch: [41][100/400] Time 0.352 (0.397) Data 0.001 (0.040) Loss 1.179 (1.076) Epoch: [41][110/400] Time 0.397 (0.393) Data 0.001 (0.037) Loss 1.798 (1.127) Epoch: [41][120/400] Time 0.353 (0.390) Data 0.001 (0.034) Loss 1.256 (1.164) Epoch: [41][130/400] Time 0.369 (0.388) Data 0.000 (0.031) Loss 1.696 (1.195) Epoch: [41][140/400] Time 0.354 (0.398) Data 0.000 (0.041) Loss 1.644 (1.228) Epoch: [41][150/400] Time 0.353 (0.395) Data 0.001 (0.038) Loss 1.341 (1.247) Epoch: [41][160/400] Time 0.354 (0.393) Data 0.001 (0.036) Loss 1.449 (1.266) Epoch: [41][170/400] Time 0.364 (0.391) Data 0.000 (0.034) Loss 1.767 (1.280) Epoch: [41][180/400] Time 0.363 (0.400) Data 0.001 (0.042) Loss 1.540 (1.295) Epoch: [41][190/400] Time 0.356 (0.398) Data 0.001 (0.040) Loss 1.829 (1.315) Epoch: [41][200/400] Time 0.354 (0.395) Data 0.001 (0.038) Loss 2.040 (1.340) Epoch: [41][210/400] Time 0.353 (0.394) Data 0.001 (0.036) Loss 1.267 (1.351) Epoch: [41][220/400] Time 0.359 (0.392) Data 0.000 (0.034) Loss 1.565 (1.362) Epoch: [41][230/400] Time 0.357 (0.398) Data 0.001 (0.041) Loss 1.555 (1.368) Epoch: [41][240/400] Time 0.351 (0.397) Data 0.001 (0.039) Loss 1.458 (1.377) Epoch: [41][250/400] Time 0.355 (0.395) Data 0.001 (0.038) Loss 1.978 (1.390) Epoch: [41][260/400] Time 0.364 (0.393) Data 0.000 (0.036) Loss 1.634 (1.397) Epoch: [41][270/400] Time 0.353 (0.399) Data 0.001 (0.041) Loss 1.419 (1.397) Epoch: [41][280/400] Time 0.352 (0.397) Data 0.001 (0.040) Loss 1.578 (1.402) Epoch: [41][290/400] Time 0.353 (0.396) Data 0.001 (0.038) Loss 1.037 (1.407) Epoch: [41][300/400] Time 0.354 (0.394) Data 0.001 (0.037) Loss 1.339 (1.418) Epoch: [41][310/400] Time 0.414 (0.399) Data 0.001 (0.041) Loss 1.368 (1.424) Epoch: [41][320/400] Time 0.362 (0.398) Data 0.001 (0.040) Loss 1.917 (1.428) Epoch: [41][330/400] Time 0.363 (0.397) Data 0.001 (0.039) Loss 1.997 (1.442) Epoch: [41][340/400] Time 0.353 (0.396) Data 0.001 (0.038) Loss 2.095 (1.454) Epoch: [41][350/400] Time 0.365 (0.395) Data 0.000 (0.037) Loss 1.098 (1.453) Epoch: [41][360/400] Time 0.353 (0.399) Data 0.001 (0.041) Loss 1.919 (1.456) Epoch: [41][370/400] Time 0.355 (0.398) Data 0.001 (0.039) Loss 1.852 (1.460) Epoch: [41][380/400] Time 0.351 (0.396) Data 0.001 (0.038) Loss 1.355 (1.464) Epoch: [41][390/400] Time 0.364 (0.395) Data 0.000 (0.037) Loss 1.235 (1.468) Epoch: [41][400/400] Time 0.352 (0.399) Data 0.001 (0.041) Loss 1.871 (1.472) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.163) Data 0.000 (0.030) Computing jaccard distance... Jaccard distance computing time cost: 18.165592432022095 ==> Statistics for epoch 42: 710 clusters Epoch: [42][10/400] Time 0.351 (0.403) Data 0.001 (0.046) Loss 0.131 (0.209) Epoch: [42][20/400] Time 0.350 (0.377) Data 0.001 (0.023) Loss 0.079 (0.190) Epoch: [42][30/400] Time 0.352 (0.369) Data 0.001 (0.016) Loss 0.111 (0.185) Epoch: [42][40/400] Time 0.362 (0.366) Data 0.000 (0.012) Loss 0.156 (0.177) Epoch: [42][50/400] Time 0.352 (0.397) Data 0.001 (0.042) Loss 1.428 (0.371) Epoch: [42][60/400] Time 0.406 (0.390) Data 0.001 (0.035) Loss 1.790 (0.603) Epoch: [42][70/400] Time 0.352 (0.385) Data 0.001 (0.030) Loss 1.906 (0.781) Epoch: [42][80/400] Time 0.351 (0.381) Data 0.000 (0.027) Loss 2.170 (0.908) Epoch: [42][90/400] Time 0.374 (0.398) Data 0.001 (0.042) Loss 1.977 (1.024) Epoch: [42][100/400] Time 0.362 (0.394) Data 0.001 (0.038) Loss 2.042 (1.082) Epoch: [42][110/400] Time 0.362 (0.391) Data 0.001 (0.035) Loss 1.651 (1.124) Epoch: [42][120/400] Time 0.353 (0.388) Data 0.000 (0.032) Loss 1.648 (1.170) Epoch: [42][130/400] Time 0.360 (0.385) Data 0.000 (0.030) Loss 1.396 (1.199) Epoch: [42][140/400] Time 0.352 (0.397) Data 0.000 (0.040) Loss 1.425 (1.234) Epoch: [42][150/400] Time 0.352 (0.394) Data 0.001 (0.038) Loss 1.445 (1.253) Epoch: [42][160/400] Time 0.351 (0.391) Data 0.000 (0.035) Loss 1.912 (1.262) Epoch: [42][170/400] Time 0.355 (0.389) Data 0.000 (0.033) Loss 1.455 (1.283) Epoch: [42][180/400] Time 0.350 (0.397) Data 0.000 (0.041) Loss 1.667 (1.300) Epoch: [42][190/400] Time 0.350 (0.395) Data 0.001 (0.039) Loss 1.394 (1.316) Epoch: [42][200/400] Time 0.352 (0.393) Data 0.001 (0.037) Loss 1.452 (1.334) Epoch: [42][210/400] Time 0.352 (0.391) Data 0.001 (0.035) Loss 1.692 (1.346) Epoch: [42][220/400] Time 0.362 (0.390) Data 0.000 (0.034) Loss 1.589 (1.358) Epoch: [42][230/400] Time 0.351 (0.396) Data 0.001 (0.040) Loss 1.390 (1.371) Epoch: [42][240/400] Time 0.354 (0.394) Data 0.000 (0.038) Loss 1.540 (1.383) Epoch: [42][250/400] Time 0.351 (0.392) Data 0.001 (0.037) Loss 1.383 (1.396) Epoch: [42][260/400] Time 0.363 (0.391) Data 0.000 (0.035) Loss 1.978 (1.408) Epoch: [42][270/400] Time 0.351 (0.397) Data 0.001 (0.041) Loss 1.521 (1.420) Epoch: [42][280/400] Time 0.352 (0.395) Data 0.001 (0.039) Loss 1.836 (1.433) Epoch: [42][290/400] Time 0.352 (0.394) Data 0.001 (0.038) Loss 1.459 (1.442) Epoch: [42][300/400] Time 0.352 (0.393) Data 0.001 (0.037) Loss 1.829 (1.448) Epoch: [42][310/400] Time 0.350 (0.397) Data 0.001 (0.041) Loss 1.895 (1.453) Epoch: [42][320/400] Time 0.353 (0.396) Data 0.000 (0.040) Loss 1.434 (1.459) Epoch: [42][330/400] Time 0.363 (0.395) Data 0.001 (0.039) Loss 1.398 (1.461) Epoch: [42][340/400] Time 0.352 (0.394) Data 0.000 (0.038) Loss 1.933 (1.464) Epoch: [42][350/400] Time 0.363 (0.393) Data 0.000 (0.037) Loss 1.301 (1.468) Epoch: [42][360/400] Time 0.354 (0.397) Data 0.001 (0.041) Loss 1.466 (1.469) Epoch: [42][370/400] Time 0.348 (0.396) Data 0.001 (0.040) Loss 2.005 (1.477) Epoch: [42][380/400] Time 0.351 (0.395) Data 0.001 (0.039) Loss 2.237 (1.481) Epoch: [42][390/400] Time 0.363 (0.394) Data 0.000 (0.038) Loss 1.526 (1.485) Epoch: [42][400/400] Time 0.351 (0.397) Data 0.000 (0.041) Loss 1.959 (1.495) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.128 (0.160) Data 0.000 (0.028) Computing jaccard distance... Jaccard distance computing time cost: 18.162556648254395 ==> Statistics for epoch 43: 707 clusters Epoch: [43][10/400] Time 0.350 (0.415) Data 0.001 (0.051) Loss 0.180 (0.211) Epoch: [43][20/400] Time 0.362 (0.386) Data 0.001 (0.026) Loss 0.165 (0.194) Epoch: [43][30/400] Time 0.350 (0.375) Data 0.001 (0.017) Loss 0.104 (0.188) Epoch: [43][40/400] Time 0.362 (0.370) Data 0.000 (0.013) Loss 0.180 (0.183) Epoch: [43][50/400] Time 0.361 (0.405) Data 0.000 (0.047) Loss 1.803 (0.381) Epoch: [43][60/400] Time 0.348 (0.398) Data 0.000 (0.040) Loss 2.111 (0.618) Epoch: [43][70/400] Time 0.362 (0.392) Data 0.001 (0.034) Loss 2.024 (0.790) Epoch: [43][80/400] Time 0.353 (0.388) Data 0.000 (0.030) Loss 1.710 (0.911) Epoch: [43][90/400] Time 0.349 (0.405) Data 0.001 (0.046) Loss 1.285 (1.008) Epoch: [43][100/400] Time 0.354 (0.400) Data 0.000 (0.042) Loss 1.357 (1.059) Epoch: [43][110/400] Time 0.363 (0.396) Data 0.001 (0.038) Loss 1.604 (1.109) Epoch: [43][120/400] Time 0.354 (0.393) Data 0.000 (0.035) Loss 1.543 (1.151) Epoch: [43][130/400] Time 0.363 (0.390) Data 0.000 (0.032) Loss 1.795 (1.179) Epoch: [43][140/400] Time 0.351 (0.400) Data 0.001 (0.042) Loss 1.096 (1.210) Epoch: [43][150/400] Time 0.351 (0.397) Data 0.001 (0.039) Loss 1.239 (1.239) Epoch: [43][160/400] Time 0.353 (0.394) Data 0.000 (0.037) Loss 2.171 (1.260) Epoch: [43][170/400] Time 0.363 (0.392) Data 0.000 (0.034) Loss 1.504 (1.283) Epoch: [43][180/400] Time 0.351 (0.401) Data 0.001 (0.043) Loss 1.284 (1.303) Epoch: [43][190/400] Time 0.353 (0.398) Data 0.001 (0.041) Loss 1.322 (1.317) Epoch: [43][200/400] Time 0.353 (0.396) Data 0.001 (0.039) Loss 1.440 (1.329) Epoch: [43][210/400] Time 0.352 (0.394) Data 0.001 (0.037) Loss 1.088 (1.333) Epoch: [43][220/400] Time 0.369 (0.393) Data 0.000 (0.035) Loss 1.715 (1.350) Epoch: [43][230/400] Time 0.362 (0.399) Data 0.001 (0.042) Loss 1.376 (1.363) Epoch: [43][240/400] Time 0.352 (0.398) Data 0.000 (0.040) Loss 2.252 (1.370) Epoch: [43][250/400] Time 0.354 (0.396) Data 0.001 (0.039) Loss 1.209 (1.380) Epoch: [43][260/400] Time 0.364 (0.394) Data 0.000 (0.037) Loss 2.110 (1.398) Epoch: [43][270/400] Time 0.353 (0.400) Data 0.001 (0.042) Loss 1.596 (1.408) Epoch: [43][280/400] Time 0.351 (0.399) Data 0.001 (0.041) Loss 1.804 (1.415) Epoch: [43][290/400] Time 0.354 (0.397) Data 0.001 (0.040) Loss 1.572 (1.424) Epoch: [43][300/400] Time 0.351 (0.396) Data 0.001 (0.038) Loss 2.234 (1.432) Epoch: [43][310/400] Time 0.349 (0.400) Data 0.001 (0.043) Loss 2.016 (1.439) Epoch: [43][320/400] Time 0.351 (0.399) Data 0.000 (0.041) Loss 1.284 (1.437) Epoch: [43][330/400] Time 0.351 (0.397) Data 0.000 (0.040) Loss 1.405 (1.444) Epoch: [43][340/400] Time 0.352 (0.396) Data 0.000 (0.039) Loss 0.942 (1.445) Epoch: [43][350/400] Time 0.359 (0.395) Data 0.000 (0.038) Loss 1.682 (1.448) Epoch: [43][360/400] Time 0.354 (0.399) Data 0.000 (0.041) Loss 1.433 (1.452) Epoch: [43][370/400] Time 0.374 (0.397) Data 0.001 (0.040) Loss 1.473 (1.455) Epoch: [43][380/400] Time 0.351 (0.396) Data 0.001 (0.039) Loss 1.421 (1.462) Epoch: [43][390/400] Time 0.364 (0.395) Data 0.000 (0.038) Loss 1.107 (1.467) Epoch: [43][400/400] Time 0.352 (0.399) Data 0.000 (0.042) Loss 1.955 (1.477) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.128 (0.162) Data 0.000 (0.029) Computing jaccard distance... Jaccard distance computing time cost: 18.062583684921265 ==> Statistics for epoch 44: 710 clusters Epoch: [44][10/400] Time 0.351 (0.416) Data 0.001 (0.058) Loss 0.100 (0.171) Epoch: [44][20/400] Time 0.349 (0.386) Data 0.001 (0.029) Loss 0.175 (0.184) Epoch: [44][30/400] Time 0.349 (0.375) Data 0.001 (0.020) Loss 0.096 (0.175) Epoch: [44][40/400] Time 0.362 (0.371) Data 0.000 (0.015) Loss 0.086 (0.161) Epoch: [44][50/400] Time 0.361 (0.403) Data 0.001 (0.045) Loss 1.622 (0.371) Epoch: [44][60/400] Time 0.353 (0.394) Data 0.000 (0.038) Loss 1.611 (0.603) Epoch: [44][70/400] Time 0.361 (0.389) Data 0.001 (0.033) Loss 2.232 (0.758) Epoch: [44][80/400] Time 0.354 (0.384) Data 0.000 (0.029) Loss 1.322 (0.880) Epoch: [44][90/400] Time 0.375 (0.404) Data 0.001 (0.047) Loss 1.389 (0.979) Epoch: [44][100/400] Time 0.350 (0.399) Data 0.001 (0.042) Loss 1.671 (1.035) Epoch: [44][110/400] Time 0.352 (0.395) Data 0.001 (0.038) Loss 1.356 (1.089) Epoch: [44][120/400] Time 0.352 (0.391) Data 0.001 (0.035) Loss 0.993 (1.121) Epoch: [44][130/400] Time 0.363 (0.389) Data 0.000 (0.033) Loss 1.272 (1.156) Epoch: [44][140/400] Time 0.350 (0.400) Data 0.001 (0.043) Loss 1.814 (1.187) Epoch: [44][150/400] Time 0.353 (0.397) Data 0.001 (0.040) Loss 1.670 (1.219) Epoch: [44][160/400] Time 0.354 (0.394) Data 0.000 (0.038) Loss 1.312 (1.232) Epoch: [44][170/400] Time 0.362 (0.392) Data 0.000 (0.036) Loss 1.745 (1.254) Epoch: [44][180/400] Time 0.350 (0.401) Data 0.000 (0.044) Loss 2.123 (1.275) Epoch: [44][190/400] Time 0.362 (0.399) Data 0.001 (0.041) Loss 1.214 (1.291) Epoch: [44][200/400] Time 0.379 (0.396) Data 0.001 (0.039) Loss 1.859 (1.312) Epoch: [44][210/400] Time 0.362 (0.394) Data 0.001 (0.038) Loss 1.947 (1.331) Epoch: [44][220/400] Time 0.362 (0.393) Data 0.000 (0.036) Loss 1.031 (1.338) Epoch: [44][230/400] Time 0.350 (0.399) Data 0.001 (0.042) Loss 1.748 (1.343) Epoch: [44][240/400] Time 0.359 (0.397) Data 0.001 (0.040) Loss 1.870 (1.358) Epoch: [44][250/400] Time 0.350 (0.396) Data 0.001 (0.038) Loss 1.781 (1.368) Epoch: [44][260/400] Time 0.362 (0.394) Data 0.000 (0.037) Loss 1.317 (1.380) Epoch: [44][270/400] Time 0.351 (0.399) Data 0.001 (0.042) Loss 1.718 (1.392) Epoch: [44][280/400] Time 0.374 (0.398) Data 0.001 (0.041) Loss 1.636 (1.401) Epoch: [44][290/400] Time 0.352 (0.396) Data 0.001 (0.039) Loss 1.231 (1.410) Epoch: [44][300/400] Time 0.349 (0.395) Data 0.001 (0.038) Loss 1.921 (1.413) Epoch: [44][310/400] Time 0.361 (0.400) Data 0.001 (0.042) Loss 1.383 (1.420) Epoch: [44][320/400] Time 0.350 (0.399) Data 0.001 (0.041) Loss 1.726 (1.432) Epoch: [44][330/400] Time 0.402 (0.397) Data 0.000 (0.040) Loss 1.563 (1.437) Epoch: [44][340/400] Time 0.352 (0.396) Data 0.001 (0.039) Loss 1.032 (1.444) Epoch: [44][350/400] Time 0.362 (0.395) Data 0.000 (0.038) Loss 1.573 (1.448) Epoch: [44][360/400] Time 0.351 (0.398) Data 0.000 (0.041) Loss 1.933 (1.449) Epoch: [44][370/400] Time 0.361 (0.397) Data 0.001 (0.040) Loss 1.482 (1.451) Epoch: [44][380/400] Time 0.362 (0.396) Data 0.000 (0.039) Loss 1.352 (1.454) Epoch: [44][390/400] Time 0.361 (0.395) Data 0.000 (0.038) Loss 1.528 (1.458) Epoch: [44][400/400] Time 0.350 (0.398) Data 0.001 (0.041) Loss 1.346 (1.459) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.161) Data 0.000 (0.029) Computing jaccard distance... Jaccard distance computing time cost: 21.04724669456482 ==> Statistics for epoch 45: 711 clusters Epoch: [45][10/400] Time 0.349 (0.424) Data 0.001 (0.063) Loss 0.153 (0.183) Epoch: [45][20/400] Time 0.350 (0.388) Data 0.000 (0.032) Loss 0.216 (0.170) Epoch: [45][30/400] Time 0.361 (0.376) Data 0.001 (0.021) Loss 0.097 (0.163) Epoch: [45][40/400] Time 0.361 (0.371) Data 0.000 (0.016) Loss 0.113 (0.162) Epoch: [45][50/400] Time 0.352 (0.403) Data 0.000 (0.048) Loss 1.543 (0.351) Epoch: [45][60/400] Time 0.350 (0.395) Data 0.001 (0.040) Loss 2.020 (0.570) Epoch: [45][70/400] Time 0.354 (0.389) Data 0.001 (0.034) Loss 1.971 (0.753) Epoch: [45][80/400] Time 0.350 (0.384) Data 0.000 (0.030) Loss 1.234 (0.868) Epoch: [45][90/400] Time 0.342 (0.401) Data 0.001 (0.046) Loss 1.229 (0.964) Epoch: [45][100/400] Time 0.353 (0.397) Data 0.000 (0.042) Loss 1.315 (1.020) Epoch: [45][110/400] Time 0.363 (0.393) Data 0.000 (0.038) Loss 1.671 (1.056) Epoch: [45][120/400] Time 0.353 (0.390) Data 0.001 (0.035) Loss 1.119 (1.102) Epoch: [45][130/400] Time 0.363 (0.388) Data 0.000 (0.032) Loss 1.328 (1.139) Epoch: [45][140/400] Time 0.353 (0.399) Data 0.001 (0.043) Loss 0.924 (1.176) Epoch: [45][150/400] Time 0.353 (0.396) Data 0.001 (0.040) Loss 1.637 (1.204) Epoch: [45][160/400] Time 0.369 (0.394) Data 0.001 (0.038) Loss 1.266 (1.219) Epoch: [45][170/400] Time 0.364 (0.391) Data 0.000 (0.036) Loss 1.894 (1.242) Epoch: [45][180/400] Time 0.350 (0.399) Data 0.000 (0.043) Loss 1.150 (1.257) Epoch: [45][190/400] Time 0.352 (0.397) Data 0.001 (0.041) Loss 2.173 (1.273) Epoch: [45][200/400] Time 0.351 (0.394) Data 0.001 (0.039) Loss 1.493 (1.288) Epoch: [45][210/400] Time 0.353 (0.392) Data 0.001 (0.037) Loss 1.505 (1.299) Epoch: [45][220/400] Time 0.363 (0.391) Data 0.000 (0.035) Loss 1.737 (1.314) Epoch: [45][230/400] Time 0.368 (0.397) Data 0.001 (0.041) Loss 2.321 (1.332) Epoch: [45][240/400] Time 0.353 (0.395) Data 0.000 (0.039) Loss 1.704 (1.346) Epoch: [45][250/400] Time 0.364 (0.394) Data 0.001 (0.038) Loss 1.752 (1.354) Epoch: [45][260/400] Time 0.365 (0.392) Data 0.000 (0.036) Loss 2.220 (1.366) Epoch: [45][270/400] Time 0.351 (0.398) Data 0.000 (0.042) Loss 1.526 (1.373) Epoch: [45][280/400] Time 0.365 (0.397) Data 0.000 (0.040) Loss 1.280 (1.377) Epoch: [45][290/400] Time 0.352 (0.395) Data 0.000 (0.039) Loss 1.631 (1.381) Epoch: [45][300/400] Time 0.374 (0.394) Data 0.001 (0.038) Loss 1.594 (1.390) Epoch: [45][310/400] Time 0.377 (0.399) Data 0.001 (0.042) Loss 1.777 (1.398) Epoch: [45][320/400] Time 0.352 (0.398) Data 0.000 (0.041) Loss 1.706 (1.402) Epoch: [45][330/400] Time 0.351 (0.397) Data 0.001 (0.040) Loss 1.670 (1.403) Epoch: [45][340/400] Time 0.351 (0.395) Data 0.001 (0.039) Loss 1.918 (1.414) Epoch: [45][350/400] Time 0.365 (0.394) Data 0.000 (0.038) Loss 2.267 (1.425) Epoch: [45][360/400] Time 0.355 (0.398) Data 0.000 (0.041) Loss 1.025 (1.431) Epoch: [45][370/400] Time 0.350 (0.397) Data 0.001 (0.040) Loss 1.433 (1.431) Epoch: [45][380/400] Time 0.370 (0.396) Data 0.001 (0.039) Loss 1.009 (1.433) Epoch: [45][390/400] Time 0.363 (0.395) Data 0.000 (0.038) Loss 1.035 (1.432) Epoch: [45][400/400] Time 0.348 (0.399) Data 0.000 (0.042) Loss 1.340 (1.434) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.158) Data 0.000 (0.027) Computing jaccard distance... Jaccard distance computing time cost: 18.08151888847351 ==> Statistics for epoch 46: 711 clusters Epoch: [46][10/400] Time 0.399 (0.430) Data 0.001 (0.066) Loss 0.155 (0.199) Epoch: [46][20/400] Time 0.351 (0.391) Data 0.001 (0.033) Loss 0.134 (0.200) Epoch: [46][30/400] Time 0.351 (0.378) Data 0.001 (0.022) Loss 0.098 (0.187) Epoch: [46][40/400] Time 0.363 (0.372) Data 0.000 (0.017) Loss 0.085 (0.180) Epoch: [46][50/400] Time 0.352 (0.407) Data 0.001 (0.051) Loss 1.418 (0.354) Epoch: [46][60/400] Time 0.350 (0.398) Data 0.001 (0.043) Loss 1.741 (0.585) Epoch: [46][70/400] Time 0.354 (0.392) Data 0.001 (0.037) Loss 1.096 (0.742) Epoch: [46][80/400] Time 0.342 (0.387) Data 0.001 (0.032) Loss 1.535 (0.850) Epoch: [46][90/400] Time 0.353 (0.404) Data 0.000 (0.049) Loss 1.479 (0.952) Epoch: [46][100/400] Time 0.351 (0.400) Data 0.001 (0.044) Loss 1.431 (1.020) Epoch: [46][110/400] Time 0.363 (0.396) Data 0.001 (0.040) Loss 1.496 (1.060) Epoch: [46][120/400] Time 0.355 (0.392) Data 0.001 (0.037) Loss 1.418 (1.109) Epoch: [46][130/400] Time 0.362 (0.390) Data 0.000 (0.034) Loss 1.688 (1.133) Epoch: [46][140/400] Time 0.351 (0.402) Data 0.000 (0.045) Loss 1.732 (1.168) Epoch: [46][150/400] Time 0.351 (0.399) Data 0.001 (0.042) Loss 1.358 (1.199) Epoch: [46][160/400] Time 0.353 (0.396) Data 0.001 (0.040) Loss 1.810 (1.219) Epoch: [46][170/400] Time 0.361 (0.394) Data 0.000 (0.037) Loss 1.586 (1.231) Epoch: [46][180/400] Time 0.351 (0.403) Data 0.000 (0.046) Loss 1.650 (1.247) Epoch: [46][190/400] Time 0.404 (0.400) Data 0.001 (0.043) Loss 1.439 (1.257) Epoch: [46][200/400] Time 0.352 (0.398) Data 0.000 (0.041) Loss 1.318 (1.285) Epoch: [46][210/400] Time 0.364 (0.396) Data 0.001 (0.039) Loss 2.036 (1.298) Epoch: [46][220/400] Time 0.363 (0.395) Data 0.000 (0.037) Loss 1.743 (1.314) Epoch: [46][230/400] Time 0.362 (0.402) Data 0.001 (0.044) Loss 1.166 (1.324) Epoch: [46][240/400] Time 0.352 (0.400) Data 0.001 (0.042) Loss 1.346 (1.330) Epoch: [46][250/400] Time 0.362 (0.398) Data 0.001 (0.040) Loss 1.890 (1.340) Epoch: [46][260/400] Time 0.363 (0.397) Data 0.000 (0.039) Loss 1.414 (1.350) Epoch: [46][270/400] Time 0.353 (0.402) Data 0.001 (0.044) Loss 1.933 (1.366) Epoch: [46][280/400] Time 0.394 (0.401) Data 0.001 (0.043) Loss 1.450 (1.377) Epoch: [46][290/400] Time 0.353 (0.399) Data 0.001 (0.041) Loss 1.843 (1.385) Epoch: [46][300/400] Time 0.351 (0.397) Data 0.001 (0.040) Loss 1.477 (1.392) Epoch: [46][310/400] Time 0.366 (0.402) Data 0.001 (0.045) Loss 1.166 (1.396) Epoch: [46][320/400] Time 0.353 (0.401) Data 0.000 (0.043) Loss 1.909 (1.408) Epoch: [46][330/400] Time 0.355 (0.399) Data 0.001 (0.042) Loss 0.863 (1.415) Epoch: [46][340/400] Time 0.351 (0.398) Data 0.000 (0.041) Loss 1.236 (1.418) Epoch: [46][350/400] Time 0.363 (0.397) Data 0.000 (0.039) Loss 1.864 (1.420) Epoch: [46][360/400] Time 0.350 (0.401) Data 0.001 (0.043) Loss 1.548 (1.426) Epoch: [46][370/400] Time 0.372 (0.399) Data 0.001 (0.042) Loss 2.012 (1.433) Epoch: [46][380/400] Time 0.352 (0.398) Data 0.001 (0.041) Loss 1.236 (1.436) Epoch: [46][390/400] Time 0.363 (0.397) Data 0.000 (0.040) Loss 1.478 (1.440) Epoch: [46][400/400] Time 0.352 (0.401) Data 0.000 (0.043) Loss 1.541 (1.445) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.162) Data 0.000 (0.030) Computing jaccard distance... Jaccard distance computing time cost: 18.107317209243774 ==> Statistics for epoch 47: 711 clusters Epoch: [47][10/400] Time 0.352 (0.414) Data 0.001 (0.053) Loss 0.259 (0.188) Epoch: [47][20/400] Time 0.350 (0.384) Data 0.001 (0.027) Loss 0.162 (0.175) Epoch: [47][30/400] Time 0.360 (0.374) Data 0.001 (0.018) Loss 0.084 (0.162) Epoch: [47][40/400] Time 0.360 (0.370) Data 0.000 (0.014) Loss 0.134 (0.160) Epoch: [47][50/400] Time 0.362 (0.403) Data 0.001 (0.046) Loss 1.716 (0.362) Epoch: [47][60/400] Time 0.353 (0.396) Data 0.000 (0.038) Loss 1.623 (0.611) Epoch: [47][70/400] Time 0.350 (0.390) Data 0.001 (0.033) Loss 1.651 (0.768) Epoch: [47][80/400] Time 0.353 (0.385) Data 0.000 (0.029) Loss 1.758 (0.875) Epoch: [47][90/400] Time 0.365 (0.404) Data 0.000 (0.047) Loss 1.572 (0.954) Epoch: [47][100/400] Time 0.352 (0.400) Data 0.001 (0.042) Loss 1.840 (1.007) Epoch: [47][110/400] Time 0.351 (0.396) Data 0.000 (0.038) Loss 1.568 (1.036) Epoch: [47][120/400] Time 0.352 (0.392) Data 0.001 (0.035) Loss 1.460 (1.069) Epoch: [47][130/400] Time 0.362 (0.390) Data 0.000 (0.033) Loss 1.767 (1.103) Epoch: [47][140/400] Time 0.352 (0.399) Data 0.001 (0.042) Loss 1.280 (1.132) Epoch: [47][150/400] Time 0.362 (0.396) Data 0.001 (0.040) Loss 1.597 (1.168) Epoch: [47][160/400] Time 0.352 (0.394) Data 0.001 (0.037) Loss 1.718 (1.201) Epoch: [47][170/400] Time 0.362 (0.392) Data 0.000 (0.035) Loss 0.969 (1.217) Epoch: [47][180/400] Time 0.353 (0.401) Data 0.000 (0.044) Loss 1.759 (1.229) Epoch: [47][190/400] Time 0.360 (0.398) Data 0.001 (0.041) Loss 1.096 (1.246) Epoch: [47][200/400] Time 0.353 (0.396) Data 0.000 (0.039) Loss 1.713 (1.260) Epoch: [47][210/400] Time 0.361 (0.394) Data 0.001 (0.037) Loss 1.732 (1.281) Epoch: [47][220/400] Time 0.362 (0.393) Data 0.000 (0.036) Loss 1.835 (1.304) Epoch: [47][230/400] Time 0.356 (0.399) Data 0.001 (0.042) Loss 1.515 (1.315) Epoch: [47][240/400] Time 0.353 (0.397) Data 0.000 (0.040) Loss 1.414 (1.325) Epoch: [47][250/400] Time 0.363 (0.395) Data 0.000 (0.038) Loss 1.460 (1.333) Epoch: [47][260/400] Time 0.362 (0.394) Data 0.000 (0.037) Loss 1.325 (1.346) Epoch: [47][270/400] Time 0.353 (0.399) Data 0.000 (0.042) Loss 1.254 (1.352) Epoch: [47][280/400] Time 0.355 (0.398) Data 0.000 (0.040) Loss 1.654 (1.357) Epoch: [47][290/400] Time 0.352 (0.396) Data 0.000 (0.039) Loss 1.070 (1.363) Epoch: [47][300/400] Time 0.354 (0.395) Data 0.000 (0.038) Loss 1.354 (1.372) Epoch: [47][310/400] Time 0.370 (0.399) Data 0.001 (0.042) Loss 1.563 (1.378) Epoch: [47][320/400] Time 0.353 (0.398) Data 0.000 (0.041) Loss 1.460 (1.387) Epoch: [47][330/400] Time 0.359 (0.397) Data 0.000 (0.039) Loss 1.687 (1.392) Epoch: [47][340/400] Time 0.355 (0.395) Data 0.000 (0.038) Loss 1.769 (1.396) Epoch: [47][350/400] Time 0.363 (0.394) Data 0.000 (0.037) Loss 1.445 (1.403) Epoch: [47][360/400] Time 0.355 (0.399) Data 0.000 (0.041) Loss 2.206 (1.409) Epoch: [47][370/400] Time 0.364 (0.398) Data 0.000 (0.040) Loss 1.943 (1.410) Epoch: [47][380/400] Time 0.355 (0.397) Data 0.000 (0.039) Loss 1.526 (1.411) Epoch: [47][390/400] Time 0.364 (0.396) Data 0.000 (0.038) Loss 1.535 (1.410) Epoch: [47][400/400] Time 0.353 (0.399) Data 0.000 (0.042) Loss 1.390 (1.411) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.160) Data 0.000 (0.027) Computing jaccard distance... Jaccard distance computing time cost: 18.148759126663208 ==> Statistics for epoch 48: 709 clusters Epoch: [48][10/400] Time 0.351 (0.418) Data 0.000 (0.054) Loss 0.182 (0.190) Epoch: [48][20/400] Time 0.351 (0.386) Data 0.000 (0.027) Loss 0.112 (0.171) Epoch: [48][30/400] Time 0.356 (0.375) Data 0.000 (0.018) Loss 0.071 (0.161) Epoch: [48][40/400] Time 0.363 (0.370) Data 0.000 (0.014) Loss 0.066 (0.155) Epoch: [48][50/400] Time 0.350 (0.407) Data 0.000 (0.049) Loss 1.437 (0.322) Epoch: [48][60/400] Time 0.355 (0.399) Data 0.000 (0.041) Loss 2.100 (0.550) Epoch: [48][70/400] Time 0.353 (0.393) Data 0.000 (0.035) Loss 1.520 (0.708) Epoch: [48][80/400] Time 0.353 (0.389) Data 0.000 (0.031) Loss 1.874 (0.848) Epoch: [48][90/400] Time 0.347 (0.406) Data 0.001 (0.047) Loss 1.481 (0.942) Epoch: [48][100/400] Time 0.358 (0.401) Data 0.000 (0.042) Loss 1.607 (1.003) Epoch: [48][110/400] Time 0.371 (0.397) Data 0.000 (0.039) Loss 1.408 (1.061) Epoch: [48][120/400] Time 0.352 (0.393) Data 0.000 (0.035) Loss 1.642 (1.106) Epoch: [48][130/400] Time 0.364 (0.391) Data 0.000 (0.033) Loss 1.447 (1.133) Epoch: [48][140/400] Time 0.354 (0.401) Data 0.000 (0.044) Loss 1.327 (1.156) Epoch: [48][150/400] Time 0.362 (0.399) Data 0.001 (0.041) Loss 2.500 (1.184) Epoch: [48][160/400] Time 0.354 (0.396) Data 0.000 (0.038) Loss 1.433 (1.206) Epoch: [48][170/400] Time 0.364 (0.394) Data 0.000 (0.036) Loss 0.955 (1.232) Epoch: [48][180/400] Time 0.344 (0.402) Data 0.000 (0.044) Loss 1.659 (1.252) Epoch: [48][190/400] Time 0.362 (0.399) Data 0.001 (0.041) Loss 0.991 (1.257) Epoch: [48][200/400] Time 0.353 (0.397) Data 0.001 (0.039) Loss 2.188 (1.290) Epoch: [48][210/400] Time 0.352 (0.395) Data 0.001 (0.037) Loss 1.919 (1.307) Epoch: [48][220/400] Time 0.363 (0.394) Data 0.000 (0.036) Loss 1.209 (1.316) Epoch: [48][230/400] Time 0.363 (0.400) Data 0.001 (0.042) Loss 2.199 (1.335) Epoch: [48][240/400] Time 0.355 (0.398) Data 0.000 (0.040) Loss 1.371 (1.350) Epoch: [48][250/400] Time 0.355 (0.396) Data 0.001 (0.038) Loss 1.599 (1.358) Epoch: [48][260/400] Time 0.364 (0.395) Data 0.000 (0.037) Loss 1.218 (1.362) Epoch: [48][270/400] Time 0.355 (0.400) Data 0.000 (0.042) Loss 2.434 (1.374) Epoch: [48][280/400] Time 0.355 (0.399) Data 0.000 (0.041) Loss 1.247 (1.376) Epoch: [48][290/400] Time 0.355 (0.397) Data 0.000 (0.039) Loss 1.527 (1.383) Epoch: [48][300/400] Time 0.355 (0.396) Data 0.000 (0.038) Loss 1.490 (1.389) Epoch: [48][310/400] Time 0.377 (0.401) Data 0.001 (0.043) Loss 1.530 (1.395) Epoch: [48][320/400] Time 0.353 (0.400) Data 0.000 (0.041) Loss 1.689 (1.405) Epoch: [48][330/400] Time 0.352 (0.398) Data 0.001 (0.040) Loss 1.653 (1.412) Epoch: [48][340/400] Time 0.354 (0.397) Data 0.000 (0.039) Loss 1.388 (1.413) Epoch: [48][350/400] Time 0.364 (0.396) Data 0.000 (0.038) Loss 1.497 (1.417) Epoch: [48][360/400] Time 0.354 (0.400) Data 0.000 (0.042) Loss 1.494 (1.423) Epoch: [48][370/400] Time 0.363 (0.399) Data 0.000 (0.041) Loss 1.442 (1.429) Epoch: [48][380/400] Time 0.369 (0.398) Data 0.000 (0.039) Loss 1.703 (1.432) Epoch: [48][390/400] Time 0.364 (0.397) Data 0.000 (0.038) Loss 1.562 (1.434) Epoch: [48][400/400] Time 0.377 (0.401) Data 0.000 (0.042) Loss 1.649 (1.438) ==> Create pseudo labels for unlabeled data Extract Features: [50/51] Time 0.129 (0.160) Data 0.000 (0.028) Computing jaccard distance... Jaccard distance computing time cost: 18.608444452285767 ==> Statistics for epoch 49: 709 clusters Epoch: [49][10/400] Time 0.351 (0.417) Data 0.000 (0.054) Loss 0.151 (0.166) Epoch: [49][20/400] Time 0.352 (0.385) Data 0.000 (0.027) Loss 0.313 (0.176) Epoch: [49][30/400] Time 0.351 (0.374) Data 0.000 (0.018) Loss 0.198 (0.176) Epoch: [49][40/400] Time 0.363 (0.370) Data 0.000 (0.014) Loss 0.085 (0.161) Epoch: [49][50/400] Time 0.352 (0.404) Data 0.001 (0.047) Loss 1.400 (0.342) Epoch: [49][60/400] Time 0.353 (0.397) Data 0.000 (0.039) Loss 2.070 (0.582) Epoch: [49][70/400] Time 0.355 (0.391) Data 0.000 (0.034) Loss 1.482 (0.739) Epoch: [49][80/400] Time 0.353 (0.387) Data 0.000 (0.030) Loss 2.275 (0.853) Epoch: [49][90/400] Time 0.363 (0.406) Data 0.000 (0.047) Loss 1.722 (0.925) Epoch: [49][100/400] Time 0.353 (0.400) Data 0.000 (0.043) Loss 1.248 (0.977) Epoch: [49][110/400] Time 0.369 (0.397) Data 0.000 (0.039) Loss 1.271 (1.035) Epoch: [49][120/400] Time 0.354 (0.393) Data 0.000 (0.036) Loss 1.528 (1.084) Epoch: [49][130/400] Time 0.364 (0.390) Data 0.000 (0.033) Loss 1.895 (1.132) Epoch: [49][140/400] Time 0.354 (0.400) Data 0.000 (0.043) Loss 1.840 (1.170) Epoch: [49][150/400] Time 0.352 (0.397) Data 0.001 (0.040) Loss 1.704 (1.212) Epoch: [49][160/400] Time 0.398 (0.395) Data 0.000 (0.037) Loss 2.271 (1.241) Epoch: [49][170/400] Time 0.363 (0.393) Data 0.000 (0.035) Loss 2.027 (1.259) Epoch: [49][180/400] Time 0.363 (0.401) Data 0.000 (0.043) Loss 2.174 (1.275) Epoch: [49][190/400] Time 0.353 (0.399) Data 0.000 (0.041) Loss 2.098 (1.295) Epoch: [49][200/400] Time 0.376 (0.396) Data 0.000 (0.039) Loss 1.620 (1.300) Epoch: [49][210/400] Time 0.353 (0.395) Data 0.000 (0.037) Loss 1.721 (1.313) Epoch: [49][220/400] Time 0.363 (0.393) Data 0.000 (0.035) Loss 1.708 (1.329) Epoch: [49][230/400] Time 0.357 (0.399) Data 0.001 (0.042) Loss 2.011 (1.338) Epoch: [49][240/400] Time 0.354 (0.398) Data 0.001 (0.040) Loss 1.132 (1.342) Epoch: [49][250/400] Time 0.357 (0.396) Data 0.001 (0.038) Loss 1.602 (1.350) Epoch: [49][260/400] Time 0.363 (0.395) Data 0.000 (0.037) Loss 2.304 (1.359) Epoch: [49][270/400] Time 0.362 (0.400) Data 0.000 (0.042) Loss 1.903 (1.366) Epoch: [49][280/400] Time 0.355 (0.398) Data 0.000 (0.040) Loss 1.882 (1.369) Epoch: [49][290/400] Time 0.363 (0.397) Data 0.001 (0.039) Loss 1.252 (1.375) Epoch: [49][300/400] Time 0.354 (0.395) Data 0.000 (0.038) Loss 1.802 (1.379) Epoch: [49][310/400] Time 0.421 (0.400) Data 0.000 (0.042) Loss 1.358 (1.386) Epoch: [49][320/400] Time 0.354 (0.399) Data 0.000 (0.041) Loss 1.961 (1.395) Epoch: [49][330/400] Time 0.357 (0.398) Data 0.000 (0.040) Loss 1.538 (1.406) Epoch: [49][340/400] Time 0.353 (0.397) Data 0.000 (0.039) Loss 2.063 (1.406) Epoch: [49][350/400] Time 0.365 (0.395) Data 0.000 (0.037) Loss 1.933 (1.405) Epoch: [49][360/400] Time 0.354 (0.399) Data 0.000 (0.041) Loss 1.433 (1.406) Epoch: [49][370/400] Time 0.353 (0.398) Data 0.000 (0.040) Loss 1.375 (1.409) Epoch: [49][380/400] Time 0.353 (0.397) Data 0.000 (0.039) Loss 1.282 (1.410) Epoch: [49][390/400] Time 0.363 (0.396) Data 0.000 (0.038) Loss 1.522 (1.420) Epoch: [49][400/400] Time 0.349 (0.400) Data 0.001 (0.042) Loss 1.714 (1.428) Extract Features: [50/76] Time 0.133 (0.160) Data 0.000 (0.027) Mean AP: 87.0% * Finished epoch 49 model mAP: 87.0% best: 87.0% * ==> Test with the best model: => Loaded checkpoint '/data0/developer/cluster-contrast/examples/logs/gem-hard/model_best.pth.tar' Extract Features: [50/76] Time 0.134 (0.162) Data 0.000 (0.028) Mean AP: 87.0% CMC Scores: top-1 94.6% top-5 98.2% top-10 98.8% Total running time: 2:44:06.629802 ================================================ FILE: examples/pretrained/resnet50-19c8e357.pth ================================================ [File too large to display: 97.8 MB] ================================================ FILE: examples/test.py ================================================ from __future__ import print_function, absolute_import import argparse import os.path as osp import random import numpy as np import sys import torch from torch import nn from torch.backends import cudnn from torch.utils.data import DataLoader from clustercontrast import datasets from clustercontrast import models from clustercontrast.models.dsbn import convert_dsbn, convert_bn from clustercontrast.evaluators import Evaluator from clustercontrast.utils.data import transforms as T from clustercontrast.utils.data.preprocessor import Preprocessor from clustercontrast.utils.logging import Logger from clustercontrast.utils.serialization import load_checkpoint, save_checkpoint, copy_state_dict def get_data(name, data_dir, height, width, batch_size, workers): root = osp.join(data_dir, name) dataset = datasets.create(name, root) normalizer = T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]) test_transformer = T.Compose([ T.Resize((height, width), interpolation=3), T.ToTensor(), normalizer ]) test_loader = DataLoader( Preprocessor(list(set(dataset.query) | set(dataset.gallery)), root=dataset.images_dir, transform=test_transformer), batch_size=batch_size, num_workers=workers, shuffle=False, pin_memory=True) return dataset, test_loader def main(): args = parser.parse_args() if args.seed is not None: random.seed(args.seed) np.random.seed(args.seed) torch.manual_seed(args.seed) cudnn.deterministic = True main_worker(args) def main_worker(args): cudnn.benchmark = True log_dir = osp.dirname(args.resume) sys.stdout = Logger(osp.join(log_dir, 'log_test.txt')) print("==========\nArgs:{}\n==========".format(args)) # Create data loaders dataset, test_loader = get_data(args.dataset, args.data_dir, args.height, args.width, args.batch_size, args.workers) # Create model model = models.create(args.arch, pretrained=False, num_features=args.features, dropout=args.dropout, num_classes=0, pooling_type=args.pooling_type) if args.dsbn: print("==> Load the model with domain-specific BNs") convert_dsbn(model) # Load from checkpoint checkpoint = load_checkpoint(args.resume) copy_state_dict(checkpoint['state_dict'], model, strip='module.') if args.dsbn: print("==> Test with {}-domain BNs".format("source" if args.test_source else "target")) convert_bn(model, use_target=(not args.test_source)) model.cuda() model = nn.DataParallel(model) # Evaluator model.eval() evaluator = Evaluator(model) evaluator.evaluate(test_loader, dataset.query, dataset.gallery, cmc_flag=True, rerank=args.rerank) return if __name__ == '__main__': parser = argparse.ArgumentParser(description="Testing the model") # data parser.add_argument('-d', '--dataset', type=str, default='market1501') parser.add_argument('-b', '--batch-size', type=int, default=256) parser.add_argument('-j', '--workers', type=int, default=4) parser.add_argument('--height', type=int, default=256, help="input height") parser.add_argument('--width', type=int, default=128, help="input width") # model parser.add_argument('-a', '--arch', type=str, default='resnet50', choices=models.names()) parser.add_argument('--features', type=int, default=0) parser.add_argument('--dropout', type=float, default=0) parser.add_argument('--resume', type=str, default="/media/yixuan/DATA/cluster-contrast/market-res50/logs/model_best.pth.tar", metavar='PATH') # testing configs parser.add_argument('--rerank', action='store_true', help="evaluation only") parser.add_argument('--dsbn', action='store_true', help="test on the model with domain-specific BN") parser.add_argument('--test-source', action='store_true', help="test on the source domain") parser.add_argument('--seed', type=int, default=1) # path working_dir = osp.dirname(osp.abspath(__file__)) parser.add_argument('--data-dir', type=str, metavar='PATH', default='/media/yixuan/Project/guangyuan/workpalces/SpCL/examples/data') parser.add_argument('--pooling-type', type=str, default='gem') parser.add_argument('--embedding_features_path', type=str, default='/media/yixuan/Project/guangyuan/workpalces/SpCL/embedding_features/mark1501_res50_ibn/') main() ================================================ FILE: run_code.sh ================================================ CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl.py -b 256 -a resnet50 -d market1501 --iters 200 --momentum 0.1 --eps 0.6 --num-instances 16 # CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl_infomap.py -b 256 -a resnet50 -d market1501 --iters 200 --momentum 0.1 --eps 0.5 --k1 15 --k2 4 --num-instances 16 # CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl.py -b 256 -a resnet50 -d msmt17 --iters 400 --momentum 0.1 --eps 0.6 --num-instances 16 # CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl_infomap.py -b 256 -a resnet50 -d msmt17 --iters 400 --momentum 0.1 --eps 0.5 --k1 15 --k2 4 --num-instances 16 # CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl.py -b 256 -a resnet50 -d dukemtmcreid --iters 200 --momentum 0.1 --eps 0.6 --num-instances 16 # CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl_infomap.py -b 256 -a resnet50 -d dukemtmcreid --iters 200 --momentum 0.1 --eps 0.5 --k1 15 --k2 4 --num-instances 16 # CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl.py -b 256 -a resnet50 -d veri --iters 400 --momentum 0.1 --eps 0.6 --num-instances 16 --height 224 --width 224 # CUDA_VISIBLE_DEVICES=0,1,2,3 python examples/cluster_contrast_train_usl_infomap.py -b 256 -a resnet50 -d veri --iters 400 --momentum 0.1 --eps 0.5 --k1 15 --k2 4 --num-instances 16 --height 224 --width 224 ================================================ FILE: setup.py ================================================ from setuptools import setup, find_packages setup(name='ClusterContrast', version='1.0.0', description='Cluster Contrast for Unsupervised Person Re-Identification', author='GuangYuan wang', author_email='yixuan.wgy@alibaba-inc.com', # url='', install_requires=[ 'numpy', 'torch', 'torchvision', 'six', 'h5py', 'Pillow', 'scipy', 'scikit-learn', 'metric-learn', 'faiss_gpu'], packages=find_packages(), keywords=[ 'Unsupervised Learning', 'Contrastive Learning', 'Object Re-identification' ])