Showing preview only (1,179K chars total). Download the full file or copy to clipboard to get everything.
Repository: yanxp/MetaR-CNN
Branch: master
Commit: 5a4487e78e40
Files: 117
Total size: 1.1 MB
Directory structure:
gitextract_940a232f/
├── .gitignore
├── README.md
├── _init_paths.py
├── cfgs/
│ ├── res101_ms.yml
│ └── res50.yml
├── lib/
│ ├── datasets/
│ │ ├── VOCdevkit-matlab-wrapper/
│ │ │ ├── get_voc_opts.m
│ │ │ ├── voc_eval.m
│ │ │ └── xVOCap.m
│ │ ├── __init__.py
│ │ ├── coco.py
│ │ ├── ds_utils.py
│ │ ├── factory.py
│ │ ├── imdb.py
│ │ ├── metadata.py
│ │ ├── pascal_voc.py
│ │ ├── pascal_voc_rbg.py
│ │ ├── tools/
│ │ │ ├── compute_prior.py
│ │ │ └── mcg_munge.py
│ │ └── voc_eval.py
│ ├── model/
│ │ ├── __init__.py
│ │ ├── faster_rcnn/
│ │ │ ├── __init__.py
│ │ │ ├── faster_rcnn.py
│ │ │ ├── resnet.py
│ │ │ ├── trail.py
│ │ │ └── vgg16.py
│ │ ├── nms/
│ │ │ ├── .gitignore
│ │ │ ├── __init__.py
│ │ │ ├── _ext/
│ │ │ │ ├── __init__.py
│ │ │ │ └── nms/
│ │ │ │ └── __init__.py
│ │ │ ├── build.py
│ │ │ ├── nms_cpu.py
│ │ │ ├── nms_gpu.py
│ │ │ ├── nms_kernel.cu
│ │ │ ├── nms_wrapper.py
│ │ │ └── src/
│ │ │ ├── nms_cuda.h
│ │ │ ├── nms_cuda_kernel.cu
│ │ │ ├── nms_cuda_kernel.cu.o
│ │ │ └── nms_cuda_kernel.h
│ │ ├── roi_align/
│ │ │ ├── __init__.py
│ │ │ ├── _ext/
│ │ │ │ ├── __init__.py
│ │ │ │ └── roi_align/
│ │ │ │ └── __init__.py
│ │ │ ├── build.py
│ │ │ ├── functions/
│ │ │ │ ├── __init__.py
│ │ │ │ └── roi_align.py
│ │ │ ├── modules/
│ │ │ │ ├── __init__.py
│ │ │ │ └── roi_align.py
│ │ │ └── src/
│ │ │ ├── roi_align_cuda.c
│ │ │ ├── roi_align_cuda.h
│ │ │ ├── roi_align_kernel.cu
│ │ │ ├── roi_align_kernel.cu.o
│ │ │ └── roi_align_kernel.h
│ │ ├── roi_crop/
│ │ │ ├── __init__.py
│ │ │ ├── _ext/
│ │ │ │ ├── __init__.py
│ │ │ │ └── roi_crop/
│ │ │ │ └── __init__.py
│ │ │ ├── build.py
│ │ │ ├── functions/
│ │ │ │ ├── __init__.py
│ │ │ │ ├── crop_resize.py
│ │ │ │ ├── gridgen.py
│ │ │ │ └── roi_crop.py
│ │ │ ├── modules/
│ │ │ │ ├── __init__.py
│ │ │ │ ├── gridgen.py
│ │ │ │ └── roi_crop.py
│ │ │ └── src/
│ │ │ ├── roi_crop.c
│ │ │ ├── roi_crop.h
│ │ │ ├── roi_crop_cuda.c
│ │ │ ├── roi_crop_cuda.h
│ │ │ ├── roi_crop_cuda_kernel.cu
│ │ │ ├── roi_crop_cuda_kernel.cu.o
│ │ │ └── roi_crop_cuda_kernel.h
│ │ ├── roi_pooling/
│ │ │ ├── __init__.py
│ │ │ ├── _ext/
│ │ │ │ ├── __init__.py
│ │ │ │ └── roi_pooling/
│ │ │ │ └── __init__.py
│ │ │ ├── build.py
│ │ │ ├── functions/
│ │ │ │ ├── __init__.py
│ │ │ │ └── roi_pool.py
│ │ │ ├── modules/
│ │ │ │ ├── __init__.py
│ │ │ │ └── roi_pool.py
│ │ │ └── src/
│ │ │ ├── roi_pooling.c
│ │ │ ├── roi_pooling.cu.o
│ │ │ ├── roi_pooling.h
│ │ │ ├── roi_pooling_cuda.c
│ │ │ ├── roi_pooling_cuda.h
│ │ │ ├── roi_pooling_kernel.cu
│ │ │ └── roi_pooling_kernel.h
│ │ ├── rpn/
│ │ │ ├── __init__.py
│ │ │ ├── anchor_target_layer.py
│ │ │ ├── bbox_transform.py
│ │ │ ├── generate_anchors.py
│ │ │ ├── proposal_layer.py
│ │ │ ├── proposal_layer_region.py
│ │ │ ├── proposal_target_layer_cascade.py
│ │ │ ├── proposal_target_layer_cascade_region.py
│ │ │ ├── rpn.py
│ │ │ └── rpn_region.py
│ │ └── utils/
│ │ ├── .gitignore
│ │ ├── __init__.py
│ │ ├── bbox.pyx
│ │ ├── blob.py
│ │ ├── config.py
│ │ └── net_utils.py
│ ├── pycocotools/
│ │ ├── UPSTREAM_REV
│ │ ├── __init__.py
│ │ ├── _mask.c
│ │ ├── _mask.pyx
│ │ ├── coco.py
│ │ ├── cocoeval.py
│ │ ├── license.txt
│ │ ├── mask.py
│ │ ├── maskApi.c
│ │ └── maskApi.h
│ ├── roi_data_layer/
│ │ ├── __init__.py
│ │ ├── minibatch.py
│ │ ├── roibatchLoader.py
│ │ └── roidb.py
│ └── setup.py
├── test_metarcnn.py
└── train_metarcnn.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
attent*
log*
*.pth
checkpoint
1st
allmetaclass
base*
output*
split*
result*
*.sh
================================================
FILE: README.md
================================================
## Meta R-CNN : Towards General Solver for Instance-level Low-shot Learning.
Code for reproducing the results in the following paper, and the code is built on top of [jwyang/faster-rcnn.pytorch](https://github.com/jwyang/faster-rcnn.pytorch)
**<a href="https://yanxp.github.io/metarcnn.html">Meta R-CNN : Towards General Solver for Instance-level Low-shot Learning</a>**
<a href="https://yanxp.github.io/">Xiaopeng Yan*</a>,
<a href="http://cziliang.com">Ziliang Chen*</a>,
Anni Xu, Xiaoxi Wang,
<a href="https://lemondan.github.io/">Xiaodan Liang</a>,
<a href="http://www.linliang.net/">Liang Lin</a>
Sun Yat-Sen University, Presented at *IEEE International Conference on Computer Vision [(ICCV2019)](http://iccv2019.thecvf.com/)*
<p align=center><img width="80%" src="demo/metarcnn.png"/></p>
### License
For Academic Research Use Only!
### Requirements
+ python packages
+ PyTorch = 0.3.1
*This project can not support pytorch 0.4, higher version will not recur results.*
+ Torchvision >= 0.2.0
+ cython
+ pyyaml
+ easydict
+ opencv-python
+ matplotlib
+ numpy
+ scipy
+ tensorboardX
You can install above package using ```pip```:
```sh
pip install Cython easydict matplotlib opencv-python pyyaml scipy
```
+ CUDA 8.0
+ gcc >= 4.9
### Misc
Tested on Ubuntu 14.04 with a Titan X GPU (12G) and Intel(R) Xeon(R) CPU E5-2623 v3 @ 3.00GHz.
### Getting Started
Clone the repo:
```
https://github.com/yanxp/MetaR-CNN.git
```
### Compilation
Compile the CUDA dependencies:
```sh
cd {repo_root}/lib
sh make.sh
```
It will compile all the modules you need, including NMS, ROI_Pooing, ROI_Crop and ROI_Align.
### Data Preparation
Create a data folder under the repo,
```sh
cd {repo_root}
mkdir data
```
**PASCAL_VOC 07+12**: Please follow the instructions in [py-faster-rcnn](https://github.com/rbgirshick/py-faster-rcnn#beyond-the-demo-installation-for-training-and-testing-models) to prepare VOC datasets. Actually, you can refer to any others. After downloading the data, create softlinks in the folder data/.
please download the three base classes [splits](https://pan.baidu.com/s/11IxGujTTegLEXFsaiohV_Q)[[GoogleDrive](https://drive.google.com/drive/folders/14gtxnxWokk3eO6Oe5SrEG6_R9Dt6efT8?usp=sharing)] and put them into VOC2007 and VOC2012 ImageSets/Main dirs.
### Training
We used [ResNet101](https://www.dropbox.com/s/iev3tkbz5wyyuz9/resnet101_caffe.pth?dl=0) pretrained model on ImageNet in our experiments. Download it and put it into the data/pretrained_model/.
for example, if you want to train the first split of base and novel class with meta learning, just run:
#### the first phase
```sh
$>CUDA_VISIBLE_DEVICES=0 python train_metarcnn.py --dataset pascal_voc_0712 --epochs 21 --bs 4 --nw 8 --log_dir checkpoint --save_dir models/meta/first --meta_type 1 --meta_train True --meta_loss True
```
#### the second phase
```sh
$>CUDA_VISIBLE_DEVICES=0 python train_metarcnn.py --dataset pascal_voc_0712 --epochs 30 --bs 4 --nw 8 --log_dir checkpoint --save_dir models/meta/first --r True --checksession 1 --checkepoch 20 --checkpoint 3081 --phase 2 --shots 10 --meta_train True --meta_loss True --meta_type 1
```
### Testing
if you want to evaluate the performance of meta trained model, simply run:
```sh
$>CUDA_VISIBLE_DEVICES=0 python test_metarcnn.py --dataset pascal_voc_0712 --net metarcnn --load_dir models/meta/first --checksession 10 --checkepoch 30 --checkpoint 111 --shots 10 --meta_type 1 --meta_test True --meta_loss True --phase 2
```
we provide the part models with meta training and without meta training in the following:
[Meta Models](https://pan.baidu.com/s/1N3PW9WTi82lbdURNAz7EFA)[[GoogleDrive](https://drive.google.com/file/d/19gapxklxKCwYIyGszOMhQKNDqYOLeubn/view?usp=sharing)] and [WoMeta Models](https://pan.baidu.com/s/1GkjUJmaOaEWzh3z2fs7ieA)[[GoogleDrive](https://drive.google.com/file/d/1G6xYH9M_bAAqUec1ARufv0ELi_pd7ERj/view?usp=sharing)]
### Citation
```
@inproceedings{yan2019meta,
title={Meta r-cnn: Towards general solver for instance-level low-shot learning},
author={Yan, Xiaopeng and Chen, Ziliang and Xu, Anni and Wang, Xiaoxi and Liang, Xiaodan and Lin, Liang},
booktitle={Proceedings of the IEEE International Conference on Computer Vision},
pages={9577--9586},
year={2019}
}
```
### Contact
If you have any questions about this repo, please feel free to contact [yanxp3@mail3.sysu.edu.cn](mailto:yanxp3@mail3.sysu.edu.cn).
================================================
FILE: _init_paths.py
================================================
import os.path as osp
import sys
import os
if os.listdir('data/cache/'):
os.system('rm data/cache/*')
def add_path(path):
if path not in sys.path:
sys.path.insert(0, path)
this_dir = osp.dirname(__file__)
# Add lib to PYTHONPATH
lib_path = osp.join(this_dir, 'lib')
add_path(lib_path)
coco_path = osp.join(this_dir, 'data', 'coco', 'PythonAPI')
add_path(coco_path)
vg_path = osp.join(this_dir, 'data', 'vgapi')
add_path(vg_path)
================================================
FILE: cfgs/res101_ms.yml
================================================
EXP_DIR: res101
TRAIN:
HAS_RPN: True
BBOX_NORMALIZE_TARGETS_PRECOMPUTED: True
RPN_POSITIVE_OVERLAP: 0.7
RPN_BATCHSIZE: 256
PROPOSAL_METHOD: gt
BG_THRESH_LO: 0.0
DISPLAY: 20
BATCH_SIZE: 128
WEIGHT_DECAY: 0.0001
MAX_SIZE: 1000
SCALES: [600]
DOUBLE_BIAS: False
RCNN_BBOX_WEIGHT: 1
TEST:
SCALES: [600]
HAS_RPN: True
POOLING_SIZE: 7
POOLING_MODE: align
CROP_RESIZE_WITH_MAX_POOL: False
================================================
FILE: cfgs/res50.yml
================================================
EXP_DIR: res50
TRAIN:
HAS_RPN: True
BBOX_NORMALIZE_TARGETS_PRECOMPUTED: True
RPN_POSITIVE_OVERLAP: 0.7
RPN_BATCHSIZE: 256
PROPOSAL_METHOD: gt
BG_THRESH_LO: 0.0
DISPLAY: 20
BATCH_SIZE: 128
WEIGHT_DECAY: 0.0001
MAX_SIZE: 1000
SCALES: [600]
DOUBLE_BIAS: False
RCNN_BBOX_WEIGHT: 1
TEST:
SCALES: [600]
HAS_RPN: True
POOLING_SIZE: 7
POOLING_MODE: align
CROP_RESIZE_WITH_MAX_POOL: False
================================================
FILE: lib/datasets/VOCdevkit-matlab-wrapper/get_voc_opts.m
================================================
function VOCopts = get_voc_opts(path)
tmp = pwd;
cd(path);
try
addpath('VOCcode');
VOCinit;
catch
rmpath('VOCcode');
cd(tmp);
error(sprintf('VOCcode directory not found under %s', path));
end
rmpath('VOCcode');
cd(tmp);
================================================
FILE: lib/datasets/VOCdevkit-matlab-wrapper/voc_eval.m
================================================
function res = voc_eval(path, comp_id, test_set, output_dir)
VOCopts = get_voc_opts(path);
VOCopts.testset = test_set;
for i = 1:length(VOCopts.classes)
cls = VOCopts.classes{i};
res(i) = voc_eval_cls(cls, VOCopts, comp_id, output_dir);
end
fprintf('\n~~~~~~~~~~~~~~~~~~~~\n');
fprintf('Results:\n');
aps = [res(:).ap]';
fprintf('%.1f\n', aps * 100);
fprintf('%.1f\n', mean(aps) * 100);
fprintf('~~~~~~~~~~~~~~~~~~~~\n');
function res = voc_eval_cls(cls, VOCopts, comp_id, output_dir)
test_set = VOCopts.testset;
year = VOCopts.dataset(4:end);
addpath(fullfile(VOCopts.datadir, 'VOCcode'));
res_fn = sprintf(VOCopts.detrespath, comp_id, cls);
recall = [];
prec = [];
ap = 0;
ap_auc = 0;
do_eval = (str2num(year) <= ) | ~strcmp(test_set, 'test');
if do_eval
% Bug in VOCevaldet requires that tic has been called first
tic;
[recall, prec, ap] = VOCevaldet(VOCopts, comp_id, cls, true);
ap_auc = xVOCap(recall, prec);
% force plot limits
ylim([0 1]);
xlim([0 1]);
print(gcf, '-djpeg', '-r0', ...
[output_dir '/' cls '_pr.jpg']);
end
fprintf('!!! %s : %.4f %.4f\n', cls, ap, ap_auc);
res.recall = recall;
res.prec = prec;
res.ap = ap;
res.ap_auc = ap_auc;
save([output_dir '/' cls '_pr.mat'], ...
'res', 'recall', 'prec', 'ap', 'ap_auc');
rmpath(fullfile(VOCopts.datadir, 'VOCcode'));
================================================
FILE: lib/datasets/VOCdevkit-matlab-wrapper/xVOCap.m
================================================
function ap = xVOCap(rec,prec)
% From the PASCAL VOC 2011 devkit
mrec=[0 ; rec ; 1];
mpre=[0 ; prec ; 0];
for i=numel(mpre)-1:-1:1
mpre(i)=max(mpre(i),mpre(i+1));
end
i=find(mrec(2:end)~=mrec(1:end-1))+1;
ap=sum((mrec(i)-mrec(i-1)).*mpre(i));
================================================
FILE: lib/datasets/__init__.py
================================================
# --------------------------------------------------------
# Fast R-CNN
# Copyright (c) 2015 Microsoft
# Licensed under The MIT License [see LICENSE for details]
# Written by Ross Girshick
# --------------------------------------------------------
================================================
FILE: lib/datasets/coco.py
================================================
# --------------------------------------------------------
# Fast/er R-CNN
# Licensed under The MIT License [see LICENSE for details]
# Written by Ross Girshick and Xinlei Chen
# --------------------------------------------------------
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from datasets.imdb import imdb
import datasets.ds_utils as ds_utils
from model.utils.config import cfg
import os.path as osp
import sys
import os
import numpy as np
import scipy.sparse
import scipy.io as sio
import pickle
import json
import uuid
# COCO API
from pycocotools.coco import COCO
from pycocotools.cocoeval import COCOeval
from pycocotools import mask as COCOmask
class coco(imdb):
def __init__(self, image_set, year):
imdb.__init__(self, 'coco_' + year + '_' + image_set)
# COCO specific config options
self.config = {'use_salt': True,
'cleanup': True}
# name, paths
self._year = year
self._image_set = image_set
self._data_path = osp.join(cfg.DATA_DIR, 'coco'+self._year)
# load COCO API, classes, class <-> id mappings
self._COCO = COCO(self._get_ann_file())
cats = self._COCO.loadCats(self._COCO.getCatIds())
self._classes = tuple(['__background__'] + [c['name'] for c in cats])
self._class_to_ind = dict(list(zip(self.classes, list(range(self.num_classes)))))
self._class_to_coco_cat_id = dict(list(zip([c['name'] for c in cats],
self._COCO.getCatIds())))
self._image_index = self._load_image_set_index()
# Default to roidb handler
self.set_proposal_method('gt')
self.competition_mode(False)
# Some image sets are "views" (i.e. subsets) into others.
# For example, minival2014 is a random 5000 image subset of val2014.
# This mapping tells us where the view's images and proposals come from.
self._view_map = {
'minival2014': 'val2014', # 5k val2014 subset
'valminusminival2014': 'val2014', # val2014 \setminus minival2014
'test-dev2015': 'test2015',
'valminuscapval2014': 'val2014',
'capval2014': 'val2014',
'captest2014': 'val2014'
}
coco_name = image_set + year # e.g., "val2014"
self._data_name = (self._view_map[coco_name]
if coco_name in self._view_map
else coco_name)
# Dataset splits that have ground-truth annotations (test splits
# do not have gt annotations)
self._gt_splits = ('train', 'val', 'minival')
def _get_ann_file(self):
prefix = 'instances' if self._image_set.find('test') == -1 \
else 'image_info'
return osp.join(self._data_path, 'annotations',
prefix + '_' + self._image_set + self._year + '.json')
def _load_image_set_index(self):
"""
Load image ids.
"""
image_ids = self._COCO.getImgIds()
return image_ids
def _get_widths(self):
anns = self._COCO.loadImgs(self._image_index)
widths = [ann['width'] for ann in anns]
return widths
def image_path_at(self, i):
"""
Return the absolute path to image i in the image sequence.
"""
return self.image_path_from_index(self._image_index[i])
def image_id_at(self, i):
"""
Return the absolute path to image i in the image sequence.
"""
return self._image_index[i]
def image_path_from_index(self, index):
"""
Construct an image path from the image's "index" identifier.
"""
# Example image path for index=119993:
# images/train2014/COCO_train2014_000000119993.jpg
if self._year == '2017':
file_name = str(index).zfill(12) + '.jpg'
elif self._year == '2014':
file_name = ('COCO_' + self._data_name + '_' +
str(index).zfill(12) + '.jpg')
image_path = osp.join(self._data_path, 'images',
self._data_name, file_name)
assert osp.exists(image_path), \
'Path does not exist: {}'.format(image_path)
return image_path
def gt_roidb(self):
"""
Return the database of ground-truth regions of interest.
This function loads/saves from/to a cache file to speed up future calls.
"""
cache_file = osp.join(self.cache_path, self.name + '_gt_roidb.pkl')
if osp.exists(cache_file):
with open(cache_file, 'rb') as fid:
roidb = pickle.load(fid)
print('{} gt roidb loaded from {}'.format(self.name, cache_file))
return roidb
gt_roidb = [self._load_coco_annotation(index)
for index in self._image_index]
with open(cache_file, 'wb') as fid:
pickle.dump(gt_roidb, fid, pickle.HIGHEST_PROTOCOL)
print('wrote gt roidb to {}'.format(cache_file))
return gt_roidb
def _load_coco_annotation(self, index):
"""
Loads COCO bounding-box instance annotations. Crowd instances are
handled by marking their overlaps (with all categories) to -1. This
overlap value means that crowd "instances" are excluded from training.
"""
im_ann = self._COCO.loadImgs(index)[0]
width = im_ann['width']
height = im_ann['height']
annIds = self._COCO.getAnnIds(imgIds=index, iscrowd=None)
objs = self._COCO.loadAnns(annIds)
# Sanitize bboxes -- some are invalid
valid_objs = []
for obj in objs:
x1 = np.max((0, obj['bbox'][0]))
y1 = np.max((0, obj['bbox'][1]))
x2 = np.min((width - 1, x1 + np.max((0, obj['bbox'][2] - 1))))
y2 = np.min((height - 1, y1 + np.max((0, obj['bbox'][3] - 1))))
if obj['area'] > 0 and x2 >= x1 and y2 >= y1:
obj['clean_bbox'] = [x1, y1, x2, y2]
valid_objs.append(obj)
objs = valid_objs
num_objs = len(objs)
boxes = np.zeros((num_objs, 4), dtype=np.uint16)
gt_classes = np.zeros((num_objs), dtype=np.int32)
overlaps = np.zeros((num_objs, self.num_classes), dtype=np.float32)
seg_areas = np.zeros((num_objs), dtype=np.float32)
# Lookup table to map from COCO category ids to our internal class
# indices
coco_cat_id_to_class_ind = dict([(self._class_to_coco_cat_id[cls],
self._class_to_ind[cls])
for cls in self._classes[1:]])
for ix, obj in enumerate(objs):
cls = coco_cat_id_to_class_ind[obj['category_id']]
boxes[ix, :] = obj['clean_bbox']
gt_classes[ix] = cls
seg_areas[ix] = obj['area']
if obj['iscrowd']:
# Set overlap to -1 for all classes for crowd objects
# so they will be excluded during training
overlaps[ix, :] = -1.0
else:
overlaps[ix, cls] = 1.0
ds_utils.validate_boxes(boxes, width=width, height=height)
overlaps = scipy.sparse.csr_matrix(overlaps)
return {'width': width,
'height': height,
'boxes': boxes,
'gt_classes': gt_classes,
'gt_overlaps': overlaps,
'flipped': False,
'seg_areas': seg_areas}
def _get_widths(self):
return [r['width'] for r in self.roidb]
def append_flipped_images(self):
num_images = self.num_images
widths = self._get_widths()
for i in range(num_images):
boxes = self.roidb[i]['boxes'].copy()
oldx1 = boxes[:, 0].copy()
oldx2 = boxes[:, 2].copy()
boxes[:, 0] = widths[i] - oldx2 - 1
boxes[:, 2] = widths[i] - oldx1 - 1
assert (boxes[:, 2] >= boxes[:, 0]).all()
entry = {'width': widths[i],
'height': self.roidb[i]['height'],
'boxes': boxes,
'gt_classes': self.roidb[i]['gt_classes'],
'gt_overlaps': self.roidb[i]['gt_overlaps'],
'flipped': True,
'seg_areas': self.roidb[i]['seg_areas']}
self.roidb.append(entry)
self._image_index = self._image_index * 2
def _get_box_file(self, index):
# first 14 chars / first 22 chars / all chars + .mat
# COCO_val2014_0/COCO_val2014_000000447/COCO_val2014_000000447991.mat
file_name = ('COCO_' + self._data_name +
'_' + str(index).zfill(12) + '.mat')
return osp.join(file_name[:14], file_name[:22], file_name)
def _print_detection_eval_metrics(self, coco_eval):
IoU_lo_thresh = 0.5
IoU_hi_thresh = 0.95
def _get_thr_ind(coco_eval, thr):
ind = np.where((coco_eval.params.iouThrs > thr - 1e-5) &
(coco_eval.params.iouThrs < thr + 1e-5))[0][0]
iou_thr = coco_eval.params.iouThrs[ind]
assert np.isclose(iou_thr, thr)
return ind
ind_lo = _get_thr_ind(coco_eval, IoU_lo_thresh)
ind_hi = _get_thr_ind(coco_eval, IoU_hi_thresh)
# precision has dims (iou, recall, cls, area range, max dets)
# area range index 0: all area ranges
# max dets index 2: 100 per image
precision = \
coco_eval.eval['precision'][ind_lo:(ind_hi + 1), :, :, 0, 2]
ap_default = np.mean(precision[precision > -1])
print(('~~~~ Mean and per-category AP @ IoU=[{:.2f},{:.2f}] '
'~~~~').format(IoU_lo_thresh, IoU_hi_thresh))
print('{:.1f}'.format(100 * ap_default))
for cls_ind, cls in enumerate(self.classes):
if cls == '__background__':
continue
# minus 1 because of __background__
precision = coco_eval.eval['precision'][ind_lo:(ind_hi + 1), :, cls_ind - 1, 0, 2]
ap = np.mean(precision[precision > -1])
print('{}: {:.1f}'.format(cls, 100 * ap))
print('~~~~ Summary metrics ~~~~')
coco_eval.summarize()
def _do_detection_eval(self, res_file, output_dir):
ann_type = 'bbox'
coco_dt = self._COCO.loadRes(res_file)
coco_eval = COCOeval(self._COCO, coco_dt)
coco_eval.params.useSegm = (ann_type == 'segm')
coco_eval.evaluate()
coco_eval.accumulate()
self._print_detection_eval_metrics(coco_eval)
eval_file = osp.join(output_dir, 'detection_results.pkl')
with open(eval_file, 'wb') as fid:
pickle.dump(coco_eval, fid, pickle.HIGHEST_PROTOCOL)
print('Wrote COCO eval results to: {}'.format(eval_file))
def _coco_results_one_category(self, boxes, cat_id):
results = []
for im_ind, index in enumerate(self.image_index):
dets = boxes[im_ind].astype(np.float)
if dets == []:
continue
scores = dets[:, -1]
xs = dets[:, 0]
ys = dets[:, 1]
ws = dets[:, 2] - xs + 1
hs = dets[:, 3] - ys + 1
results.extend(
[{'image_id': index,
'category_id': cat_id,
'bbox': [xs[k], ys[k], ws[k], hs[k]],
'score': scores[k]} for k in range(dets.shape[0])])
return results
def _write_coco_results_file(self, all_boxes, res_file):
# [{"image_id": 42,
# "category_id": 18,
# "bbox": [258.15,41.29,348.26,243.78],
# "score": 0.236}, ...]
results = []
for cls_ind, cls in enumerate(self.classes):
if cls == '__background__':
continue
print('Collecting {} results ({:d}/{:d})'.format(cls, cls_ind,
self.num_classes - 1))
coco_cat_id = self._class_to_coco_cat_id[cls]
results.extend(self._coco_results_one_category(all_boxes[cls_ind],
coco_cat_id))
print('Writing results json to {}'.format(res_file))
with open(res_file, 'w') as fid:
json.dump(results, fid)
def evaluate_detections(self, all_boxes, output_dir):
res_file = osp.join(output_dir, ('detections_' +
self._image_set +
self._year +
'_results'))
if self.config['use_salt']:
res_file += '_{}'.format(str(uuid.uuid4()))
res_file += '.json'
self._write_coco_results_file(all_boxes, res_file)
# Only do evaluation on non-test sets
if self._image_set.find('test') == -1:
self._do_detection_eval(res_file, output_dir)
# Optionally cleanup results json file
if self.config['cleanup']:
os.remove(res_file)
def competition_mode(self, on):
if on:
self.config['use_salt'] = False
self.config['cleanup'] = False
else:
self.config['use_salt'] = True
self.config['cleanup'] = True
================================================
FILE: lib/datasets/ds_utils.py
================================================
# --------------------------------------------------------
# Fast/er R-CNN
# Licensed under The MIT License [see LICENSE for details]
# Written by Ross Girshick
# --------------------------------------------------------
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import numpy as np
def unique_boxes(boxes, scale=1.0):
"""Return indices of unique boxes."""
v = np.array([1, 1e3, 1e6, 1e9])
hashes = np.round(boxes * scale).dot(v)
_, index = np.unique(hashes, return_index=True)
return np.sort(index)
def xywh_to_xyxy(boxes):
"""Convert [x y w h] box format to [x1 y1 x2 y2] format."""
return np.hstack((boxes[:, 0:2], boxes[:, 0:2] + boxes[:, 2:4] - 1))
def xyxy_to_xywh(boxes):
"""Convert [x1 y1 x2 y2] box format to [x y w h] format."""
return np.hstack((boxes[:, 0:2], boxes[:, 2:4] - boxes[:, 0:2] + 1))
def validate_boxes(boxes, width=0, height=0):
"""Check that a set of boxes are valid."""
x1 = boxes[:, 0]
y1 = boxes[:, 1]
x2 = boxes[:, 2]
y2 = boxes[:, 3]
assert (x1 >= 0).all()
assert (y1 >= 0).all()
assert (x2 >= x1).all()
assert (y2 >= y1).all()
assert (x2 < width).all()
assert (y2 < height).all()
def filter_small_boxes(boxes, min_size):
w = boxes[:, 2] - boxes[:, 0]
h = boxes[:, 3] - boxes[:, 1]
keep = np.where((w >= min_size) & (h > min_size))[0]
return keep
================================================
FILE: lib/datasets/factory.py
================================================
# --------------------------------------------------------
# Fast R-CNN
# Copyright (c) 2015 Microsoft
# Licensed under The MIT License [see LICENSE for details]
# Written by Ross Girshick
# --------------------------------------------------------
"""Factory method for easily getting imdbs by name."""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
__sets = {}
from datasets.coco import coco
from datasets.pascal_voc import pascal_voc
# # Set up voc_<year>_<split>
for year in ['2007', '2012']:
for split in ['train', 'val', 'trainval', 'test','shots', 'train_first_split', 'train_second_split', 'train_third_split']:
name = 'voc_{}_{}'.format(year, split)
__sets[name] = (lambda split=split, year=year: pascal_voc(split, year))
for year in ['2014']:
for split in ['train', 'val', 'minival', 'valminusminival', 'trainval']:
name = 'coco_{}_{}'.format(year, split)
__sets[name] = (lambda split=split, year=year: coco(split, year))
for year in ['2017']:
for split in ['train', 'val']:
name = 'coco_{}_{}'.format(year, split)
__sets[name] = (lambda split=split, year=year: coco(split, year))
def get_imdb(name):
"""Get an imdb (image database) by name."""
if name not in __sets:
raise KeyError('Unknown dataset: {}'.format(name))
return __sets[name]()
def list_imdbs():
"""List all registered imdbs."""
return list(__sets.keys())
================================================
FILE: lib/datasets/imdb.py
================================================
# --------------------------------------------------------
# Fast R-CNN
# Copyright (c) 2015 Microsoft
# Licensed under The MIT License [see LICENSE for details]
# Written by Ross Girshick and Xinlei Chen
# --------------------------------------------------------
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import os
import os.path as osp
import PIL
from model.utils.cython_bbox import bbox_overlaps
import numpy as np
import scipy.sparse
from model.utils.config import cfg
import pdb
ROOT_DIR = osp.join(osp.dirname(__file__), '..', '..')
class imdb(object):
"""Image database."""
def __init__(self, name, classes=None):
self._name = name
self._num_classes = 0
if not classes:
self._classes = []
else:
self._classes = classes
self._image_index = []
self._obj_proposer = 'gt'
self._roidb = None
self._roidb_handler = self.default_roidb
# Use this dict for storing dataset specific config options
self.config = {}
@property
def name(self):
return self._name
@property
def num_classes(self):
return len(self._classes)
def set_classes(self,classes):
self._classes = classes
@property
def classes(self):
return self._classes
@property
def image_index(self):
return self._image_index
@property
def roidb_handler(self):
return self._roidb_handler
@roidb_handler.setter
def roidb_handler(self, val):
self._roidb_handler = val
def set_proposal_method(self, method):
method = eval('self.' + method + '_roidb')
self.roidb_handler = method
@property
def roidb(self):
# A roidb is a list of dictionaries, each with the following keys:
# boxes
# gt_overlaps
# gt_classes
# flipped
if self._roidb is not None:
return self._roidb
self._roidb = self.roidb_handler()
return self._roidb
def set_roidb(self,roidb):
self._roidb = roidb
@property
def cache_path(self):
cache_path = osp.abspath(osp.join(cfg.DATA_DIR, 'cache'))
if not os.path.exists(cache_path):
os.makedirs(cache_path)
return cache_path
@property
def num_images(self):
return len(self.image_index)
def image_path_at(self, i):
raise NotImplementedError
def image_id_at(self, i):
raise NotImplementedError
def default_roidb(self):
raise NotImplementedError
def evaluate_detections(self, all_boxes, output_dir=None):
"""
all_boxes is a list of length number-of-classes.
Each list element is a list of length number-of-images.
Each of those list elements is either an empty list []
or a numpy array of detection.
all_boxes[class][image] = [] or np.array of shape #dets x 5
"""
raise NotImplementedError
def _get_widths(self):
return [PIL.Image.open(self.image_path_at(i)).size[0]
for i in range(self.num_images)]
def append_flipped_images(self):
num_images = self.num_images
widths = self._get_widths()
for i in range(num_images):
boxes = self.roidb[i]['boxes'].copy()
oldx1 = boxes[:, 0].copy()
oldx2 = boxes[:, 2].copy()
boxes[:, 0] = widths[i] - oldx2 - 1
boxes[:, 2] = widths[i] - oldx1 - 1
assert (boxes[:, 2] >= boxes[:, 0]).all()
entry = {'boxes': boxes,
'gt_overlaps': self.roidb[i]['gt_overlaps'],
'gt_classes': self.roidb[i]['gt_classes'],
'flipped': True}
self.roidb.append(entry)
self._image_index = self._image_index * 2
def evaluate_recall(self, candidate_boxes=None, thresholds=None,
area='all', limit=None):
"""Evaluate detection proposal recall metrics.
Returns:
results: dictionary of results with keys
'ar': average recall
'recalls': vector recalls at each IoU overlap threshold
'thresholds': vector of IoU overlap thresholds
'gt_overlaps': vector of all ground-truth overlaps
"""
# Record max overlap value for each gt box
# Return vector of overlap values
areas = {'all': 0, 'small': 1, 'medium': 2, 'large': 3,
'96-128': 4, '128-256': 5, '256-512': 6, '512-inf': 7}
area_ranges = [[0 ** 2, 1e5 ** 2], # all
[0 ** 2, 32 ** 2], # small
[32 ** 2, 96 ** 2], # medium
[96 ** 2, 1e5 ** 2], # large
[96 ** 2, 128 ** 2], # 96-128
[128 ** 2, 256 ** 2], # 128-256
[256 ** 2, 512 ** 2], # 256-512
[512 ** 2, 1e5 ** 2], # 512-inf
]
assert area in areas, 'unknown area range: {}'.format(area)
area_range = area_ranges[areas[area]]
gt_overlaps = np.zeros(0)
num_pos = 0
for i in range(self.num_images):
# Checking for max_overlaps == 1 avoids including crowd annotations
# (...pretty hacking :/)
max_gt_overlaps = self.roidb[i]['gt_overlaps'].toarray().max(axis=1)
gt_inds = np.where((self.roidb[i]['gt_classes'] > 0) &
(max_gt_overlaps == 1))[0]
gt_boxes = self.roidb[i]['boxes'][gt_inds, :]
gt_areas = self.roidb[i]['seg_areas'][gt_inds]
valid_gt_inds = np.where((gt_areas >= area_range[0]) &
(gt_areas <= area_range[1]))[0]
gt_boxes = gt_boxes[valid_gt_inds, :]
num_pos += len(valid_gt_inds)
if candidate_boxes is None:
# If candidate_boxes is not supplied, the default is to use the
# non-ground-truth boxes from this roidb
non_gt_inds = np.where(self.roidb[i]['gt_classes'] == 0)[0]
boxes = self.roidb[i]['boxes'][non_gt_inds, :]
else:
boxes = candidate_boxes[i]
if boxes.shape[0] == 0:
continue
if limit is not None and boxes.shape[0] > limit:
boxes = boxes[:limit, :]
overlaps = bbox_overlaps(boxes.astype(np.float),
gt_boxes.astype(np.float))
_gt_overlaps = np.zeros((gt_boxes.shape[0]))
for j in range(gt_boxes.shape[0]):
# find which proposal box maximally covers each gt box
argmax_overlaps = overlaps.argmax(axis=0)
# and get the iou amount of coverage for each gt box
max_overlaps = overlaps.max(axis=0)
# find which gt box is 'best' covered (i.e. 'best' = most iou)
gt_ind = max_overlaps.argmax()
gt_ovr = max_overlaps.max()
assert (gt_ovr >= 0)
# find the proposal box that covers the best covered gt box
box_ind = argmax_overlaps[gt_ind]
# record the iou coverage of this gt box
_gt_overlaps[j] = overlaps[box_ind, gt_ind]
assert (_gt_overlaps[j] == gt_ovr)
# mark the proposal box and the gt box as used
overlaps[box_ind, :] = -1
overlaps[:, gt_ind] = -1
# append recorded iou coverage level
gt_overlaps = np.hstack((gt_overlaps, _gt_overlaps))
gt_overlaps = np.sort(gt_overlaps)
if thresholds is None:
step = 0.05
thresholds = np.arange(0.5, 0.95 + 1e-5, step)
recalls = np.zeros_like(thresholds)
# compute recall for each iou threshold
for i, t in enumerate(thresholds):
recalls[i] = (gt_overlaps >= t).sum() / float(num_pos)
# ar = 2 * np.trapz(recalls, thresholds)
ar = recalls.mean()
return {'ar': ar, 'recalls': recalls, 'thresholds': thresholds,
'gt_overlaps': gt_overlaps}
def create_roidb_from_box_list(self, box_list, gt_roidb):
assert len(box_list) == self.num_images, \
'Number of boxes must match number of ground-truth images'
roidb = []
for i in range(self.num_images):
boxes = box_list[i]
num_boxes = boxes.shape[0]
overlaps = np.zeros((num_boxes, self.num_classes), dtype=np.float32)
if gt_roidb is not None and gt_roidb[i]['boxes'].size > 0:
gt_boxes = gt_roidb[i]['boxes']
gt_classes = gt_roidb[i]['gt_classes']
gt_overlaps = bbox_overlaps(boxes.astype(np.float),
gt_boxes.astype(np.float))
argmaxes = gt_overlaps.argmax(axis=1)
maxes = gt_overlaps.max(axis=1)
I = np.where(maxes > 0)[0]
overlaps[I, gt_classes[argmaxes[I]]] = maxes[I]
overlaps = scipy.sparse.csr_matrix(overlaps)
roidb.append({
'boxes': boxes,
'gt_classes': np.zeros((num_boxes,), dtype=np.int32),
'gt_overlaps': overlaps,
'flipped': False,
'seg_areas': np.zeros((num_boxes,), dtype=np.float32),
})
return roidb
@staticmethod
def merge_roidbs(a, b):
assert len(a) == len(b)
for i in range(len(a)):
a[i]['boxes'] = np.vstack((a[i]['boxes'], b[i]['boxes']))
a[i]['gt_classes'] = np.hstack((a[i]['gt_classes'],
b[i]['gt_classes']))
a[i]['gt_overlaps'] = scipy.sparse.vstack([a[i]['gt_overlaps'],
b[i]['gt_overlaps']])
a[i]['seg_areas'] = np.hstack((a[i]['seg_areas'],
b[i]['seg_areas']))
return a
def competition_mode(self, on):
"""Turn competition mode on or off."""
pass
================================================
FILE: lib/datasets/metadata.py
================================================
# --------------------------------------------------------
# Pytorch Meta R-CNN
# Written by Anny Xu, Xiaopeng Yan, based on the code from Jianwei Yang
# --------------------------------------------------------
import os
import os.path
import sys
import torch.utils.data as data
import cv2
import torch
import random
import numpy as np
if sys.version_info[0] == 2:
import xml.etree.cElementTree as ET
else:
import xml.etree.ElementTree as ET
from model.utils.config import cfg
import collections
class MetaDataset(data.Dataset):
"""Meta Dataset
Arguments:
root (string): filepath to VOCdevkit folder.
image_set (string): imageset to use (eg. 'train', 'val')
metaclass(string): the class name
img_size(int) : the PRN network input size
shot(int): the number of instances
shuffle(bool)
"""
def __init__(self, root, image_sets, metaclass, img_size, shots=1, shuffle=False, phase=1):
self.root = root
self.image_set = image_sets
self.img_size = img_size
self.metaclass = metaclass
self.shots = shots
if phase == 2:
self.shots = shots * 3
self.shuffle=shuffle
self._annopath = os.path.join('%s', 'Annotations', '%s.xml')
self._imgpath = os.path.join('%s', 'JPEGImages', '%s.jpg')
self.shot_path = open(os.path.join(self.root, 'VOC2007', 'ImageSets/Main/shots.txt'), 'w') # the default saved path
self.ids = list()
for (year, name) in image_sets:
self._year = year
rootpath = os.path.join(self.root, 'VOC' + year)
for line in open(os.path.join(rootpath, 'ImageSets', 'Main', name + '.txt')):
self.ids.append((rootpath, line.strip()))
class_to_idx = dict(zip(self.metaclass, range(len(self.metaclass)))) # class to index mapping
self.prndata = []
self.prncls = []
prn_image, prn_mask = self.get_prndata()
for i in range(shots):
cls = []
data = []
for n, key in enumerate(list(prn_image.keys())):
img = torch.from_numpy(np.array(prn_image[key][i]))
img = img.unsqueeze(0)
mask = torch.from_numpy(np.array(prn_mask[key][i]))
mask = mask.unsqueeze(0)
mask = mask.unsqueeze(3)
imgmask = torch.cat([img, mask], dim=3)
data.append(imgmask.permute(0, 3, 1, 2).contiguous())
cls.append(class_to_idx[key])
self.prncls.append(cls)
self.prndata.append(torch.cat(data,dim=0))
def __getitem__(self, index):
return self.prndata[index],self.prncls[index]
def get_prndata(self):
'''
:return: the construct prn input data
'''
if self.shuffle:
random.shuffle(self.ids)
prn_image = collections.defaultdict(list)
prn_mask = collections.defaultdict(list)
classes = collections.defaultdict(int)
for cls in self.metaclass:
classes[cls] = 0
for img_id in self.ids:
target = ET.parse(self._annopath % img_id).getroot()
img = cv2.imread(self._imgpath % img_id, cv2.IMREAD_COLOR)
img = img[:, :, ::-1]
img = img.astype(np.float32, copy=False)
img -= cfg.PIXEL_MEANS
height, width, _ = img.shape
mask = np.zeros((self.img_size, self.img_size), dtype=np.float32)
h, w, _ = img.shape
y_ration = float(h) / self.img_size
x_ration = float(w) / self.img_size
img_resize = cv2.resize(img, (self.img_size, self.img_size), interpolation=cv2.INTER_LINEAR)
for obj in target.iter('object'):
difficult = int(obj.find('difficult').text) == 1
if difficult:
continue
name = obj.find('name').text.strip()
if name not in self.metaclass:
continue
if classes[name] >= self.shots:
break
classes[name] += 1
bbox = obj.find('bndbox')
pts = ['xmin', 'ymin', 'xmax', 'ymax']
bndbox = []
for i, pt in enumerate(pts):
cur_pt = int(float(bbox.find(pt).text)) - 1
if i % 2 == 0:
cur_pt = int(cur_pt / x_ration)
bndbox.append(cur_pt)
elif i % 2 == 1:
cur_pt = int(cur_pt / y_ration)
bndbox.append(cur_pt)
mask[bndbox[1]:bndbox[3], bndbox[0]:bndbox[2]] = 1
prn_image[name].append(img_resize)
prn_mask[name].append(mask)
self.shot_path.write(str(img_id[1])+'\n')
break
if len(classes)>0 and min(classes.values()) == self.shots:
break
self.shot_path.close()
return prn_image, prn_mask
def __len__(self):
return len(self.prndata)
================================================
FILE: lib/datasets/pascal_voc.py
================================================
from __future__ import print_function
from __future__ import absolute_import
# --------------------------------------------------------
# Fast R-CNN
# Copyright (c) 2015 Microsoft
# Licensed under The MIT License [see LICENSE for details]
# Written by Ross Girshick
# --------------------------------------------------------
import xml.dom.minidom as minidom
import os
# import PIL
import numpy as np
import scipy.sparse
import subprocess
import math
import glob
import uuid
import scipy.io as sio
import xml.etree.ElementTree as ET
import pickle
from .imdb import imdb
from .imdb import ROOT_DIR
from . import ds_utils
from .voc_eval import voc_eval
import random
# TODO: make fast_rcnn irrelevant
# >>>> obsolete, because it depends on sth outside of this project
from model.utils.config import cfg
try:
xrange # Python 2
except NameError:
xrange = range # Python 3
# <<<< obsolete
class pascal_voc(imdb):
def __init__(self, image_set, year, devkit_path=None):
imdb.__init__(self, 'voc_' + year + '_' + image_set)
self._year = year
self._image_set = image_set
self._devkit_path = self._get_default_path() if devkit_path is None \
else devkit_path
self._data_path = os.path.join(self._devkit_path, 'VOC' + self._year)
#first split
if cfg.TRAIN.META_TYPE == 1:
self._classes = ['__background__'] + cfg.TRAIN.ALLCLASSES_FIRST
#second split
if cfg.TRAIN.META_TYPE == 2:
self._classes = ['__background__'] + cfg.TRAIN.ALLCLASSES_SECOND
#third split
if cfg.TRAIN.META_TYPE == 3:
self._classes = ['__background__'] + cfg.TRAIN.ALLCLASSES_THIRD
self._class_to_ind = dict(zip(self.classes, xrange(self.num_classes)))
self._image_ext = '.jpg'
self._image_index = self._load_image_set_index()
# Default to roidb handler
# self._roidb_handler = self.selective_search_roidb
self._roidb_handler = self.gt_roidb
self._salt = str(uuid.uuid4())
self._comp_id = 'comp4'
# PASCAL specific config options
self.config = {'cleanup': True,
'use_salt': True,
'use_diff': False,
'matlab_eval': False,
'rpn_file': None,
'min_size': 2}
assert os.path.exists(self._devkit_path), \
'VOCdevkit path does not exist: {}'.format(self._devkit_path)
assert os.path.exists(self._data_path), \
'Path does not exist: {}'.format(self._data_path)
def image_path_at(self, i):
"""
Return the absolute path to image i in the image sequence.
"""
return self.image_path_from_index(self._image_index[i])
def image_id_at(self, i):
"""
Return the absolute path to image i in the image sequence.
"""
return i
def image_path_from_index(self, index):
"""
Construct an image path from the image's "index" identifier.
"""
# file_name = str(index).zfill(6)
image_path = os.path.join(self._data_path, 'JPEGImages',
index + self._image_ext)
assert os.path.exists(image_path), \
'Path does not exist: {}'.format(image_path)
return image_path
def _load_image_set_index(self):
"""
Load the indexes listed in this dataset's image set file.
"""
# Example path to image set file:
# self._devkit_path + /VOCdevkit2007/VOC2007/ImageSets/Main/val.txt
image_set_file = os.path.join(self._data_path, 'ImageSets', 'Main',
self._image_set + '.txt')
assert os.path.exists(image_set_file), \
'Path does not exist: {}'.format(image_set_file)
with open(image_set_file) as f:
image_index = [x.strip() for x in f.readlines()]
return image_index
def _get_default_path(self):
"""
Return the default path where PASCAL VOC is expected to be installed.
"""
return os.path.join(cfg.DATA_DIR, 'VOCdevkit' + self._year)
def gt_roidb(self):
"""
Return the database of ground-truth regions of interest.
This function loads/saves from/to a cache file to speed up future calls.
"""
cache_file = os.path.join(self.cache_path, self.name + '_gt_roidb.pkl')
if os.path.exists(cache_file):
with open(cache_file, 'rb') as fid:
roidb = pickle.load(fid)
print('{} gt roidb loaded from {}'.format(self.name, cache_file))
return roidb
gt_roidb = [self._load_pascal_annotation(index)
for index in self.image_index]
with open(cache_file, 'wb') as fid:
pickle.dump(gt_roidb, fid, pickle.HIGHEST_PROTOCOL)
print('wrote gt roidb to {}'.format(cache_file))
return gt_roidb
def selective_search_roidb(self):
"""
Return the database of selective search regions of interest.
Ground-truth ROIs are also included.
This function loads/saves from/to a cache file to speed up future calls.
"""
cache_file = os.path.join(self.cache_path,
self.name + '_selective_search_roidb.pkl')
if os.path.exists(cache_file):
with open(cache_file, 'rb') as fid:
roidb = pickle.load(fid)
print('{} ss roidb loaded from {}'.format(self.name, cache_file))
return roidb
if int(self._year) == 2007 or self._image_set != 'test':
gt_roidb = self.gt_roidb()
ss_roidb = self._load_selective_search_roidb(gt_roidb)
roidb = imdb.merge_roidbs(gt_roidb, ss_roidb)
else:
roidb = self._load_selective_search_roidb(None)
with open(cache_file, 'wb') as fid:
pickle.dump(roidb, fid, pickle.HIGHEST_PROTOCOL)
print('wrote ss roidb to {}'.format(cache_file))
return roidb
def rpn_roidb(self):
if int(self._year) == 2007 or self._image_set != 'test':
gt_roidb = self.gt_roidb()
rpn_roidb = self._load_rpn_roidb(gt_roidb)
roidb = imdb.merge_roidbs(gt_roidb, rpn_roidb)
else:
roidb = self._load_rpn_roidb(None)
return roidb
def _load_rpn_roidb(self, gt_roidb):
filename = self.config['rpn_file']
print('loading {}'.format(filename))
assert os.path.exists(filename), \
'rpn data not found at: {}'.format(filename)
with open(filename, 'rb') as f:
box_list = pickle.load(f)
return self.create_roidb_from_box_list(box_list, gt_roidb)
def _load_selective_search_roidb(self, gt_roidb):
filename = os.path.abspath(os.path.join(cfg.DATA_DIR,
'selective_search_data',
self.name + '.mat'))
assert os.path.exists(filename), \
'Selective search data not found at: {}'.format(filename)
raw_data = sio.loadmat(filename)['boxes'].ravel()
box_list = []
for i in xrange(raw_data.shape[0]):
boxes = raw_data[i][:, (1, 0, 3, 2)] - 1
keep = ds_utils.unique_boxes(boxes)
boxes = boxes[keep, :]
keep = ds_utils.filter_small_boxes(boxes, self.config['min_size'])
boxes = boxes[keep, :]
box_list.append(boxes)
return self.create_roidb_from_box_list(box_list, gt_roidb)
def _load_pascal_annotation(self, index):
"""
Load image and bounding boxes info from XML file in the PASCAL VOC
format.
"""
filename = os.path.join(self._data_path, 'Annotations', index + '.xml')
tree = ET.parse(filename)
objs = tree.findall('object')
# if not self.config['use_diff']:
# # Exclude the samples labeled as difficult
# non_diff_objs = [
# obj for obj in objs if int(obj.find('difficult').text) == 0]
# # if len(non_diff_objs) != len(objs):
# # print 'Removed {} difficult objects'.format(
# # len(objs) - len(non_diff_objs))
# objs = non_diff_objs
num_objs = len(objs)
boxes = np.zeros((num_objs, 4), dtype=np.uint16)
gt_classes = np.zeros((num_objs), dtype=np.int32)
overlaps = np.zeros((num_objs, self.num_classes), dtype=np.float32)
# "Seg" area for pascal is just the box area
seg_areas = np.zeros((num_objs), dtype=np.float32)
ishards = np.zeros((num_objs), dtype=np.int32)
# Load object bounding boxes into a data frame.
for ix, obj in enumerate(objs):
bbox = obj.find('bndbox')
# Make pixel indexes 0-based
x1 = float(bbox.find('xmin').text) - 1
y1 = float(bbox.find('ymin').text) - 1
x2 = float(bbox.find('xmax').text) - 1
y2 = float(bbox.find('ymax').text) - 1
diffc = obj.find('difficult')
difficult = 0 if diffc == None else int(diffc.text)
ishards[ix] = difficult
if obj.find('name').text.lower().strip() not in self._classes:
continue
cls = self._class_to_ind[obj.find('name').text.lower().strip()]
boxes[ix, :] = [x1, y1, x2, y2]
gt_classes[ix] = cls
overlaps[ix, cls] = 1.0
seg_areas[ix] = (x2 - x1 + 1) * (y2 - y1 + 1)
overlaps = scipy.sparse.csr_matrix(overlaps)
return {'boxes': boxes,
'gt_classes': gt_classes,
'gt_ishard': ishards,
'gt_overlaps': overlaps,
'flipped': False,
'seg_areas': seg_areas}
def _get_comp_id(self):
comp_id = (self._comp_id + '_' + self._salt if self.config['use_salt']
else self._comp_id)
return comp_id
def _get_voc_results_file_template(self):
# VOCdevkit/results/VOC2007/Main/<comp_id>_det_test_aeroplane.txt
filename = self._get_comp_id() + '_det_' + self._image_set + '_{:s}.txt'
filedir = os.path.join(self._devkit_path, 'results', 'VOC' + self._year, 'Main')
if not os.path.exists(filedir):
os.makedirs(filedir)
path = os.path.join(filedir, filename)
return path
def _write_voc_results_file(self, all_boxes):
for cls_ind, cls in enumerate(self.classes):
if cls == '__background__':
continue
print('Writing {} VOC results file'.format(cls))
filename = self._get_voc_results_file_template().format(cls)
with open(filename, 'wt') as f:
for im_ind, index in enumerate(self.image_index):
dets = all_boxes[cls_ind][im_ind]
if dets == []:
continue
# the VOCdevkit expects 1-based indices
for k in xrange(dets.shape[0]):
f.write('{:s} {:.3f} {:.1f} {:.1f} {:.1f} {:.1f}\n'.
format(index, dets[k, -1],
dets[k, 0] + 1, dets[k, 1] + 1,
dets[k, 2] + 1, dets[k, 3] + 1))
def _do_python_eval(self, output_dir='output', **kwargs):
annopath = os.path.join(
self._devkit_path,
'VOC' + self._year,
'Annotations',
'{:s}.xml')
imagesetfile = os.path.join(
self._devkit_path,
'VOC' + self._year,
'ImageSets',
'Main',
self._image_set + '.txt')
cachedir = os.path.join(self._devkit_path, 'annotations_cache')
aps = []
import time,csv
now = time.strftime("%Y-%m-%d-%H-%M-%S")
############################### changed by xan 2019/1/31 begin################################
save_dir = 'results'
if not os.path.exists(save_dir):
os.mkdir(save_dir)
path = str(now) + '_'
for k in kwargs:
if k in ('checksession','checkepoch','checkpoint','meta_test','shots'):
path = path + k + '_' + str(kwargs[k])
csvfile = open(os.path.join(save_dir, path + '.csv'), 'w')
############################## end ###########################################################
writer = csv.writer(csvfile)
# The PASCAL VOC metric changed in 2010
use_07_metric = True if int(self._year) < 2010 else False
print('VOC07 metric? ' + ('Yes' if use_07_metric else 'No'))
if not os.path.isdir(output_dir):
os.mkdir(output_dir)
cls_names = []
ap_values = []
for i, cls in enumerate(self._classes):
if cls == '__background__':
continue
filename = self._get_voc_results_file_template().format(cls)
rec, prec, ap = voc_eval(
filename, annopath, imagesetfile, cls, cachedir, ovthresh=0.5,
use_07_metric=use_07_metric)
aps += [ap]
print('AP for {} = {:.3f}'.format(cls, ap))
cls_names.append(cls)
ap_values.append(("%.1f" % (ap*100)))
if i == 15:
cls_names.append('mean')
tmp = np.mean(aps)*100
ap_values.append(("%.1f" % tmp))
if i == 20:
cls_names.append('mean')
tmp = np.mean(aps[-5:])*100
ap_values.append(("%.1f" % tmp))
with open(os.path.join(output_dir, cls + '_pr.pkl'), 'wb') as f:
pickle.dump({'rec': rec, 'prec': prec, 'ap': ap}, f)
print('Mean AP = {:.4f}'.format(np.mean(aps)))
writer.writerow(cls_names)
writer.writerow(ap_values)
csvfile.close()
print('~~~~~~~~')
print('Results:')
for ap in aps:
print('{:.3f}'.format(ap))
print('{:.3f}'.format(np.mean(aps)))
print('~~~~~~~~')
print('')
print('--------------------------------------------------------------')
print('Results computed with the **unofficial** Python eval code.')
print('Results should be very close to the official MATLAB eval code.')
print('Recompute with `./tools/reval.py --matlab ...` for your paper.')
print('-- Thanks, The Management')
print('--------------------------------------------------------------')
def _do_matlab_eval(self, output_dir='output'):
print('-----------------------------------------------------')
print('Computing results with the official MATLAB eval code.')
print('-----------------------------------------------------')
path = os.path.join(cfg.ROOT_DIR, 'lib', 'datasets',
'VOCdevkit-matlab-wrapper')
cmd = 'cd {} && '.format(path)
cmd += '{:s} -nodisplay -nodesktop '.format(cfg.MATLAB)
cmd += '-r "dbstop if error; '
cmd += 'voc_eval(\'{:s}\',\'{:s}\',\'{:s}\',\'{:s}\'); quit;"' \
.format(self._devkit_path, self._get_comp_id(),
self._image_set, output_dir)
print('Running:\n{}'.format(cmd))
status = subprocess.call(cmd, shell=True)
def evaluate_detections(self, all_boxes, output_dir, **kwargs):
self._write_voc_results_file(all_boxes)
self._do_python_eval(output_dir, **kwargs)
if self.config['matlab_eval']:
self._do_matlab_eval(output_dir)
if self.config['cleanup']:
for cls in self._classes:
if cls == '__background__':
continue
filename = self._get_voc_results_file_template().format(cls)
os.remove(filename)
def competition_mode(self, on):
if on:
self.config['use_salt'] = False
self.config['cleanup'] = False
else:
self.config['use_salt'] = True
self.config['cleanup'] = True
if __name__ == '__main__':
d = pascal_voc('trainval', '2007')
res = d.roidb
from IPython import embed;
embed()
================================================
FILE: lib/datasets/pascal_voc_rbg.py
================================================
# --------------------------------------------------------
# Fast R-CNN
# Copyright (c) 2015 Microsoft
# Licensed under The MIT License [see LICENSE for details]
# Written by Ross Girshick and Xinlei Chen
# --------------------------------------------------------
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import os
from datasets.imdb import imdb
import scipy.sparse
import pickle
import subprocess
import uuid
from .voc_eval import voc_eval
from model.utils.config import cfg
import pdb
import os
import os.path
import sys
import torch.utils.data as data
import cv2
import numpy as np
if sys.version_info[0] == 2:
import xml.etree.cElementTree as ET
else:
import xml.etree.ElementTree as ET
class VOCDetection(data.Dataset):
"""VOC Detection Dataset Object
input is image, target is annotation
Arguments:
root (string): filepath to VOCdevkit folder.
image_set (string): imageset to use (eg. 'train', 'val', 'test')
transform (callable, optional): transformation to perform on the
input image
target_transform (callable, optional): transformation to perform on the
target `annotation`
(eg: take in caption string, return tensor of word indices)
dataset_name (string, optional): which dataset to load
(default: 'VOC2007')
"""
def __init__(self, root, image_sets, img_size, preproc=None, target_transform=None,
dataset_name='VOC0712'):
self.root = root
self.image_set = image_sets
self.img_size = img_size
self.preproc = preproc
self.target_transform = target_transform
self.name = dataset_name
self._annopath = os.path.join('%s', 'Annotations', '%s.xml')
self._imgpath = os.path.join('%s', 'JPEGImages', '%s.jpg')
self.ids = list()
for (year, name) in image_sets:
self._year = year
rootpath = os.path.join(self.root, 'VOC' + year)
for line in open(os.path.join(rootpath, 'ImageSets', 'Main', name + '.txt')):
self.ids.append((rootpath, line.strip()))
def __getitem__(self, index):
img_id = self.ids[index]
target = ET.parse(self._annopath % img_id).getroot()
img = cv2.imread(self._imgpath % img_id, cv2.IMREAD_COLOR)
height, width, _ = img.shape
mask = np.zeros((self.img_size, self.img_size), dtype=np.uint8)
h, w, _ = img.shape
y_ration = float(h) / self.img_size
x_ration = float(w) / self.img_size
img_resize = cv2.resize(img, (self.img_size, self.img_size))
labels = []
for obj in target.iter('object'):
difficult = int(obj.find('difficult').text) == 1
if difficult:
continue
name = obj.find('name').text.strip()
labels.append(name)
bbox = obj.find('bndbox')
pts = ['xmin', 'ymin', 'xmax', 'ymax']
bndbox = []
for i, pt in enumerate(pts):
cur_pt = int(float(bbox.find(pt).text)) - 1
if i % 2 == 0:
cur_pt = int(cur_pt / x_ration)
bndbox.append(cur_pt)
elif i % 2 == 1:
cur_pt = int(cur_pt / y_ration)
bndbox.append(cur_pt)
mask[bndbox[0]:bndbox[2], bndbox[1]:bndbox[3]] = 1
return img_resize, mask, labels
def __len__(self):
return len(self.ids)
class pascal_voc(imdb):
def __init__(self, image_set, year, devkit_path=None):
imdb.__init__(self, 'voc_' + year + '_' + image_set)
self._year = year
self._image_set = image_set
self._devkit_path = self._get_default_path() if devkit_path is None \
else devkit_path
self._data_path = os.path.join(self._devkit_path, 'VOC' + self._year)
self._classes = ('__background__', # always index 0
'aeroplane', 'bicycle', 'bird', 'boat',
'bottle', 'bus', 'car', 'cat', 'chair',
'cow', 'diningtable', 'dog', 'horse',
'motorbike', 'person', 'pottedplant',
'sheep', 'sofa', 'train', 'tvmonitor')
self._class_to_ind = dict(list(zip(self.classes, list(range(self.num_classes)))))
self._image_ext = '.jpg'
self._image_index = self._load_image_set_index()
# Default to roidb handler
self._roidb_handler = self.gt_roidb
self._salt = str(uuid.uuid4())
self._comp_id = 'comp4'
# PASCAL specific config options
self.config = {'cleanup': True,
'use_salt': True,
'use_diff': False,
'matlab_eval': False,
'rpn_file': None}
assert os.path.exists(self._devkit_path), \
'VOCdevkit path does not exist: {}'.format(self._devkit_path)
assert os.path.exists(self._data_path), \
'Path does not exist: {}'.format(self._data_path)
def image_path_at(self, i):
"""
Return the absolute path to image i in the image sequence.
"""
return self.image_path_from_index(self._image_index[i])
def image_path_from_index(self, index):
"""
Construct an image path from the image's "index" identifier.
"""
image_path = os.path.join(self._data_path, 'JPEGImages',
index + self._image_ext)
assert os.path.exists(image_path), \
'Path does not exist: {}'.format(image_path)
return image_path
def _load_image_set_index(self):
"""
Load the indexes listed in this dataset's image set file.
"""
# Example path to image set file:
# self._devkit_path + /VOCdevkit2007/VOC2007/ImageSets/Main/val.txt
image_set_file = os.path.join(self._data_path, 'ImageSets', 'Main',
self._image_set + '.txt')
assert os.path.exists(image_set_file), \
'Path does not exist: {}'.format(image_set_file)
with open(image_set_file) as f:
image_index = [x.strip() for x in f.readlines()]
return image_index
def _get_default_path(self):
"""
Return the default path where PASCAL VOC is expected to be installed.
"""
return os.path.join(cfg.DATA_DIR, 'VOCdevkit' + self._year)
def gt_roidb(self):
"""
Return the database of ground-truth regions of interest.
This function loads/saves from/to a cache file to speed up future calls.
"""
cache_file = os.path.join(self.cache_path, self.name + '_gt_roidb.pkl')
if os.path.exists(cache_file):
with open(cache_file, 'rb') as fid:
try:
roidb = pickle.load(fid)
except:
roidb = pickle.load(fid, encoding='bytes')
print('{} gt roidb loaded from {}'.format(self.name, cache_file))
return roidb
gt_roidb = [self._load_pascal_annotation(index)
for index in self.image_index]
with open(cache_file, 'wb') as fid:
pickle.dump(gt_roidb, fid, pickle.HIGHEST_PROTOCOL)
print('wrote gt roidb to {}'.format(cache_file))
return gt_roidb
def rpn_roidb(self):
if int(self._year) == 2007 or self._image_set != 'test':
gt_roidb = self.gt_roidb()
rpn_roidb = self._load_rpn_roidb(gt_roidb)
roidb = imdb.merge_roidbs(gt_roidb, rpn_roidb)
else:
roidb = self._load_rpn_roidb(None)
return roidb
def _load_rpn_roidb(self, gt_roidb):
filename = self.config['rpn_file']
print('loading {}'.format(filename))
assert os.path.exists(filename), \
'rpn data not found at: {}'.format(filename)
with open(filename, 'rb') as f:
box_list = pickle.load(f)
return self.create_roidb_from_box_list(box_list, gt_roidb)
def _load_pascal_annotation(self, index):
"""
Load image and bounding boxes info from XML file in the PASCAL VOC
format.
"""
filename = os.path.join(self._data_path, 'Annotations', index + '.xml')
tree = ET.parse(filename)
objs = tree.findall('object')
if not self.config['use_diff']:
# Exclude the samples labeled as difficult
non_diff_objs = [
obj for obj in objs if int(obj.find('difficult').text) == 0]
# if len(non_diff_objs) != len(objs):
# print 'Removed {} difficult objects'.format(
# len(objs) - len(non_diff_objs))
objs = non_diff_objs
num_objs = len(objs)
boxes = np.zeros((num_objs, 4), dtype=np.uint16)
gt_classes = np.zeros((num_objs), dtype=np.int32)
overlaps = np.zeros((num_objs, self.num_classes), dtype=np.float32)
# "Seg" area for pascal is just the box area
seg_areas = np.zeros((num_objs), dtype=np.float32)
# Load object bounding boxes into a data frame.
for ix, obj in enumerate(objs):
bbox = obj.find('bndbox')
# Make pixel indexes 0-based
x1 = float(bbox.find('xmin').text) - 1
y1 = float(bbox.find('ymin').text) - 1
x2 = float(bbox.find('xmax').text) - 1
y2 = float(bbox.find('ymax').text) - 1
cls = self._class_to_ind[obj.find('name').text.lower().strip()]
boxes[ix, :] = [x1, y1, x2, y2]
gt_classes[ix] = cls
overlaps[ix, cls] = 1.0
seg_areas[ix] = (x2 - x1 + 1) * (y2 - y1 + 1)
overlaps = scipy.sparse.csr_matrix(overlaps)
return {'boxes': boxes,
'gt_classes': gt_classes,
'gt_overlaps': overlaps,
'flipped': False,
'seg_areas': seg_areas}
def _get_comp_id(self):
comp_id = (self._comp_id + '_' + self._salt if self.config['use_salt']
else self._comp_id)
return comp_id
def _get_voc_results_file_template(self):
# VOCdevkit/results/VOC2007/Main/<comp_id>_det_test_aeroplane.txt
filename = self._get_comp_id() + '_det_' + self._image_set + '_{:s}.txt'
path = os.path.join(
self._devkit_path,
'results',
'VOC' + self._year,
'Main',
filename)
return path
def _write_voc_results_file(self, all_boxes):
for cls_ind, cls in enumerate(self.classes):
if cls == '__background__':
continue
print('Writing {} VOC results file'.format(cls))
filename = self._get_voc_results_file_template().format(cls)
with open(filename, 'wt') as f:
for im_ind, index in enumerate(self.image_index):
dets = all_boxes[cls_ind][im_ind]
if dets == []:
continue
# the VOCdevkit expects 1-based indices
for k in range(dets.shape[0]):
f.write('{:s} {:.3f} {:.1f} {:.1f} {:.1f} {:.1f}\n'.
format(index, dets[k, -1],
dets[k, 0] + 1, dets[k, 1] + 1,
dets[k, 2] + 1, dets[k, 3] + 1))
def _do_python_eval(self, output_dir='output'):
annopath = os.path.join(
self._devkit_path,
'VOC' + self._year,
'Annotations',
'{:s}.xml')
imagesetfile = os.path.join(
self._devkit_path,
'VOC' + self._year,
'ImageSets',
'Main',
self._image_set + '.txt')
cachedir = os.path.join(self._devkit_path, 'annotations_cache')
aps = []
# The PASCAL VOC metric changed in 2010
use_07_metric = True if int(self._year) < 2010 else False
print('VOC07 metric? ' + ('Yes' if use_07_metric else 'No'))
if not os.path.isdir(output_dir):
os.mkdir(output_dir)
for i, cls in enumerate(self._classes):
if cls == '__background__':
continue
filename = self._get_voc_results_file_template().format(cls)
rec, prec, ap = voc_eval(
filename, annopath, imagesetfile, cls, cachedir, ovthresh=0.5,
use_07_metric=use_07_metric)
aps += [ap]
print(('AP for {} = {:.4f}'.format(cls, ap)))
with open(os.path.join(output_dir, cls + '_pr.pkl'), 'wb') as f:
pickle.dump({'rec': rec, 'prec': prec, 'ap': ap}, f)
print(('Mean AP = {:.4f}'.format(np.mean(aps))))
print('~~~~~~~~')
print('Results:')
for ap in aps:
print(('{:.3f}'.format(ap)))
print(('{:.3f}'.format(np.mean(aps))))
print('~~~~~~~~')
print('')
print('--------------------------------------------------------------')
print('Results computed with the **unofficial** Python eval code.')
print('Results should be very close to the official MATLAB eval code.')
print('Recompute with `./tools/reval.py --matlab ...` for your paper.')
print('-- Thanks, The Management')
print('--------------------------------------------------------------')
def _do_matlab_eval(self, output_dir='output'):
print('-----------------------------------------------------')
print('Computing results with the official MATLAB eval code.')
print('-----------------------------------------------------')
path = os.path.join(cfg.ROOT_DIR, 'lib', 'datasets',
'VOCdevkit-matlab-wrapper')
cmd = 'cd {} && '.format(path)
cmd += '{:s} -nodisplay -nodesktop '.format(cfg.MATLAB)
cmd += '-r "dbstop if error; '
cmd += 'voc_eval(\'{:s}\',\'{:s}\',\'{:s}\',\'{:s}\'); quit;"' \
.format(self._devkit_path, self._get_comp_id(),
self._image_set, output_dir)
print(('Running:\n{}'.format(cmd)))
status = subprocess.call(cmd, shell=True)
def evaluate_detections(self, all_boxes, output_dir):
pdb.set_trace()
self._write_voc_results_file(all_boxes)
self._do_python_eval(output_dir)
if self.config['matlab_eval']:
self._do_matlab_eval(output_dir)
if self.config['cleanup']:
for cls in self._classes:
if cls == '__background__':
continue
filename = self._get_voc_results_file_template().format(cls)
os.remove(filename)
def competition_mode(self, on):
if on:
self.config['use_salt'] = False
self.config['cleanup'] = False
else:
self.config['use_salt'] = True
self.config['cleanup'] = True
if __name__ == '__main__':
from datasets.pascal_voc import pascal_voc
d = pascal_voc('trainval', '2007')
res = d.roidb
from IPython import embed;
embed()
================================================
FILE: lib/datasets/tools/compute_prior.py
================================================
import numpy as np
import pickle
import os
import sys
NUM_ATTR_REL = 200
def cout_w(prob, num=NUM_ATTR_REL,dim=1):
prob_weight = prob[:, :num]
sum_value = np.sum(prob_weight, keepdims=True, axis=dim) + 0.1
prob_weight = prob_weight / np.repeat(sum_value, prob_weight.shape[dim], axis=dim)
return prob_weight
def cp_kl(a, b):
# compute kl diverse
if np.sum(a) == 0 or np.sum(b) == 0:
return 1
sum_ = a * np.log(a / b)
all_value = [x for x in sum_ if str(x) != 'nan' and str(x) != 'inf']
kl = np.sum(all_value)
return kl
def compute_js(attr_prob):
cls_num = attr_prob.shape[0]
similarity = np.zeros((cls_num, cls_num))
similarity[0, 1:] = 1
similarity[1:, 0] = 1
for i in range(1, cls_num):
if i % 50 == 0:
print('had proccessed {} cls...\n'.format(i))
for j in range(1, cls_num):
if i == j:
similarity[i,j] = 0
else:
similarity[i,j] = 0.5 * (cp_kl(attr_prob[i, :], 0.5*(attr_prob[i, :] + attr_prob[j,:]))
+ cp_kl(attr_prob[j, :], 0.5*(attr_prob[i, :] + attr_prob[j, :])))
return similarity
if __name__=='__main__':
data_path = '/data/VisualGenome/graph/'
dim_ = 1000
## Compute attribute knowledge by JS-diversion
graph_a = pickle.load(open(data_path + 'vg_attr_frequency_1000.pkl', 'rb'))
## You can get part of graph_a and match name with your datasets
# We give an example of compute graph of VisualGenome with 1000 classes
# first line of graph_a is background
graph_a = cout_w(graph_a, num=len(graph_a))
graph_a = compute_js(graph_a)
graph_a = 1 - graph_a
pickle.dump(graph_a, open(data_path + 'vg_graph_a.pkl', 'wb'))
## Compute relation knowledge
graph_r = pickle.load(open(data_path + 'vg_pair_frequency_1000.pkl', 'rb'))
## You can get part of graph_a and match name with your datasets
# We give an example of compute graph of VisualGenome with 1000 classes
relation_matrix = np.zeros((dim_, dim_))
relation_matrix = graph_r + graph_r.transpose()
relation_matrix_row_sum = relation_matrix.sum(1)
for i in range(dim_):
relation_matrix[i, i] = relation_matrix_row_sum[i] + 1.
prob_relation_matrix = np.zeros((dim_, dim_))
for i in range(dim_):
for j in range(dim_):
prob_relation_matrix[i, j] = relation_matrix[i, j] / (
np.sqrt(relation_matrix[i, i]) * np.sqrt(relation_matrix[j, j]))
prob_relation_matrix_ba = np.zeros((dim_ + 1, dim_ + 1))
prob_relation_matrix_ba[1:, 1:] = prob_relation_matrix
print(prob_relation_matrix_ba.shape)
pickle.dump(prob_relation_matrix_ba, open(data_path + 'vg_graph_r.pkl', 'wb'))
================================================
FILE: lib/datasets/tools/mcg_munge.py
================================================
from __future__ import print_function
import os
import sys
"""Hacky tool to convert file system layout of MCG boxes downloaded from
http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/mcg/
so that it's consistent with those computed by Jan Hosang (see:
http://www.mpi-inf.mpg.de/departments/computer-vision-and-multimodal-
computing/research/object-recognition-and-scene-understanding/how-
good-are-detection-proposals-really/)
NB: Boxes from the MCG website are in (y1, x1, y2, x2) order.
Boxes from Hosang et al. are in (x1, y1, x2, y2) order.
"""
def munge(src_dir):
# stored as: ./MCG-COCO-val2014-boxes/COCO_val2014_000000193401.mat
# want: ./MCG/mat/COCO_val2014_0/COCO_val2014_000000141/COCO_val2014_000000141334.mat
files = os.listdir(src_dir)
for fn in files:
base, ext = os.path.splitext(fn)
# first 14 chars / first 22 chars / all chars + .mat
# COCO_val2014_0/COCO_val2014_000000447/COCO_val2014_000000447991.mat
first = base[:14]
second = base[:22]
dst_dir = os.path.join('MCG', 'mat', first, second)
if not os.path.exists(dst_dir):
os.makedirs(dst_dir)
src = os.path.join(src_dir, fn)
dst = os.path.join(dst_dir, fn)
print('MV: {} -> {}'.format(src, dst))
os.rename(src, dst)
if __name__ == '__main__':
# src_dir should look something like:
# src_dir = 'MCG-COCO-val2014-boxes'
src_dir = sys.argv[1]
munge(src_dir)
================================================
FILE: lib/datasets/voc_eval.py
================================================
# --------------------------------------------------------
# Fast/er R-CNN
# Licensed under The MIT License [see LICENSE for details]
# Written by Bharath Hariharan
# --------------------------------------------------------
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import xml.etree.ElementTree as ET
import os
import pickle
import numpy as np
def parse_rec(filename):
""" Parse a PASCAL VOC xml file """
tree = ET.parse(filename)
objects = []
for obj in tree.findall('object'):
obj_struct = {}
obj_struct['name'] = obj.find('name').text
obj_struct['pose'] = obj.find('pose').text
obj_struct['truncated'] = int(obj.find('truncated').text)
obj_struct['difficult'] = int(obj.find('difficult').text)
bbox = obj.find('bndbox')
obj_struct['bbox'] = [int(bbox.find('xmin').text),
int(bbox.find('ymin').text),
int(bbox.find('xmax').text),
int(bbox.find('ymax').text)]
objects.append(obj_struct)
return objects
def voc_ap(rec, prec, use_07_metric=False):
""" ap = voc_ap(rec, prec, [use_07_metric])
Compute VOC AP given precision and recall.
If use_07_metric is true, uses the
VOC 07 11 point method (default:False).
"""
if use_07_metric:
# 11 point metric
ap = 0.
for t in np.arange(0., 1.1, 0.1):
if np.sum(rec >= t) == 0:
p = 0
else:
p = np.max(prec[rec >= t])
ap = ap + p / 11.
else:
# correct AP calculation
# first append sentinel values at the end
mrec = np.concatenate(([0.], rec, [1.]))
mpre = np.concatenate(([0.], prec, [0.]))
# compute the precision envelope
for i in range(mpre.size - 1, 0, -1):
mpre[i - 1] = np.maximum(mpre[i - 1], mpre[i])
# to calculate area under PR curve, look for points
# where X axis (recall) changes value
i = np.where(mrec[1:] != mrec[:-1])[0]
# and sum (\Delta recall) * prec
ap = np.sum((mrec[i + 1] - mrec[i]) * mpre[i + 1])
return ap
def voc_eval(detpath,
annopath,
imagesetfile,
classname,
cachedir,
ovthresh=0.5,
use_07_metric=False):
"""rec, prec, ap = voc_eval(detpath,
annopath,
imagesetfile,
classname,
[ovthresh],
[use_07_metric])
Top level function that does the PASCAL VOC evaluation.
detpath: Path to detections
detpath.format(classname) should produce the detection results file.
annopath: Path to annotations
annopath.format(imagename) should be the xml annotations file.
imagesetfile: Text file containing the list of images, one image per line.
classname: Category name (duh)
cachedir: Directory for caching the annotations
[ovthresh]: Overlap threshold (default = 0.5)
[use_07_metric]: Whether to use VOC07's 11 point AP computation
(default False)
"""
# assumes detections are in detpath.format(classname)
# assumes annotations are in annopath.format(imagename)
# assumes imagesetfile is a text file with each line an image name
# cachedir caches the annotations in a pickle file
# first load gt
if not os.path.isdir(cachedir):
os.mkdir(cachedir)
cachefile = os.path.join(cachedir, 'annots.pkl')
# read list of images
with open(imagesetfile, 'r') as f:
lines = f.readlines()
imagenames = [x.strip() for x in lines]
if not os.path.isfile(cachefile):
# load annotations
recs = {}
for i, imagename in enumerate(imagenames):
recs[imagename] = parse_rec(annopath.format(imagename))
if i % 100 == 0:
print('Reading annotation for {:d}/{:d}'.format(
i + 1, len(imagenames)))
# save
print('Saving cached annotations to {:s}'.format(cachefile))
with open(cachefile, 'wb') as f:
pickle.dump(recs, f)
else:
# load
with open(cachefile, 'rb') as f:
try:
recs = pickle.load(f)
except:
recs = pickle.load(f, encoding='bytes')
# extract gt objects for this class
class_recs = {}
npos = 0
for imagename in imagenames:
R = [obj for obj in recs[imagename] if obj['name'] == classname]
bbox = np.array([x['bbox'] for x in R])
difficult = np.array([x['difficult'] for x in R]).astype(np.bool)
det = [False] * len(R)
npos = npos + sum(~difficult)
class_recs[imagename] = {'bbox': bbox,
'difficult': difficult,
'det': det}
# read dets
detfile = detpath.format(classname)
with open(detfile, 'r') as f:
lines = f.readlines()
if len(lines) == 0:
# No detection examples
return 0, 0, 0, 0, npos
splitlines = [x.strip().split(' ') for x in lines]
image_ids = [x[0] for x in splitlines]
confidence = np.array([float(x[1]) for x in splitlines])
BB = np.array([[float(z) for z in x[2:]] for x in splitlines])
nd = len(image_ids)
tp = np.zeros(nd)
fp = np.zeros(nd)
if BB.shape[0] > 0:
# sort by confidence
sorted_ind = np.argsort(-confidence)
sorted_scores = np.sort(-confidence)
BB = BB[sorted_ind, :]
image_ids = [image_ids[x] for x in sorted_ind]
# go down dets and mark TPs and FPs
for d in range(nd):
R = class_recs[image_ids[d]]
bb = BB[d, :].astype(float)
ovmax = -np.inf
BBGT = R['bbox'].astype(float)
if BBGT.size > 0:
# compute overlaps
# intersection
ixmin = np.maximum(BBGT[:, 0], bb[0])
iymin = np.maximum(BBGT[:, 1], bb[1])
ixmax = np.minimum(BBGT[:, 2], bb[2])
iymax = np.minimum(BBGT[:, 3], bb[3])
iw = np.maximum(ixmax - ixmin + 1., 0.)
ih = np.maximum(iymax - iymin + 1., 0.)
inters = iw * ih
# union
uni = ((bb[2] - bb[0] + 1.) * (bb[3] - bb[1] + 1.) +
(BBGT[:, 2] - BBGT[:, 0] + 1.) *
(BBGT[:, 3] - BBGT[:, 1] + 1.) - inters)
overlaps = inters / uni
ovmax = np.max(overlaps)
jmax = np.argmax(overlaps)
if ovmax > ovthresh:
if not R['difficult'][jmax]:
if not R['det'][jmax]:
tp[d] = 1.
R['det'][jmax] = 1
else:
fp[d] = 1.
else:
fp[d] = 1.
# compute precision recall
fp = np.cumsum(fp)
tp = np.cumsum(tp)
rec = tp / float(npos)
# avoid divide by zero in case the first detection matches a difficult
# ground truth
prec = tp / np.maximum(tp + fp, np.finfo(np.float64).eps)
ap = voc_ap(rec, prec, use_07_metric)
return rec, prec, ap
================================================
FILE: lib/model/__init__.py
================================================
================================================
FILE: lib/model/faster_rcnn/__init__.py
================================================
================================================
FILE: lib/model/faster_rcnn/faster_rcnn.py
================================================
# --------------------------------------------------------
# Pytorch Meta R-CNN
# Written by Anny Xu, Xiaopeng Yan, based on the code from Jianwei Yang
# --------------------------------------------------------
import random
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.autograd import Variable
import torchvision.models as models
from torch.autograd import Variable
import numpy as np
from model.utils.config import cfg
from model.rpn.rpn import _RPN
from model.roi_pooling.modules.roi_pool import _RoIPooling
from model.roi_crop.modules.roi_crop import _RoICrop
from model.roi_align.modules.roi_align import RoIAlignAvg
from model.rpn.proposal_target_layer_cascade import _ProposalTargetLayer
import time
import pdb
from model.utils.net_utils import _smooth_l1_loss, _crop_pool_layer, _affine_grid_gen, _affine_theta
import pickle
class _fasterRCNN(nn.Module):
""" faster RCNN """
def __init__(self, classes, class_agnostic, meta_train, meta_test=None, meta_loss=None):
super(_fasterRCNN, self).__init__()
self.classes = classes
self.n_classes = len(classes)
self.class_agnostic = class_agnostic
self.meta_train = meta_train
self.meta_test = meta_test
self.meta_loss = meta_loss
# loss
self.RCNN_loss_cls = 0
self.RCNN_loss_bbox = 0
# define rpn
self.RCNN_rpn = _RPN(self.dout_base_model)
self.RCNN_proposal_target = _ProposalTargetLayer(self.n_classes)
self.RCNN_roi_pool = _RoIPooling(cfg.POOLING_SIZE, cfg.POOLING_SIZE, 1.0 / 16.0)
self.RCNN_roi_align = RoIAlignAvg(cfg.POOLING_SIZE, cfg.POOLING_SIZE, 1.0 / 16.0)
self.grid_size = cfg.POOLING_SIZE * 2 if cfg.CROP_RESIZE_WITH_MAX_POOL else cfg.POOLING_SIZE
self.RCNN_roi_crop = _RoICrop()
def forward(self, im_data_list, im_info_list, gt_boxes_list, num_boxes_list, average_shot=None,
mean_class_attentions=None):
# return attentions for testing
if average_shot:
prn_data = im_data_list[0] # len(metaclass)*4*224*224
attentions = self.prn_network(prn_data)
return attentions
# extract attentions for training
if self.meta_train and self.training:
prn_data = im_data_list[0] # len(metaclass)*4*224*224
# feed prn data to prn_network
attentions = self.prn_network(prn_data)
prn_cls = im_info_list[0] # len(metaclass)
im_data = im_data_list[-1]
im_info = im_info_list[-1]
gt_boxes = gt_boxes_list[-1]
num_boxes = num_boxes_list[-1]
batch_size = im_data.size(0)
im_info = im_info.data
gt_boxes = gt_boxes.data
num_boxes = num_boxes.data
# feed image data to base model to obtain base feature map
base_feat = self.RCNN_base(self.rcnn_conv1(im_data))
# feed base feature map tp RPN to obtain rois
rois, rpn_loss_cls, rpn_loss_bbox = self.RCNN_rpn(base_feat, im_info, gt_boxes, num_boxes)
# if it is training phase, then use ground truth bboxes for refining
if self.training:
roi_data = self.RCNN_proposal_target(rois, gt_boxes, num_boxes)
rois, rois_label, rois_target, rois_inside_ws, rois_outside_ws = roi_data
rois_label = Variable(rois_label.view(-1).long())
rois_target = Variable(rois_target.view(-1, rois_target.size(2)))
rois_inside_ws = Variable(rois_inside_ws.view(-1, rois_inside_ws.size(2)))
rois_outside_ws = Variable(rois_outside_ws.view(-1, rois_outside_ws.size(2)))
else:
rois_label = None
rois_target = None
rois_inside_ws = None
rois_outside_ws = None
rpn_loss_cls = 0
rpn_loss_bbox = 0
rois = Variable(rois)
# do roi pooling based on predicted rois
if cfg.POOLING_MODE == 'crop':
# pooled_feat_anchor = _crop_pool_layer(base_feat, rois.view(-1, 5))
grid_xy = _affine_grid_gen(rois.view(-1, 5), base_feat.size()[2:], self.grid_size)
grid_yx = torch.stack([grid_xy.data[:, :, :, 1], grid_xy.data[:, :, :, 0]], 3).contiguous()
pooled_feat = self.RCNN_roi_crop(base_feat, Variable(grid_yx).detach())
if cfg.CROP_RESIZE_WITH_MAX_POOL:
pooled_feat = F.max_pool2d(pooled_feat, 2, 2)
elif cfg.POOLING_MODE == 'align':
pooled_feat = self.RCNN_roi_align(base_feat, rois.view(-1, 5)) # (b*128)*1024*7*7
elif cfg.POOLING_MODE == 'pool':
pooled_feat = self.RCNN_roi_pool(base_feat, rois.view(-1, 5))
# feed pooled features to top model
pooled_feat = self._head_to_tail(pooled_feat) # (b*128)*2048
# meta training phase
if self.meta_train:
rcnn_loss_cls = []
rcnn_loss_bbox = []
# pooled feature maps need to operate channel-wise multiplication with the corresponding class's attentions of every roi of image
for b in range(batch_size):
zero = Variable(torch.FloatTensor([0]).cuda())
proposal_labels = rois_label[b * 128:(b + 1) * 128].data.cpu().numpy()[0]
unique_labels = list(np.unique(proposal_labels)) # the unique rois labels of the input image
for i in range(attentions.size(0)): # attentions len(attentions)*2048
if prn_cls[i].numpy()[0] + 1 not in unique_labels:
rcnn_loss_cls.append(zero)
rcnn_loss_bbox.append(zero)
continue
channel_wise_feat = pooled_feat[b * cfg.TRAIN.BATCH_SIZE:(b + 1) * cfg.TRAIN.BATCH_SIZE, :] * \
attentions[i] # 128x2048 channel-wise multiple
bbox_pred = self.RCNN_bbox_pred(channel_wise_feat) # 128 * 4
if self.training and not self.class_agnostic:
# select the corresponding columns according to roi labels
bbox_pred_view = bbox_pred.view(bbox_pred.size(0), int(bbox_pred.size(1) / 4), 4)
bbox_pred_select = torch.gather(bbox_pred_view, 1,
rois_label[
b * cfg.TRAIN.BATCH_SIZE:(b + 1) * cfg.TRAIN.BATCH_SIZE].view(
rois_label[b * cfg.TRAIN.BATCH_SIZE:(
b + 1) * cfg.TRAIN.BATCH_SIZE].size(
0), 1, 1).expand(
rois_label[b * cfg.TRAIN.BATCH_SIZE:(
b + 1) * cfg.TRAIN.BATCH_SIZE].size(
0), 1,
4))
bbox_pred = bbox_pred_select.squeeze(1)
# compute object classification probability
cls_score = self.RCNN_cls_score(channel_wise_feat) # 128 * 21
if self.training:
# classification loss
RCNN_loss_cls = F.cross_entropy(cls_score, rois_label[b * 128:(b + 1) * 128])
rcnn_loss_cls.append(RCNN_loss_cls)
# bounding box regression L1 loss
RCNN_loss_bbox = _smooth_l1_loss(bbox_pred, rois_target[b * 128:(b + 1) * 128],
rois_inside_ws[b * 128:(b + 1) * 128],
rois_outside_ws[b * 128:(b + 1) * 128])
rcnn_loss_bbox.append(RCNN_loss_bbox)
# meta attentions loss
if self.meta_loss:
attentions_score = self.Meta_cls_score(attentions)
meta_loss = F.cross_entropy(attentions_score, Variable(torch.cat(prn_cls,dim=0).cuda()))
else:
meta_loss = 0
return rois, rpn_loss_cls, rpn_loss_bbox, rcnn_loss_cls, rcnn_loss_bbox, rois_label, 0, 0, meta_loss
elif self.meta_test:
cls_prob_list = []
bbox_pred_list = []
for i in range(len(mean_class_attentions)):
mean_attentions = mean_class_attentions[i]
channel_wise_feat = pooled_feat * mean_attentions
# compute bbox offset
bbox_pred = self.RCNN_bbox_pred(channel_wise_feat)
if self.training and not self.class_agnostic:
# select the corresponding columns according to roi labels
bbox_pred_view = bbox_pred.view(bbox_pred.size(0), int(bbox_pred.size(1) / 4), 4)
bbox_pred_select = torch.gather(bbox_pred_view, 1,
rois_label.view(rois_label.size(0), 1, 1).expand(rois_label.size(0),
1, 4))
bbox_pred = bbox_pred_select.squeeze(1)
# compute object classification probability
cls_score = self.RCNN_cls_score(channel_wise_feat)
cls_prob = F.softmax(cls_score)
RCNN_loss_cls = 0
RCNN_loss_bbox = 0
if self.training:
# classification loss
RCNN_loss_cls = F.cross_entropy(cls_score, rois_label)
# bounding box regression L1 loss
RCNN_loss_bbox = _smooth_l1_loss(bbox_pred, rois_target, rois_inside_ws, rois_outside_ws)
cls_prob = cls_prob.view(batch_size, rois.size(1), -1)
bbox_pred = bbox_pred.view(batch_size, rois.size(1), -1)
cls_prob_list.append(cls_prob)
bbox_pred_list.append(bbox_pred)
return rois, rpn_loss_cls, rpn_loss_bbox, RCNN_loss_cls, RCNN_loss_bbox, rois_label, cls_prob_list, bbox_pred_list, 0
else:
bbox_pred = self.RCNN_bbox_pred(pooled_feat)
if self.training and not self.class_agnostic:
# select the corresponding columns according to roi labels
bbox_pred_view = bbox_pred.view(bbox_pred.size(0), int(bbox_pred.size(1) / 4), 4)
bbox_pred_select = torch.gather(bbox_pred_view, 1,
rois_label.view(rois_label.size(0), 1, 1).expand(rois_label.size(0), 1,
4))
bbox_pred = bbox_pred_select.squeeze(1)
# compute object classification probability
cls_score = self.RCNN_cls_score(pooled_feat) # 128 * 1001
cls_prob = F.softmax(cls_score)
RCNN_loss_cls = 0
RCNN_loss_bbox = 0
if self.training:
# classification loss
RCNN_loss_cls = F.cross_entropy(cls_score, rois_label)
# bounding box regression L1 loss
RCNN_loss_bbox = _smooth_l1_loss(bbox_pred, rois_target, rois_inside_ws, rois_outside_ws)
cls_prob = cls_prob.view(batch_size, rois.size(1), -1)
bbox_pred = bbox_pred.view(batch_size, rois.size(1), -1)
return rois, rpn_loss_cls, rpn_loss_bbox, RCNN_loss_cls, RCNN_loss_bbox, rois_label, cls_prob, bbox_pred, 0
def _init_weights(self):
def normal_init(m, mean, stddev, truncated=False):
"""
weight initalizer: truncated normal and random normal.
"""
# x is a parameter
if truncated:
m.weight.data.normal_().fmod_(2).mul_(stddev).add_(mean) # not a perfect approximation
else:
m.weight.data.normal_(mean, stddev)
m.bias.data.zero_()
normal_init(self.RCNN_rpn.RPN_Conv, 0, 0.01, cfg.TRAIN.TRUNCATED)
normal_init(self.RCNN_rpn.RPN_cls_score, 0, 0.01, cfg.TRAIN.TRUNCATED)
normal_init(self.RCNN_rpn.RPN_bbox_pred, 0, 0.01, cfg.TRAIN.TRUNCATED)
normal_init(self.RCNN_cls_score, 0, 0.01, cfg.TRAIN.TRUNCATED)
normal_init(self.RCNN_bbox_pred, 0, 0.001, cfg.TRAIN.TRUNCATED)
def create_architecture(self):
self._init_modules()
self._init_weights()
================================================
FILE: lib/model/faster_rcnn/resnet.py
================================================
# --------------------------------------------------------
# Pytorch Meta R-CNN
# Written by Anny Xu, Xiaopeng Yan, based on code from Jianwei Yang
# --------------------------------------------------------
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from model.utils.config import cfg
from model.faster_rcnn.faster_rcnn import _fasterRCNN
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.autograd import Variable
from torch.nn import init
import math
import torch.utils.model_zoo as model_zoo
import pdb
__all__ = ['ResNet', 'resnet18', 'resnet34', 'resnet50', 'resnet101',
'resnet152']
model_urls = {
'resnet18': 'https://s3.amazonaws.com/pytorch/models/resnet18-5c106cde.pth',
'resnet34': 'https://s3.amazonaws.com/pytorch/models/resnet34-333f7ec4.pth',
'resnet50': 'https://s3.amazonaws.com/pytorch/models/resnet50-19c8e357.pth',
'resnet101': 'https://s3.amazonaws.com/pytorch/models/resnet101-5d3b4d8f.pth',
'resnet152': 'https://s3.amazonaws.com/pytorch/models/resnet152-b121ed2d.pth',
}
def conv3x3(in_planes, out_planes, stride=1):
"3x3 convolution with padding"
return nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=stride,
padding=1, bias=False)
def init_conv(conv,glu=True):
init.xavier_uniform(conv.weight)
if conv.bias is not None:
conv.bias.data.zero_()
def init_linear(linear):
init.constant(linear.weight,0)
init.constant(linear.bias, 1)
class BasicBlock(nn.Module):
expansion = 1
def __init__(self, inplanes, planes, stride=1, downsample=None):
super(BasicBlock, self).__init__()
self.conv1 = conv3x3(inplanes, planes, stride)
self.bn1 = nn.BatchNorm2d(planes)
self.relu = nn.ReLU(inplace=True)
self.conv2 = conv3x3(planes, planes)
self.bn2 = nn.BatchNorm2d(planes)
self.downsample = downsample
self.stride = stride
def forward(self, x):
residual = x
out = self.conv1(x)
out = self.bn1(out)
out = self.relu(out)
out = self.conv2(out)
out = self.bn2(out)
if self.downsample is not None:
residual = self.downsample(x)
out += residual
out = self.relu(out)
return out
class Bottleneck(nn.Module):
expansion = 4
def __init__(self, inplanes, planes, stride=1, downsample=None):
super(Bottleneck, self).__init__()
self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=1, stride=stride, bias=False) # change
self.bn1 = nn.BatchNorm2d(planes)
self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=1, # change
padding=1, bias=False)
self.bn2 = nn.BatchNorm2d(planes)
self.conv3 = nn.Conv2d(planes, planes * 4, kernel_size=1, bias=False)
self.bn3 = nn.BatchNorm2d(planes * 4)
self.relu = nn.ReLU(inplace=True)
self.downsample = downsample
self.stride = stride
def forward(self, x):
residual = x
out = self.conv1(x)
out = self.bn1(out)
out = self.relu(out)
out = self.conv2(out)
out = self.bn2(out)
out = self.relu(out)
out = self.conv3(out)
out = self.bn3(out)
if self.downsample is not None:
residual = self.downsample(x)
out += residual
out = self.relu(out)
return out
class ResNet(nn.Module):
def __init__(self, block, layers, num_classes=1000):
self.inplanes = 64
super(ResNet, self).__init__()
self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3,
bias=False)
self.bn1 = nn.BatchNorm2d(64)
self.relu = nn.ReLU(inplace=True)
self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=0, ceil_mode=True) # change
self.layer1 = self._make_layer(block, 64, layers[0])
self.layer2 = self._make_layer(block, 128, layers[1], stride=2)
self.layer3 = self._make_layer(block, 256, layers[2], stride=2)
self.layer4 = self._make_layer(block, 512, layers[3], stride=2)
# it is slightly better whereas slower to set stride = 1
# self.layer4 = self._make_layer(block, 512, layers[3], stride=1)
self.avgpool = nn.AvgPool2d(7)
self.fc = nn.Linear(512 * block.expansion, num_classes)
for m in self.modules():
if isinstance(m, nn.Conv2d):
n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
m.weight.data.normal_(0, math.sqrt(2. / n))
elif isinstance(m, nn.BatchNorm2d):
m.weight.data.fill_(1)
m.bias.data.zero_()
def _make_layer(self, block, planes, blocks, stride=1):
downsample = None
if stride != 1 or self.inplanes != planes * block.expansion:
downsample = nn.Sequential(
nn.Conv2d(self.inplanes, planes * block.expansion,
kernel_size=1, stride=stride, bias=False),
nn.BatchNorm2d(planes * block.expansion),
)
layers = []
layers.append(block(self.inplanes, planes, stride, downsample))
self.inplanes = planes * block.expansion
for i in range(1, blocks):
layers.append(block(self.inplanes, planes))
return nn.Sequential(*layers)
def forward(self, x):
x = self.conv1(x)
x = self.bn1(x)
x = self.relu(x)
x = self.maxpool(x)
x = self.layer1(x)
x = self.layer2(x)
x = self.layer3(x)
x = self.layer4(x)
x = self.avgpool(x)
x = x.view(x.size(0), -1)
x = self.fc(x)
return x
def resnet18(pretrained=False):
"""Constructs a ResNet-18 model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
"""
model = ResNet(BasicBlock, [2, 2, 2, 2])
if pretrained:
model.load_state_dict(model_zoo.load_url(model_urls['resnet18']))
return model
def resnet34(pretrained=False):
"""Constructs a ResNet-34 model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
"""
model = ResNet(BasicBlock, [3, 4, 6, 3])
if pretrained:
model.load_state_dict(model_zoo.load_url(model_urls['resnet34']))
return model
def resnet50(pretrained=False):
"""Constructs a ResNet-50 model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
"""
model = ResNet(Bottleneck, [3, 4, 6, 3])
if pretrained:
model.load_state_dict(model_zoo.load_url(model_urls['resnet50']))
return model
def resnet101(pretrained=False):
"""Constructs a ResNet-101 model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
"""
model = ResNet(Bottleneck, [3, 4, 23, 3])
if pretrained:
model.load_state_dict(model_zoo.load_url(model_urls['resnet101']))
return model
def resnet152(pretrained=False):
"""Constructs a ResNet-152 model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
"""
model = ResNet(Bottleneck, [3, 8, 36, 3])
if pretrained:
model.load_state_dict(model_zoo.load_url(model_urls['resnet152']))
return model
class resnet(_fasterRCNN):
def __init__(self, classes, num_layers=101, pretrained=False, class_agnostic=False,meta_train=True,meta_test=None,meta_loss=None):
self.model_path = 'data/pretrained_model/resnet101_caffe.pth'
self.dout_base_model = 1024
self.pretrained = pretrained
self.class_agnostic = class_agnostic
self.meta_train = meta_train
self.meta_test = meta_test
self.meta_loss = meta_loss
_fasterRCNN.__init__(self, classes, class_agnostic,meta_train,meta_test,meta_loss)
def _init_modules(self):
resnet = resnet101()
if self.pretrained == True:
print("Loading pretrained weights from %s" %(self.model_path))
state_dict = torch.load(self.model_path)
resnet.load_state_dict({k:v for k,v in state_dict.items() if k in resnet.state_dict()})
# Build resnet.
self.meta_conv1 = nn.Conv2d(4, 64, kernel_size=7, stride=2, padding=3, bias=False)
self.rcnn_conv1 = resnet.conv1
self.RCNN_base = nn.Sequential(resnet.bn1,resnet.relu,
resnet.maxpool,resnet.layer1,resnet.layer2,resnet.layer3)
self.RCNN_top = nn.Sequential(resnet.layer4)
self.sigmoid = nn.Sigmoid()
self.max_pooled = nn.MaxPool2d(2)
self.RCNN_cls_score = nn.Linear(2048, self.n_classes)
if self.meta_loss:
self.Meta_cls_score = nn.Linear(2048, self.n_classes)
if self.class_agnostic:
self.RCNN_bbox_pred = nn.Linear(2048, 4) # x,y,w,h
else:
self.RCNN_bbox_pred = nn.Linear(2048, 4 * self.n_classes)
# Fix blocks
for p in self.rcnn_conv1.parameters(): p.requires_grad=False
for p in self.RCNN_base[0].parameters(): p.requires_grad=False
assert (0 <= cfg.RESNET.FIXED_BLOCKS < 5)
if cfg.RESNET.FIXED_BLOCKS >= 4:
for p in self.RCNN_top.parameters(): p.requires_grad = False
if cfg.RESNET.FIXED_BLOCKS >= 3:
for p in self.RCNN_base[5].parameters(): p.requires_grad=False
if cfg.RESNET.FIXED_BLOCKS >= 2:
for p in self.RCNN_base[4].parameters(): p.requires_grad=False
if cfg.RESNET.FIXED_BLOCKS >= 1:
for p in self.RCNN_base[3].parameters(): p.requires_grad=False
def set_bn_fix(m):
classname = m.__class__.__name__
if classname.find('BatchNorm') != -1:
for p in m.parameters(): p.requires_grad=False
self.RCNN_base.apply(set_bn_fix)
self.RCNN_top.apply(set_bn_fix)
def train(self, mode=True):
# Override train so that the training mode is set as we want
nn.Module.train(self, mode)
if mode:
# Set fixed blocks to be in eval mode
self.RCNN_base.eval()
self.RCNN_base[4].train()
self.RCNN_base[5].train()
self.RCNN_base.eval()
def set_bn_eval(m):
classname = m.__class__.__name__
if classname.find('BatchNorm') != -1:
m.eval()
self.RCNN_base.apply(set_bn_eval)
self.RCNN_top.apply(set_bn_eval)
def _head_to_tail(self, pool5):
fc7 = self.RCNN_top(pool5).mean(3).mean(2)
return fc7
def prn_network(self,im_data):
'''
the Predictor-head Remodeling Network (PRN)
:param im_data:
:return attention vectors:
'''
base_feat = self.RCNN_base(self.meta_conv1(im_data))
feature = self._head_to_tail(self.max_pooled(base_feat))
attentions = self.sigmoid(feature)
return attentions
================================================
FILE: lib/model/faster_rcnn/trail.py
================================================
# --------------------------------------------------------
# Pytorch multi-GPU Faster R-CNN
# Licensed under The MIT License [see LICENSE for details]
# Written by Jiasen Lu, Jianwei Yang, based on code from Ross Girshick
# --------------------------------------------------------
import _init_paths
import os
import sys
import numpy as np
import argparse
import pprint
import pdb
import time
import torch
from torch.autograd import Variable
import torch.nn as nn
import torch.optim as optim
import torchvision.transforms as transforms
from torch.utils.data.sampler import Sampler
from roi_data_layer.roidb import combined_roidb
from roi_data_layer.roibatchLoader import roibatchLoader
from model.utils.config import cfg, cfg_from_file, cfg_from_list, get_output_dir
from model.utils.net_utils import weights_normal_init, save_net, load_net, \
adjust_learning_rate, save_checkpoint, clip_gradient
from model.faster_rcnn.vgg16 import vgg16
from model.faster_rcnn.resnet_GNN import resnet
import pickle
def cout_w(prob):
prob_weight = prob
sum_value = np.sum(prob_weight)
prob_weight = prob_weight / sum_value
meam_value = np.mean(prob_weight)
prob_weight = prob_weight - meam_value + 1
prob_weight = np.log(prob_weight)
return prob_weight
def cout_a(prob, NUM_ATTR_REL):
prob_weight = prob[:, : NUM_ATTR_REL]
sum_value = np.sum(prob_weight, keepdims=True, axis=1)
zero_inds = np.nonzero(sum_value == 0)[0]
sum_value[zero_inds,:] = 1.
prob_weight = prob_weight / np.repeat(sum_value, prob_weight.shape[1], axis=1)
meam_value = np.mean(prob_weight, keepdims=True, axis=1)
meam_value = np.repeat(meam_value, prob_weight.shape[1], axis=1)
prob_weight = prob_weight - meam_value + 1
prob_weight = np.log(prob_weight)
cls_cls_a = np.zeros((1001, 1001))
for row in range(prob_weight.shape[0]):
temp = np.zeros(1001)
for col in range(prob_weight.shape[1]):
xx = prob_weight[row, col] * prob_weight[:, col]
temp += xx
cls_cls_a[row] = temp
pickle.dump(cls_cls_a, open('data/vg/clscls_attr.pkl', 'wb'))
return cls_cls_a
def parse_args():
"""
Parse input arguments
"""
parser = argparse.ArgumentParser(description='Train a Fast R-CNN network')
parser.add_argument('--dataset', dest='dataset',
help='training dataset',
default='vg', type=str)
parser.add_argument('--net', dest='net',
help='vgg16, res101',
default='baseline', type=str)
parser.add_argument('--start_epoch', dest='start_epoch',
help='starting epoch',
default=1, type=int)
parser.add_argument('--epochs', dest='max_epochs',
help='number of epochs to train',
default=20, type=int)
parser.add_argument('--disp_interval', dest='disp_interval',
help='number of iterations to display',
default=100, type=int)
parser.add_argument('--checkpoint_interval', dest='checkpoint_interval',
help='number of iterations to display',
default=10000, type=int)
parser.add_argument('--save_dir', dest='save_dir',
help='directory to save models', default="exps/baseline/models",
nargs=argparse.REMAINDER)
parser.add_argument('--nw', dest='num_workers',
help='number of worker to load data',
default=2, type=int)
parser.add_argument('--cuda', dest='cuda',default=True, type=bool,
help='whether use CUDA')
parser.add_argument('--ls', dest='large_scale',
help='whether use large imag scale',
action='store_true')
parser.add_argument('--ms', dest='multi_scale',
help='whether to use multi scale training',
action='store_true')
parser.add_argument('--mGPUs', dest='mGPUs',
help='whether use multiple GPUs',
action='store_true')
parser.add_argument('--bs', dest='batch_size',
help='batch_size',
default=2, type=int)
parser.add_argument('--cag', dest='class_agnostic',default=False, type=bool,
help='whether perform class_agnostic bbox regression')
# config optimization
parser.add_argument('--o', dest='optimizer',
help='training optimizer',
default="sgd", type=str)
parser.add_argument('--lr', dest='lr',
help='starting learning rate',
default=0.001, type=float)
parser.add_argument('--lr_decay_step', dest='lr_decay_step',
help='step to do learning rate decay, unit is epoch',
default=4, type=int)
parser.add_argument('--lr_decay_gamma', dest='lr_decay_gamma',
help='learning rate decay ratio',
default=0.1, type=float)
# set training session
parser.add_argument('--s', dest='session',
help='training session',
default=1, type=int)
# resume trained model
parser.add_argument('--r', dest='resume',
help='resume checkpoint or not',
default=False, type=bool)
parser.add_argument('--checksession', dest='checksession',
help='checksession to load model',
default=1, type=int)
parser.add_argument('--checkepoch', dest='checkepoch',
help='checkepoch to load model',
default=1, type=int)
parser.add_argument('--checkpoint', dest='checkpoint',
help='checkpoint to load model',
default=0, type=int)
# log and diaplay
parser.add_argument('--use_tfboard', dest='use_tfboard',
help='whether use tensorflow tensorboard',
default=False, type=bool)
parser.add_argument('--log_dir', dest='log_dir',
help='directory to save logs', default='logs',
type=str)
args = parser.parse_args()
return args
class sampler(Sampler):
def __init__(self, train_size, batch_size):
self.num_data = train_size
self.num_per_batch = int(train_size / batch_size)
self.batch_size = batch_size
self.range = torch.arange(0,batch_size).view(1, batch_size).long()
self.leftover_flag = False
if train_size % batch_size:
self.leftover = torch.arange(self.num_per_batch*batch_size, train_size).long()
self.leftover_flag = True
def __iter__(self):
rand_num = torch.randperm(self.num_per_batch).view(-1,1) * self.batch_size
self.rand_num = rand_num.expand(self.num_per_batch, self.batch_size) + self.range
self.rand_num_view = self.rand_num.view(-1)
if self.leftover_flag:
self.rand_num_view = torch.cat((self.rand_num_view, self.leftover),0)
return iter(self.rand_num_view)
def __len__(self):
return self.num_data
args = parse_args()
print('Called with args:')
print(args)
if args.use_tfboard:
from model.utils.logger import Logger
# Set the logger
logger = Logger(args.log_dir)
if args.dataset == "pascal_voc":
args.imdb_name = "voc_2007_trainval"
args.imdbval_name = "voc_2007_test"
args.set_cfgs = ['ANCHOR_SCALES', '[8, 16, 32]', 'ANCHOR_RATIOS', '[0.5,1,2]', 'MAX_NUM_GT_BOXES', '20']
elif args.dataset == "pascal_voc_0712":
args.imdb_name = "voc_2007_trainval+voc_2012_trainval"
args.imdbval_name = "voc_2007_test"
args.set_cfgs = ['ANCHOR_SCALES', '[8, 16, 32]', 'ANCHOR_RATIOS', '[0.5,1,2]', 'MAX_NUM_GT_BOXES', '20']
elif args.dataset == "coco":
args.imdb_name = "coco_2014_train+coco_2014_valminusminival"
args.imdbval_name = "coco_2014_minival"
args.set_cfgs = ['ANCHOR_SCALES', '[4, 8, 16, 32]', 'ANCHOR_RATIOS', '[0.5,1,2]', 'MAX_NUM_GT_BOXES', '50']
elif args.dataset == "imagenet":
args.imdb_name = "imagenet_train"
args.imdbval_name = "imagenet_val"
args.set_cfgs = ['ANCHOR_SCALES', '[4, 8, 16, 32]', 'ANCHOR_RATIOS', '[0.5,1,2]', 'MAX_NUM_GT_BOXES', '30']
elif args.dataset == "vg":
# train sizes: train, smalltrain, minitrain
# train scale: ['150-50-20', '150-50-50', '500-150-80', '750-250-150', '1750-700-450', '1600-400-20']
args.imdb_name = "vg_train"
args.imdbval_name = "vg_val"
args.set_cfgs = ['ANCHOR_SCALES', '[2, 4, 8, 16, 32]', 'MAX_NUM_GT_BOXES', '50']
# args.set_cfgs = ['ANCHOR_SCALES', '[4, 8, 16, 32]', 'ANCHOR_RATIOS', '[0.5,1,2]', 'MAX_NUM_GT_BOXES', '50']
# args.cfg_file = "cfgs/{}_ls.yml".format(args.net) if args.large_scale else "cfgs/{}.yml".format(args.net)
args.cfg_file = "cfgs/res101_ms.yml"#.format(args.net + "_ms" if args.multi_scale else "")
if args.cfg_file is not None:
cfg_from_file(args.cfg_file)
if args.set_cfgs is not None:
cfg_from_list(args.set_cfgs)
print('Using config:')
pprint.pprint(cfg)
np.random.seed(cfg.RNG_SEED)
#torch.backends.cudnn.benchmark = True
if torch.cuda.is_available() and not args.cuda:
print("WARNING: You have a CUDA device, so you should probably run with --cuda")
# train set
# -- Note: Use validation set and disable the flipped to enable faster loading.
cfg.TRAIN.USE_FLIPPED = True
cfg.USE_GPU_NMS = args.cuda
imdb, roidb, ratio_list, ratio_index = combined_roidb(args.imdb_name)
train_size = len(roidb)
print('{:d} roidb entries'.format(len(roidb)))
sys.stdout.flush()
output_dir = args.save_dir[0] + "/" + args.net + "/" + args.dataset
if not os.path.exists(output_dir):
os.makedirs(output_dir)
# log_f = open(os.path.join(args.log_dir, time.strftime("%Y-%m-%d-%H:%M.txt", time.localtime())), "w")
# old_stdout = sys.stdout
# sys.stdout = log_f
sampler_batch = sampler(train_size, args.batch_size)
dataset = roibatchLoader(roidb, ratio_list, ratio_index, args.batch_size, imdb.num_classes, training=True)
dataloader = torch.utils.data.DataLoader(dataset, batch_size=args.batch_size,
sampler=sampler_batch, num_workers=args.num_workers, pin_memory=False)
# initilize the tensor holder here.
im_data = torch.FloatTensor(1)
im_info = torch.FloatTensor(1)
num_boxes = torch.LongTensor(1)
gt_boxes = torch.FloatTensor(1)
# attr_prob = torch.FloatTensor(imdb._clscls_attr)
# attr_prob = torch.FloatTensor(cout_a(imdb._class_to_attr, 200))
# pair_prob = torch.FloatTensor(cout_w(imdb.pair))
# pair_prob = pair_prob.unsqueeze(0).repeat(args.batch_size, 1, 1)
# ship to cuda
if args.cuda:
im_data = im_data.cuda()
im_info = im_info.cuda()
num_boxes = num_boxes.cuda()
gt_boxes = gt_boxes.cuda()
# make variable
im_data = Variable(im_data)
im_info = Variable(im_info)
num_boxes = Variable(num_boxes)
gt_boxes = Variable(gt_boxes)
# pair_prob = Variable(pair_prob)
# attr_prob = Variable(attr_prob)
if args.cuda:
cfg.CUDA = True
# initilize the network here.
if args.net == 'baseline':
fasterRCNN = resnet(imdb.classes, 101, pretrained=True, class_agnostic=args.class_agnostic)
elif args.net == 'relationloss':
fasterRCNN = resnet(imdb.classes, 101, pretrained=False, class_agnostic=args.class_agnostic, pair_prob=pair_prob)
elif args.net == 'attributeloss':
fasterRCNN = resnet(imdb.classes, 101, pretrained=True, class_agnostic=args.class_agnostic, attr_prob=attr_prob)
fasterRCNN.create_architecture()
lr = cfg.TRAIN.LEARNING_RATE
lr = args.lr
#tr_momentum = cfg.TRAIN.MOMENTUM
#tr_momentum = args.momentum
params = []
for key, value in dict(fasterRCNN.named_parameters()).items():
if value.requires_grad:
if 'bias' in key:
params += [{'params':[value],'lr':lr*(cfg.TRAIN.DOUBLE_BIAS + 1), \
'weight_decay': cfg.TRAIN.BIAS_DECAY and cfg.TRAIN.WEIGHT_DECAY or 0}]
else:
params += [{'params':[value],'lr':lr, 'weight_decay': cfg.TRAIN.WEIGHT_DECAY}]
if args.optimizer == "adam":
lr = lr * 0.1
optimizer = torch.optim.Adam(params)
elif args.optimizer == "sgd":
optimizer = torch.optim.SGD(params, momentum=cfg.TRAIN.MOMENTUM)
if args.resume:
load_name = os.path.join(output_dir,
'faster_rcnn_{}_{}_{}.pth'.format(args.checksession, args.checkepoch, args.checkpoint))
print("loading checkpoint %s" % (load_name))
checkpoint = torch.load(load_name)
args.session = checkpoint['session']
args.start_epoch = checkpoint['epoch']
fasterRCNN.load_state_dict(checkpoint['model'])
optimizer.load_state_dict(checkpoint['optimizer'])
lr = optimizer.param_groups[0]['lr']
if 'pooling_mode' in checkpoint.keys():
cfg.POOLING_MODE = checkpoint['pooling_mode']
print("loaded checkpoint %s" % (load_name))
if args.mGPUs:
fasterRCNN = nn.DataParallel(fasterRCNN)
if args.cuda:
fasterRCNN.cuda()
iters_per_epoch = int(train_size / args.batch_size)
fasterRCNN = resnet(imdb.classes, 101, pretrained=True, class_agnostic=args.class_agnostic)
fasterRCNN.create_architecture()
fasterRCNN.cuda()
fasterRCNN.train()
data_iter = iter(dataloader)
data = next(data_iter)
im_data.data.resize_(data[0].size()).copy_(data[0])
im_info.data.resize_(data[1].size()).copy_(data[1])
gt_boxes.data.resize_(data[2].size()).copy_(data[2])
num_boxes.data.resize_(data[3].size()).copy_(data[3])
fasterRCNN.zero_grad()
batch_size = im_data.size(0)
im_info = im_info.data
gt_boxes = gt_boxes.data
num_boxes = num_boxes.data
# feed image data to base model to obtain base feature map
base_feat = fasterRCNN.RCNN_base(im_data)
# feed base feature map tp RPN to obtain rois
rois, rpn_loss_cls, rpn_loss_bbox = fasterRCNN.RCNN_rpn(base_feat, im_info, gt_boxes, num_boxes)
# if it is training phrase, then use ground trubut bboxes for refining
if fasterRCNN.training:
roi_data = fasterRCNN.RCNN_proposal_target(rois, gt_boxes, num_boxes)
rois, rois_label, rois_target, rois_inside_ws, rois_outside_ws = roi_data
rois_label = Variable(rois_label.view(-1).long())
rois_target = Variable(rois_target.view(-1, rois_target.size(2)))
rois_inside_ws = Variable(rois_inside_ws.view(-1, rois_inside_ws.size(2)))
rois_outside_ws = Variable(rois_outside_ws.view(-1, rois_outside_ws.size(2)))
else:
rois_label = None
rois_target = None
rois_inside_ws = None
rois_outside_ws = None
rpn_loss_cls = 0
rpn_loss_bbox = 0
rois = Variable(rois)
# do roi pooling based on predicted rois
if cfg.POOLING_MODE == 'crop':
# pdb.set_trace()
# pooled_feat_anchor = _crop_pool_layer(base_feat, rois.view(-1, 5))
grid_xy = _affine_grid_gen(rois.view(-1, 5), base_feat.size()[2:], fasterRCNN.grid_size)
grid_yx = torch.stack([grid_xy.data[:,:,:,1], grid_xy.data[:,:,:,0]], 3).contiguous()
pooled_feat = fasterRCNN.RCNN_roi_crop(base_feat, Variable(grid_yx).detach())
if cfg.CROP_RESIZE_WITH_MAX_POOL:
pooled_feat = F.max_pool2d(pooled_feat, 2, 2)
elif cfg.POOLING_MODE == 'align':
pooled_feat = fasterRCNN.RCNN_roi_align(base_feat, rois.view(-1, 5))
elif cfg.POOLING_MODE == 'pool':
pooled_feat = fasterRCNN.RCNN_roi_pool(base_feat, rois.view(-1,5))
================================================
FILE: lib/model/faster_rcnn/vgg16.py
================================================
# --------------------------------------------------------
# Tensorflow Faster R-CNN
# Licensed under The MIT License [see LICENSE for details]
# Written by Xinlei Chen
# --------------------------------------------------------
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.autograd import Variable
import math
import torchvision.models as models
from model.faster_rcnn.faster_rcnn import _fasterRCNN
import pdb
class vgg16(_fasterRCNN):
def __init__(self, classes, pretrained=False, class_agnostic=False):
self.model_path = 'data/pretrained_model/vgg16_caffe.pth'
self.dout_base_model = 512
self.pretrained = pretrained
self.class_agnostic = class_agnostic
_fasterRCNN.__init__(self, classes, class_agnostic)
def _init_modules(self):
vgg = models.vgg16()
if self.pretrained:
print("Loading pretrained weights from %s" %(self.model_path))
state_dict = torch.load(self.model_path)
vgg.load_state_dict({k:v for k,v in state_dict.items() if k in vgg.state_dict()})
vgg.classifier = nn.Sequential(*list(vgg.classifier._modules.values())[:-1])
# not using the last maxpool layer
self.RCNN_base = nn.Sequential(*list(vgg.features._modules.values())[:-1])
# Fix the layers before conv3:
for layer in range(10):
for p in self.RCNN_base[layer].parameters(): p.requires_grad = False
# self.RCNN_base = _RCNN_base(vgg.features, self.classes, self.dout_base_model)
self.RCNN_top = vgg.classifier
# not using the last maxpool layer
self.RCNN_cls_score = nn.Linear(4096, self.n_classes)
if self.class_agnostic:
self.RCNN_bbox_pred = nn.Linear(4096, 4)
else:
self.RCNN_bbox_pred = nn.Linear(4096, 4 * self.n_classes)
def _head_to_tail(self, pool5):
pool5_flat = pool5.view(pool5.size(0), -1)
fc7 = self.RCNN_top(pool5_flat)
return fc7
================================================
FILE: lib/model/nms/.gitignore
================================================
*.c
*.cpp
*.so
================================================
FILE: lib/model/nms/__init__.py
================================================
================================================
FILE: lib/model/nms/_ext/__init__.py
================================================
================================================
FILE: lib/model/nms/_ext/nms/__init__.py
================================================
from torch.utils.ffi import _wrap_function
from ._nms import lib as _lib, ffi as _ffi
__all__ = []
def _import_symbols(locals):
for symbol in dir(_lib):
fn = getattr(_lib, symbol)
if callable(fn):
locals[symbol] = _wrap_function(fn, _ffi)
else:
locals[symbol] = fn
__all__.append(symbol)
_import_symbols(locals())
================================================
FILE: lib/model/nms/build.py
================================================
from __future__ import print_function
import os
import torch
from torch.utils.ffi import create_extension
#this_file = os.path.dirname(__file__)
sources = []
headers = []
defines = []
with_cuda = False
if torch.cuda.is_available():
print('Including CUDA code.')
sources += ['src/nms_cuda.c']
headers += ['src/nms_cuda.h']
defines += [('WITH_CUDA', None)]
with_cuda = True
this_file = os.path.dirname(os.path.realpath(__file__))
print(this_file)
extra_objects = ['src/nms_cuda_kernel.cu.o']
extra_objects = [os.path.join(this_file, fname) for fname in extra_objects]
print(extra_objects)
ffi = create_extension(
'_ext.nms',
headers=headers,
sources=sources,
define_macros=defines,
relative_to=__file__,
with_cuda=with_cuda,
extra_objects=extra_objects
)
if __name__ == '__main__':
ffi.build()
================================================
FILE: lib/model/nms/nms_cpu.py
================================================
from __future__ import absolute_import
import numpy as np
import torch
def nms_cpu(dets, thresh):
dets = dets.cpu().numpy()
x1 = dets[:, 0]
y1 = dets[:, 1]
x2 = dets[:, 2]
y2 = dets[:, 3]
scores = dets[:, 4]
areas = (x2 - x1 + 1) * (y2 - y1 + 1)
order = scores.argsort()[::-1]
keep = []
while order.size > 0:
i = order.item(0)
keep.append(i)
xx1 = np.maximum(x1[i], x1[order[1:]])
yy1 = np.maximum(y1[i], y1[order[1:]])
xx2 = np.minimum(x2[i], x2[order[1:]])
yy2 = np.minimum(y2[i], y2[order[1:]])
w = np.maximum(0.0, xx2 - xx1 + 1)
h = np.maximum(0.0, yy2 - yy1 + 1)
inter = w * h
ovr = inter / (areas[i] + areas[order[1:]] - inter)
inds = np.where(ovr <= thresh)[0]
order = order[inds + 1]
return torch.IntTensor(keep)
def nms_cpu_np(dets, thresh):
x1 = dets[:, 0]
y1 = dets[:, 1]
x2 = dets[:, 2]
y2 = dets[:, 3]
scores = dets[:, 4]
areas = (x2 - x1 + 1) * (y2 - y1 + 1)
order = scores.argsort()[::-1]
keep = []
while order.size > 0:
i = order.item(0)
keep.append(i)
xx1 = np.maximum(x1[i], x1[order[1:]])
yy1 = np.maximum(y1[i], y1[order[1:]])
xx2 = np.minimum(x2[i], x2[order[1:]])
yy2 = np.minimum(y2[i], y2[order[1:]])
w = np.maximum(0.0, xx2 - xx1 + 1)
h = np.maximum(0.0, yy2 - yy1 + 1)
inter = w * h
ovr = inter / (areas[i] + areas[order[1:]] - inter)
inds = np.where(ovr <= thresh)[0]
order = order[inds + 1]
return keep
def soft_nms_cpu(dets, threshold=0.001, Nt=0.3, method=1):
boxes = dets.cpu().numpy()
N = dets.shape[0]
pos = 0
maxscore = 0
maxpos = 0
for i in range(N):
maxscore = boxes[i, 4]
maxpos = i
tx1 = boxes[i,0]
ty1 = boxes[i,1]
tx2 = boxes[i,2]
ty2 = boxes[i,3]
ts = boxes[i,4]
pos = i + 1
# get max box
while pos < N:
if maxscore < boxes[pos, 4]:
maxscore = boxes[pos, 4]
maxpos = pos
pos = pos + 1
# add max box as a detection
boxes[i,0] = boxes[maxpos,0]
boxes[i,1] = boxes[maxpos,1]
boxes[i,2] = boxes[maxpos,2]
boxes[i,3] = boxes[maxpos,3]
boxes[i,4] = boxes[maxpos,4]
# swap ith box with position of max box
boxes[maxpos,0] = tx1
boxes[maxpos,1] = ty1
boxes[maxpos,2] = tx2
boxes[maxpos,3] = ty2
boxes[maxpos,4] = ts
tx1 = boxes[i,0]
ty1 = boxes[i,1]
tx2 = boxes[i,2]
ty2 = boxes[i,3]
ts = boxes[i,4]
pos = i + 1
# NMS iterations, note that N changes if detection boxes fall below threshold
while pos < N:
x1 = boxes[pos, 0]
y1 = boxes[pos, 1]
x2 = boxes[pos, 2]
y2 = boxes[pos, 3]
s = boxes[pos, 4]
area = (x2 - x1 + 1) * (y2 - y1 + 1)
iw = (min(tx2, x2) - max(tx1, x1) + 1)
if iw > 0:
ih = (min(ty2, y2) - max(ty1, y1) + 1)
if ih > 0:
ua = float((tx2 - tx1 + 1) * (ty2 - ty1 + 1) + area - iw * ih)
ov = iw * ih / ua #iou between max box and detection box
if method == 1: # linear
if ov > Nt:
weight = 1 - ov
else:
weight = 1
elif method == 2: # gaussian
weight = np.exp(-(ov * ov)/sigma)
else: # original NMS
if ov > Nt:
weight = 0
else:
weight = 1
boxes[pos, 4] = weight*boxes[pos, 4]
# if box score falls below threshold, discard the box by swapping with last box
# update N
if boxes[pos, 4] < threshold:
boxes[pos,0] = boxes[N-1, 0]
boxes[pos,1] = boxes[N-1, 1]
boxes[pos,2] = boxes[N-1, 2]
boxes[pos,3] = boxes[N-1, 3]
boxes[pos,4] = boxes[N-1, 4]
N = N - 1
pos = pos - 1
pos = pos + 1
keep = [i for i in range(N)]
return keep, boxes
def nms_domain(dets, dets_small, thresh_small=0.85, thresh_big=0.5):
# dets = dets.cpu().numpy()
# dets_small = dets_small.cpu().numpy()
x1 = dets[:, 0]
y1 = dets[:, 1]
x2 = dets[:, 2]
y2 = dets[:, 3]
scores = dets[:, 4]
x21 = dets_small[:, 0]
y21 = dets_small[:, 1]
x22 = dets_small[:, 2]
y22 = dets_small[:, 3]
scores2 = dets_small[:, 4]
areas = (x2 - x1 + 1) * (y2 - y1 + 1)
order = scores.argsort()[::-1]
areas2 = (x22 - x21 + 1) * (y22 - y21 + 1)
order2 = scores2.argsort()[::-1]
throw = set()
keep = set(list(range(len(dets_small))))
for i in range(len(dets)):
xx1 = np.maximum(x1[i], x21)
yy1 = np.maximum(y1[i], y21)
xx2 = np.minimum(x2[i], x22)
yy2 = np.minimum(y2[i], y22)
w = np.maximum(0.0, xx2 - xx1 + 1)
h = np.maximum(0.0, yy2 - yy1 + 1)
inter = w * h
ovr_1 = inter / (areas[i])
ovr_2 = inter / (areas2)
throw_array = np.where((ovr_2 > thresh_small) & (ovr_1 < thresh_big))[0].tolist()
throw.update(throw_array)
keep = list(keep - throw)
return keep
================================================
FILE: lib/model/nms/nms_gpu.py
================================================
from __future__ import absolute_import
import torch
import numpy as np
from ._ext import nms
import pdb
def nms_gpu(dets, thresh):
keep = dets.new(dets.size(0), 1).zero_().int()
num_out = dets.new(1).zero_().int()
nms.nms_cuda(keep, dets, num_out, thresh)
keep = keep[:num_out[0]]
return keep
================================================
FILE: lib/model/nms/nms_kernel.cu
================================================
// ------------------------------------------------------------------
// Faster R-CNN
// Copyright (c) 2015 Microsoft
// Licensed under The MIT License [see fast-rcnn/LICENSE for details]
// Written by Shaoqing Ren
// ------------------------------------------------------------------
#include "gpu_nms.hpp"
#include <vector>
#include <iostream>
#define CUDA_CHECK(condition) \
/* Code block avoids redefinition of cudaError_t error */ \
do { \
cudaError_t error = condition; \
if (error != cudaSuccess) { \
std::cout << cudaGetErrorString(error) << std::endl; \
} \
} while (0)
#define DIVUP(m,n) ((m) / (n) + ((m) % (n) > 0))
int const threadsPerBlock = sizeof(unsigned long long) * 8;
__device__ inline float devIoU(float const * const a, float const * const b) {
float left = max(a[0], b[0]), right = min(a[2], b[2]);
float top = max(a[1], b[1]), bottom = min(a[3], b[3]);
float width = max(right - left + 1, 0.f), height = max(bottom - top + 1, 0.f);
float interS = width * height;
float Sa = (a[2] - a[0] + 1) * (a[3] - a[1] + 1);
float Sb = (b[2] - b[0] + 1) * (b[3] - b[1] + 1);
return interS / (Sa + Sb - interS);
}
__global__ void nms_kernel(const int n_boxes, const float nms_overlap_thresh,
const float *dev_boxes, unsigned long long *dev_mask) {
const int row_start = blockIdx.y;
const int col_start = blockIdx.x;
// if (row_start > col_start) return;
const int row_size =
min(n_boxes - row_start * threadsPerBlock, threadsPerBlock);
const int col_size =
min(n_boxes - col_start * threadsPerBlock, threadsPerBlock);
__shared__ float block_boxes[threadsPerBlock * 5];
if (threadIdx.x < col_size) {
block_boxes[threadIdx.x * 5 + 0] =
dev_boxes[(threadsPerBlock * col_start + threadIdx.x) * 5 + 0];
block_boxes[threadIdx.x * 5 + 1] =
dev_boxes[(threadsPerBlock * col_start + threadIdx.x) * 5 + 1];
block_boxes[threadIdx.x * 5 + 2] =
dev_boxes[(threadsPerBlock * col_start + threadIdx.x) * 5 + 2];
block_boxes[threadIdx.x * 5 + 3] =
dev_boxes[(threadsPerBlock * col_start + threadIdx.x) * 5 + 3];
block_boxes[threadIdx.x * 5 + 4] =
dev_boxes[(threadsPerBlock * col_start + threadIdx.x) * 5 + 4];
}
__syncthreads();
if (threadIdx.x < row_size) {
const int cur_box_idx = threadsPerBlock * row_start + threadIdx.x;
const float *cur_box = dev_boxes + cur_box_idx * 5;
int i = 0;
unsigned long long t = 0;
int start = 0;
if (row_start == col_start) {
start = threadIdx.x + 1;
}
for (i = start; i < col_size; i++) {
if (devIoU(cur_box, block_boxes + i * 5) > nms_overlap_thresh) {
t |= 1ULL << i;
}
}
const int col_blocks = DIVUP(n_boxes, threadsPerBlock);
dev_mask[cur_box_idx * col_blocks + col_start] = t;
}
}
void _set_device(int device_id) {
int current_device;
CUDA_CHECK(cudaGetDevice(¤t_device));
if (current_device == device_id) {
return;
}
// The call to cudaSetDevice must come before any calls to Get, which
// may perform initialization using the GPU.
CUDA_CHECK(cudaSetDevice(device_id));
}
void _nms(int* keep_out, int* num_out, const float* boxes_host, int boxes_num,
int boxes_dim, float nms_overlap_thresh, int device_id) {
_set_device(device_id);
float* boxes_dev = NULL;
unsigned long long* mask_dev = NULL;
const int col_blocks = DIVUP(boxes_num, threadsPerBlock);
CUDA_CHECK(cudaMalloc(&boxes_dev,
boxes_num * boxes_dim * sizeof(float)));
CUDA_CHECK(cudaMemcpy(boxes_dev,
boxes_host,
boxes_num * boxes_dim * sizeof(float),
cudaMemcpyHostToDevice));
CUDA_CHECK(cudaMalloc(&mask_dev,
boxes_num * col_blocks * sizeof(unsigned long long)));
dim3 blocks(DIVUP(boxes_num, threadsPerBlock),
DIVUP(boxes_num, threadsPerBlock));
dim3 threads(threadsPerBlock);
nms_kernel<<<blocks, threads>>>(boxes_num,
nms_overlap_thresh,
boxes_dev,
mask_dev);
std::vector<unsigned long long> mask_host(boxes_num * col_blocks);
CUDA_CHECK(cudaMemcpy(&mask_host[0],
mask_dev,
sizeof(unsigned long long) * boxes_num * col_blocks,
cudaMemcpyDeviceToHost));
std::vector<unsigned long long> remv(col_blocks);
memset(&remv[0], 0, sizeof(unsigned long long) * col_blocks);
int num_to_keep = 0;
for (int i = 0; i < boxes_num; i++) {
int nblock = i / threadsPerBlock;
int inblock = i % threadsPerBlock;
if (!(remv[nblock] & (1ULL << inblock))) {
keep_out[num_to_keep++] = i;
unsigned long long *p = &mask_host[0] + i * col_blocks;
for (int j = nblock; j < col_blocks; j++) {
remv[j] |= p[j];
}
}
}
*num_out = num_to_keep;
CUDA_CHECK(cudaFree(boxes_dev));
CUDA_CHECK(cudaFree(mask_dev));
}
================================================
FILE: lib/model/nms/nms_wrapper.py
================================================
# --------------------------------------------------------
# Fast R-CNN
# Copyright (c) 2015 Microsoft
# Licensed under The MIT License [see LICENSE for details]
# Written by Ross Girshick
# --------------------------------------------------------
import torch
from model.utils.config import cfg
if torch.cuda.is_available():
from model.nms.nms_gpu import nms_gpu
from model.nms.nms_cpu import nms_cpu
def nms(dets, thresh, force_cpu=False):
"""Dispatch to either CPU or GPU NMS implementations."""
if dets.shape[0] == 0:
return []
# ---numpy version---
# original: return gpu_nms(dets, thresh, device_id=cfg.GPU_ID)
# ---pytorch version---
return nms_gpu(dets, thresh) if force_cpu == False else nms_cpu(dets, thresh)
================================================
FILE: lib/model/nms/src/nms_cuda.h
================================================
// int nms_cuda(THCudaTensor *keep_out, THCudaTensor *num_out,
// THCudaTensor *boxes_host, THCudaTensor *nms_overlap_thresh);
int nms_cuda(THCudaIntTensor *keep_out, THCudaTensor *boxes_host,
THCudaIntTensor *num_out, float nms_overlap_thresh);
================================================
FILE: lib/model/nms/src/nms_cuda_kernel.cu
================================================
// ------------------------------------------------------------------
// Faster R-CNN
// Copyright (c) 2015 Microsoft
// Licensed under The MIT License [see fast-rcnn/LICENSE for details]
// Written by Shaoqing Ren
// ------------------------------------------------------------------
#include <stdbool.h>
#include <stdio.h>
#include <vector>
#include <iostream>
#include "nms_cuda_kernel.h"
#define CUDA_WARN(XXX) \
do { if (XXX != cudaSuccess) std::cout << "CUDA Error: " << \
cudaGetErrorString(XXX) << ", at line " << __LINE__ \
<< std::endl; cudaDeviceSynchronize(); } while (0)
#define CUDA_CHECK(condition) \
/* Code block avoids redefinition of cudaError_t error */ \
do { \
cudaError_t error = condition; \
if (error != cudaSuccess) { \
std::cout << cudaGetErrorString(error) << std::endl; \
} \
} while (0)
#define DIVUP(m,n) ((m) / (n) + ((m) % (n) > 0))
int const threadsPerBlock = sizeof(unsigned long long) * 8;
__device__ inline float devIoU(float const * const a, float const * const b) {
float left = max(a[0], b[0]), right = min(a[2], b[2]);
float top = max(a[1], b[1]), bottom = min(a[3], b[3]);
float width = max(right - left + 1, 0.f), height = max(bottom - top + 1, 0.f);
float interS = width * height;
float Sa = (a[2] - a[0] + 1) * (a[3] - a[1] + 1);
float Sb = (b[2] - b[0] + 1) * (b[3] - b[1] + 1);
return interS / (Sa + Sb - interS);
}
__global__ void nms_kernel(int n_boxes, float nms_overlap_thresh,
float *dev_boxes, unsigned long long *dev_mask) {
const int row_start = blockIdx.y;
const int col_start = blockIdx.x;
// if (row_start > col_start) return;
const int row_size =
min(n_boxes - row_start * threadsPerBlock, threadsPerBlock);
const int col_size =
min(n_boxes - col_start * threadsPerBlock, threadsPerBlock);
__shared__ float block_boxes[threadsPerBlock * 5];
if (threadIdx.x < col_size) {
block_boxes[threadIdx.x * 5 + 0] =
dev_boxes[(threadsPerBlock * col_start + threadIdx.x) * 5 + 0];
block_boxes[threadIdx.x * 5 + 1] =
dev_boxes[(threadsPerBlock * col_start + threadIdx.x) * 5 + 1];
block_boxes[threadIdx.x * 5 + 2] =
dev_boxes[(threadsPerBlock * col_start + threadIdx.x) * 5 + 2];
block_boxes[threadIdx.x * 5 + 3] =
dev_boxes[(threadsPerBlock * col_start + threadIdx.x) * 5 + 3];
block_boxes[threadIdx.x * 5 + 4] =
dev_boxes[(threadsPerBlock * col_start + threadIdx.x) * 5 + 4];
}
__syncthreads();
if (threadIdx.x < row_size) {
const int cur_box_idx = threadsPerBlock * row_start + threadIdx.x;
const float *cur_box = dev_boxes + cur_box_idx * 5;
int i = 0;
unsigned long long t = 0;
int start = 0;
if (row_start == col_start) {
start = threadIdx.x + 1;
}
for (i = start; i < col_size; i++) {
if (devIoU(cur_box, block_boxes + i * 5) > nms_overlap_thresh) {
t |= 1ULL << i;
}
}
const int col_blocks = DIVUP(n_boxes, threadsPerBlock);
dev_mask[cur_box_idx * col_blocks + col_start] = t;
}
}
void nms_cuda_compute(int* keep_out, int *num_out, float* boxes_host, int boxes_num,
int boxes_dim, float nms_overlap_thresh) {
float* boxes_dev = NULL;
unsigned long long* mask_dev = NULL;
const int col_blocks = DIVUP(boxes_num, threadsPerBlock);
CUDA_CHECK(cudaMalloc(&boxes_dev,
boxes_num * boxes_dim * sizeof(float)));
CUDA_CHECK(cudaMemcpy(boxes_dev,
boxes_host,
boxes_num * boxes_dim * sizeof(float),
cudaMemcpyHostToDevice));
CUDA_CHECK(cudaMalloc(&mask_dev,
boxes_num * col_blocks * sizeof(unsigned long long)));
dim3 blocks(DIVUP(boxes_num, threadsPerBlock),
DIVUP(boxes_num, threadsPerBlock));
dim3 threads(threadsPerBlock);
// printf("i am at line %d\n", boxes_num);
// printf("i am at line %d\n", boxes_dim);
nms_kernel<<<blocks, threads>>>(boxes_num,
nms_overlap_thresh,
boxes_dev,
mask_dev);
std::vector<unsigned long long> mask_host(boxes_num * col_blocks);
CUDA_CHECK(cudaMemcpy(&mask_host[0],
mask_dev,
sizeof(unsigned long long) * boxes_num * col_blocks,
cudaMemcpyDeviceToHost));
std::vector<unsigned long long> remv(col_blocks);
memset(&remv[0], 0, sizeof(unsigned long long) * col_blocks);
// we need to create a memory for keep_out on cpu
// otherwise, the following code cannot run
int* keep_out_cpu = new int[boxes_num];
int num_to_keep = 0;
for (int i = 0; i < boxes_num; i++) {
int nblock = i / threadsPerBlock;
int inblock = i % threadsPerBlock;
if (!(remv[nblock] & (1ULL << inblock))) {
// orignal: keep_out[num_to_keep++] = i;
keep_out_cpu[num_to_keep++] = i;
unsigned long long *p = &mask_host[0] + i * col_blocks;
for (int j = nblock; j < col_blocks; j++) {
remv[j] |= p[j];
}
}
}
// copy keep_out_cpu to keep_out on gpu
CUDA_WARN(cudaMemcpy(keep_out, keep_out_cpu, boxes_num * sizeof(int),cudaMemcpyHostToDevice));
// *num_out = num_to_keep;
// original: *num_out = num_to_keep;
// copy num_to_keep to num_out on gpu
CUDA_WARN(cudaMemcpy(num_out, &num_to_keep, 1 * sizeof(int),cudaMemcpyHostToDevice));
// release cuda memory
CUDA_CHECK(cudaFree(boxes_dev));
CUDA_CHECK(cudaFree(mask_dev));
// release cpu memory
delete []keep_out_cpu;
}
================================================
FILE: lib/model/nms/src/nms_cuda_kernel.h
================================================
#ifdef __cplusplus
extern "C" {
#endif
void nms_cuda_compute(int* keep_out, int *num_out, float* boxes_host, int boxes_num,
int boxes_dim, float nms_overlap_thresh);
#ifdef __cplusplus
}
#endif
================================================
FILE: lib/model/roi_align/__init__.py
================================================
================================================
FILE: lib/model/roi_align/_ext/__init__.py
================================================
================================================
FILE: lib/model/roi_align/_ext/roi_align/__init__.py
================================================
from torch.utils.ffi import _wrap_function
from ._roi_align import lib as _lib, ffi as _ffi
__all__ = []
def _import_symbols(locals):
for symbol in dir(_lib):
fn = getattr(_lib, symbol)
if callable(fn):
locals[symbol] = _wrap_function(fn, _ffi)
else:
locals[symbol] = fn
__all__.append(symbol)
_import_symbols(locals())
================================================
FILE: lib/model/roi_align/build.py
================================================
from __future__ import print_function
import os
import torch
from torch.utils.ffi import create_extension
# sources = ['src/roi_align.c']
# headers = ['src/roi_align.h']
sources = []
headers = []
defines = []
with_cuda = False
if torch.cuda.is_available():
print('Including CUDA code.')
sources += ['src/roi_align_cuda.c']
headers += ['src/roi_align_cuda.h']
defines += [('WITH_CUDA', None)]
with_cuda = True
this_file = os.path.dirname(os.path.realpath(__file__))
print(this_file)
extra_objects = ['src/roi_align_kernel.cu.o']
extra_objects = [os.path.join(this_file, fname) for fname in extra_objects]
ffi = create_extension(
'_ext.roi_align',
headers=headers,
sources=sources,
define_macros=defines,
relative_to=__file__,
with_cuda=with_cuda,
extra_objects=extra_objects
)
if __name__ == '__main__':
ffi.build()
================================================
FILE: lib/model/roi_align/functions/__init__.py
================================================
================================================
FILE: lib/model/roi_align/functions/roi_align.py
================================================
import torch
from torch.autograd import Function
from .._ext import roi_align
# TODO use save_for_backward instead
class RoIAlignFunction(Function):
def __init__(self, aligned_height, aligned_width, spatial_scale):
self.aligned_width = int(aligned_width)
self.aligned_height = int(aligned_height)
self.spatial_scale = float(spatial_scale)
self.rois = None
self.feature_size = None
def forward(self, features, rois):
self.rois = rois
self.feature_size = features.size()
batch_size, num_channels, data_height, data_width = features.size()
num_rois = rois.size(0)
output = features.new(num_rois, num_channels, self.aligned_height, self.aligned_width).zero_()
if features.is_cuda:
roi_align.roi_align_forward_cuda(self.aligned_height,
self.aligned_width,
self.spatial_scale, features,
rois, output)
else:
raise NotImplementedError
return output
def backward(self, grad_output):
assert(self.feature_size is not None and grad_output.is_cuda)
batch_size, num_channels, data_height, data_width = self.feature_size
grad_input = self.rois.new(batch_size, num_channels, data_height,
data_width).zero_()
roi_align.roi_align_backward_cuda(self.aligned_height,
self.aligned_width,
self.spatial_scale, grad_output,
self.rois, grad_input)
# print grad_input
return grad_input, None
================================================
FILE: lib/model/roi_align/modules/__init__.py
================================================
================================================
FILE: lib/model/roi_align/modules/roi_align.py
================================================
from torch.nn.modules.module import Module
from torch.nn.functional import avg_pool2d, max_pool2d
from ..functions.roi_align import RoIAlignFunction
class RoIAlign(Module):
def __init__(self, aligned_height, aligned_width, spatial_scale):
super(RoIAlign, self).__init__()
self.aligned_width = int(aligned_width)
self.aligned_height = int(aligned_height)
self.spatial_scale = float(spatial_scale)
def forward(self, features, rois):
return RoIAlignFunction(self.aligned_height, self.aligned_width,
self.spatial_scale)(features, rois)
class RoIAlignAvg(Module):
def __init__(self, aligned_height, aligned_width, spatial_scale):
super(RoIAlignAvg, self).__init__()
self.aligned_width = int(aligned_width)
self.aligned_height = int(aligned_height)
self.spatial_scale = float(spatial_scale)
def forward(self, features, rois):
x = RoIAlignFunction(self.aligned_height+1, self.aligned_width+1,
self.spatial_scale)(features, rois)
return avg_pool2d(x, kernel_size=2, stride=1)
class RoIAlignMax(Module):
def __init__(self, aligned_height, aligned_width, spatial_scale):
super(RoIAlignMax, self).__init__()
self.aligned_width = int(aligned_width)
self.aligned_height = int(aligned_height)
self.spatial_scale = float(spatial_scale)
def forward(self, features, rois):
x = RoIAlignFunction(self.aligned_height+1, self.aligned_width+1,
self.spatial_scale)(features, rois)
return max_pool2d(x, kernel_size=2, stride=1)
================================================
FILE: lib/model/roi_align/src/roi_align_cuda.c
================================================
#include <THC/THC.h>
#include <math.h>
#include "roi_align_kernel.h"
extern THCState *state;
int roi_align_forward_cuda(int aligned_height, int aligned_width, float spatial_scale,
THCudaTensor * features, THCudaTensor * rois, THCudaTensor * output)
{
// Grab the input tensor
float * data_flat = THCudaTensor_data(state, features);
float * rois_flat = THCudaTensor_data(state, rois);
float * output_flat = THCudaTensor_data(state, output);
// Number of ROIs
int num_rois = THCudaTensor_size(state, rois, 0);
int size_rois = THCudaTensor_size(state, rois, 1);
if (size_rois != 5)
{
return 0;
}
// data height
int data_height = THCudaTensor_size(state, features, 2);
// data width
int data_width = THCudaTensor_size(state, features, 3);
// Number of channels
int num_channels = THCudaTensor_size(state, features, 1);
cudaStream_t stream = THCState_getCurrentStream(state);
ROIAlignForwardLaucher(
data_flat, spatial_scale, num_rois, data_height,
data_width, num_channels, aligned_height,
aligned_width, rois_flat,
output_flat, stream);
return 1;
}
int roi_align_backward_cuda(int aligned_height, int aligned_width, float spatial_scale,
THCudaTensor * top_grad, THCudaTensor * rois, THCudaTensor * bottom_grad)
{
// Grab the input tensor
float * top_grad_flat = THCudaTensor_data(state, top_grad);
float * rois_flat = THCudaTensor_data(state, rois);
float * bottom_grad_flat = THCudaTensor_data(state, bottom_grad);
// Number of ROIs
int num_rois = THCudaTensor_size(state, rois, 0);
int size_rois = THCudaTensor_size(state, rois, 1);
if (size_rois != 5)
{
return 0;
}
// batch size
int batch_size = THCudaTensor_size(state, bottom_grad, 0);
// data height
int data_height = THCudaTensor_size(state, bottom_grad, 2);
// data width
int data_width = THCudaTensor_size(state, bottom_grad, 3);
// Number of channels
int num_channels = THCudaTensor_size(state, bottom_grad, 1);
cudaStream_t stream = THCState_getCurrentStream(state);
ROIAlignBackwardLaucher(
top_grad_flat, spatial_scale, batch_size, num_rois, data_height,
data_width, num_channels, aligned_height,
aligned_width, rois_flat,
bottom_grad_flat, stream);
return 1;
}
================================================
FILE: lib/model/roi_align/src/roi_align_cuda.h
================================================
int roi_align_forward_cuda(int aligned_height, int aligned_width, float spatial_scale,
THCudaTensor * features, THCudaTensor * rois, THCudaTensor * output);
int roi_align_backward_cuda(int aligned_height, int aligned_width, float spatial_scale,
THCudaTensor * top_grad, THCudaTensor * rois, THCudaTensor * bottom_grad);
================================================
FILE: lib/model/roi_align/src/roi_align_kernel.cu
================================================
#ifdef __cplusplus
extern "C" {
#endif
#include <stdio.h>
#include <math.h>
#include <float.h>
#include "roi_align_kernel.h"
#define CUDA_1D_KERNEL_LOOP(i, n) \
for (int i = blockIdx.x * blockDim.x + threadIdx.x; i < n; \
i += blockDim.x * gridDim.x)
__global__ void ROIAlignForward(const int nthreads, const float* bottom_data, const float spatial_scale, const int height, const int width,
const int channels, const int aligned_height, const int aligned_width, const float* bottom_rois, float* top_data) {
CUDA_1D_KERNEL_LOOP(index, nthreads) {
// (n, c, ph, pw) is an element in the aligned output
// int n = index;
// int pw = n % aligned_width;
// n /= aligned_width;
// int ph = n % aligned_height;
// n /= aligned_height;
// int c = n % channels;
// n /= channels;
int pw = index % aligned_width;
int ph = (index / aligned_width) % aligned_height;
int c = (index / aligned_width / aligned_height) % channels;
int n = index / aligned_width / aligned_height / channels;
// bottom_rois += n * 5;
float roi_batch_ind = bottom_rois[n * 5 + 0];
float roi_start_w = bottom_rois[n * 5 + 1] * spatial_scale;
float roi_start_h = bottom_rois[n * 5 + 2] * spatial_scale;
float roi_end_w = bottom_rois[n * 5 + 3] * spatial_scale;
float roi_end_h = bottom_rois[n * 5 + 4] * spatial_scale;
// Force malformed ROIs to be 1x1
float roi_width = fmaxf(roi_end_w - roi_start_w + 1., 0.);
float roi_height = fmaxf(roi_end_h - roi_start_h + 1., 0.);
float bin_size_h = roi_height / (aligned_height - 1.);
float bin_size_w = roi_width / (aligned_width - 1.);
float h = (float)(ph) * bin_size_h + roi_start_h;
float w = (float)(pw) * bin_size_w + roi_start_w;
int hstart = fminf(floor(h), height - 2);
int wstart = fminf(floor(w), width - 2);
int img_start = roi_batch_ind * channels * height * width;
// bilinear interpolation
if (h < 0 || h >= height || w < 0 || w >= width) {
top_data[index] = 0.;
} else {
float h_ratio = h - (float)(hstart);
float w_ratio = w - (float)(wstart);
int upleft = img_start + (c * height + hstart) * width + wstart;
int upright = upleft + 1;
int downleft = upleft + width;
int downright = downleft + 1;
top_data[index] = bottom_data[upleft] * (1. - h_ratio) * (1. - w_ratio)
+ bottom_data[upright] * (1. - h_ratio) * w_ratio
+ bottom_data[downleft] * h_ratio * (1. - w_ratio)
+ bottom_data[downright] * h_ratio * w_ratio;
}
}
}
int ROIAlignForwardLaucher(const float* bottom_data, const float spatial_scale, const int num_rois, const int height, const int width,
const int channels, const int aligned_height, const int aligned_width, const float* bottom_rois, float* top_data, cudaStream_t stream) {
const int kThreadsPerBlock = 1024;
const int output_size = num_rois * aligned_height * aligned_width * channels;
cudaError_t err;
ROIAlignForward<<<(output_size + kThreadsPerBlock - 1) / kThreadsPerBlock, kThreadsPerBlock, 0, stream>>>(
output_size, bottom_data, spatial_scale, height, width, channels,
aligned_height, aligned_width, bottom_rois, top_data);
err = cudaGetLastError();
if(cudaSuccess != err) {
fprintf( stderr, "cudaCheckError() failed : %s\n", cudaGetErrorString( err ) );
exit( -1 );
}
return 1;
}
__global__ void ROIAlignBackward(const int nthreads, const float* top_diff, const float spatial_scale, const int height, const int width,
const int channels, const int aligned_height, const int aligned_width, float* bottom_diff, const float* bottom_rois) {
CUDA_1D_KERNEL_LOOP(index, nthreads) {
// (n, c, ph, pw) is an element in the aligned output
int pw = index % aligned_width;
int ph = (index / aligned_width) % aligned_height;
int c = (index / aligned_width / aligned_height) % channels;
int n = index / aligned_width / aligned_height / channels;
float roi_batch_ind = bottom_rois[n * 5 + 0];
float roi_start_w = bottom_rois[n * 5 + 1] * spatial_scale;
float roi_start_h = bottom_rois[n * 5 + 2] * spatial_scale;
float roi_end_w = bottom_rois[n * 5 + 3] * spatial_scale;
float roi_end_h = bottom_rois[n * 5 + 4] * spatial_scale;
/* int roi_start_w = round(bottom_rois[1] * spatial_scale); */
/* int roi_start_h = round(bottom_rois[2] * spatial_scale); */
/* int roi_end_w = round(bottom_rois[3] * spatial_scale); */
/* int roi_end_h = round(bottom_rois[4] * spatial_scale); */
// Force malformed ROIs to be 1x1
float roi_width = fmaxf(roi_end_w - roi_start_w + 1., 0.);
float roi_height = fmaxf(roi_end_h - roi_start_h + 1., 0.);
float bin_size_h = roi_height / (aligned_height - 1.);
float bin_size_w = roi_width / (aligned_width - 1.);
float h = (float)(ph) * bin_size_h + roi_start_h;
float w = (float)(pw) * bin_size_w + roi_start_w;
int hstart = fminf(floor(h), height - 2);
int wstart = fminf(floor(w), width - 2);
int img_start = roi_batch_ind * channels * height * width;
// bilinear interpolation
if (!(h < 0 || h >= height || w < 0 || w >= width)) {
float h_ratio = h - (float)(hstart);
float w_ratio = w - (float)(wstart);
int upleft = img_start + (c * height + hstart) * width + wstart;
int upright = upleft + 1;
int downleft = upleft + width;
int downright = downleft + 1;
atomicAdd(bottom_diff + upleft, top_diff[index] * (1. - h_ratio) * (1 - w_ratio));
atomicAdd(bottom_diff + upright, top_diff[index] * (1. - h_ratio) * w_ratio);
atomicAdd(bottom_diff + downleft, top_diff[index] * h_ratio * (1 - w_ratio));
atomicAdd(bottom_diff + downright, top_diff[index] * h_ratio * w_ratio);
}
}
}
int ROIAlignBackwardLaucher(const float* top_diff, const float spatial_scale, const int batch_size, const int num_rois, const int height, const int width,
const int channels, const int aligned_height, const int aligned_width, const float* bottom_rois, float* bottom_diff, cudaStream_t stream) {
const int kThreadsPerBlock = 1024;
const int output_size = num_rois * aligned_height * aligned_width * channels;
cudaError_t err;
ROIAlignBackward<<<(output_size + kThreadsPerBlock - 1) / kThreadsPerBlock, kThreadsPerBlock, 0, stream>>>(
output_size, top_diff, spatial_scale, height, width, channels,
aligned_height, aligned_width, bottom_diff, bottom_rois);
err = cudaGetLastError();
if(cudaSuccess != err) {
fprintf( stderr, "cudaCheckError() failed : %s\n", cudaGetErrorString( err ) );
exit( -1 );
}
return 1;
}
#ifdef __cplusplus
}
#endif
================================================
FILE: lib/model/roi_align/src/roi_align_kernel.h
================================================
#ifndef _ROI_ALIGN_KERNEL
#define _ROI_ALIGN_KERNEL
#ifdef __cplusplus
extern "C" {
#endif
__global__ void ROIAlignForward(const int nthreads, const float* bottom_data,
const float spatial_scale, const int height, const int width,
const int channels, const int aligned_height, const int aligned_width,
const float* bottom_rois, float* top_data);
int ROIAlignForwardLaucher(
const float* bottom_data, const float spatial_scale, const int num_rois, const int height,
const int width, const int channels, const int aligned_height,
const int aligned_width, const float* bottom_rois,
float* top_data, cudaStream_t stream);
__global__ void ROIAlignBackward(const int nthreads, const float* top_diff,
const float spatial_scale, const int height, const int width,
const int channels, const int aligned_height, const int aligned_width,
float* bottom_diff, const float* bottom_rois);
int ROIAlignBackwardLaucher(const float* top_diff, const float spatial_scale, const int batch_size, const int num_rois,
const int height, const int width, const int channels, const int aligned_height,
const int aligned_width, const float* bottom_rois,
float* bottom_diff, cudaStream_t stream);
#ifdef __cplusplus
}
#endif
#endif
================================================
FILE: lib/model/roi_crop/__init__.py
================================================
================================================
FILE: lib/model/roi_crop/_ext/__init__.py
================================================
================================================
FILE: lib/model/roi_crop/_ext/roi_crop/__init__.py
================================================
from torch.utils.ffi import _wrap_function
from ._roi_crop import lib as _lib, ffi as _ffi
__all__ = []
def _import_symbols(locals):
for symbol in dir(_lib):
fn = getattr(_lib, symbol)
if callable(fn):
locals[symbol] = _wrap_function(fn, _ffi)
else:
locals[symbol] = fn
__all__.append(symbol)
_import_symbols(locals())
================================================
FILE: lib/model/roi_crop/build.py
================================================
from __future__ import print_function
import os
import torch
from torch.utils.ffi import create_extension
#this_file = os.path.dirname(__file__)
sources = ['src/roi_crop.c']
headers = ['src/roi_crop.h']
defines = []
with_cuda = False
if torch.cuda.is_available():
print('Including CUDA code.')
sources += ['src/roi_crop_cuda.c']
headers += ['src/roi_crop_cuda.h']
defines += [('WITH_CUDA', None)]
with_cuda = True
this_file = os.path.dirname(os.path.realpath(__file__))
print(this_file)
extra_objects = ['src/roi_crop_cuda_kernel.cu.o']
extra_objects = [os.path.join(this_file, fname) for fname in extra_objects]
ffi = create_extension(
'_ext.roi_crop',
headers=headers,
sources=sources,
define_macros=defines,
relative_to=__file__,
with_cuda=with_cuda,
extra_objects=extra_objects
)
if __name__ == '__main__':
ffi.build()
================================================
FILE: lib/model/roi_crop/functions/__init__.py
================================================
================================================
FILE: lib/model/roi_crop/functions/crop_resize.py
================================================
# functions/add.py
import torch
from torch.autograd import Function
from .._ext import roi_crop
from cffi import FFI
ffi = FFI()
class RoICropFunction(Function):
def forward(self, input1, input2):
self.input1 = input1
self.input2 = input2
self.device_c = ffi.new("int *")
output = torch.zeros(input2.size()[0], input1.size()[1], input2.size()[1], input2.size()[2])
#print('decice %d' % torch.cuda.current_device())
if input1.is_cuda:
self.device = torch.cuda.current_device()
else:
self.device = -1
self.device_c[0] = self.device
if not input1.is_cuda:
roi_crop.BilinearSamplerBHWD_updateOutput(input1, input2, output)
else:
output = output.cuda(self.device)
roi_crop.BilinearSamplerBHWD_updateOutput_cuda(input1, input2, output)
return output
def backward(self, grad_output):
grad_input1 = torch.zeros(self.input1.size())
grad_input2 = torch.zeros(self.input2.size())
#print('backward decice %d' % self.device)
if not grad_output.is_cuda:
roi_crop.BilinearSamplerBHWD_updateGradInput(self.input1, self.input2, grad_input1, grad_input2, grad_output)
else:
grad_input1 = grad_input1.cuda(self.device)
grad_input2 = grad_input2.cuda(self.device)
roi_crop.BilinearSamplerBHWD_updateGradInput_cuda(self.input1, self.input2, grad_input1, grad_input2, grad_output)
return grad_input1, grad_input2
================================================
FILE: lib/model/roi_crop/functions/gridgen.py
================================================
# functions/add.py
import torch
from torch.autograd import Function
import numpy as np
class AffineGridGenFunction(Function):
def __init__(self, height, width,lr=1):
super(AffineGridGenFunction, self).__init__()
self.lr = lr
self.height, self.width = height, width
self.grid = np.zeros( [self.height, self.width, 3], dtype=np.float32)
self.grid[:,:,0] = np.expand_dims(np.repeat(np.expand_dims(np.arange(-1, 1, 2.0/(self.height)), 0), repeats = self.width, axis = 0).T, 0)
self.grid[:,:,1] = np.expand_dims(np.repeat(np.expand_dims(np.arange(-1, 1, 2.0/(self.width)), 0), repeats = self.height, axis = 0), 0)
# self.grid[:,:,0] = np.expand_dims(np.repeat(np.expand_dims(np.arange(-1, 1, 2.0/(self.height - 1)), 0), repeats = self.width, axis = 0).T, 0)
# self.grid[:,:,1] = np.expand_dims(np.repeat(np.expand_dims(np.arange(-1, 1, 2.0/(self.width - 1)), 0), repeats = self.height, axis = 0), 0)
self.grid[:,:,2] = np.ones([self.height, width])
self.grid = torch.from_numpy(self.grid.astype(np.float32))
#print(self.grid)
def forward(self, input1):
self.input1 = input1
output = input1.new(torch.Size([input1.size(0)]) + self.grid.size()).zero_()
self.batchgrid = input1.new(torch.Size([input1.size(0)]) + self.grid.size()).zero_()
for i in range(input1.size(0)):
self.batchgrid[i] = self.grid.astype(self.batchgrid[i])
# if input1.is_cuda:
# self.batchgrid = self.batchgrid.cuda()
# output = output.cuda()
for i in range(input1.size(0)):
output = torch.bmm(self.batchgrid.view(-1, self.height*self.width, 3), torch.transpose(input1, 1, 2)).view(-1, self.height, self.width, 2)
return output
def backward(self, grad_output):
grad_input1 = self.input1.new(self.input1.size()).zero_()
# if grad_output.is_cuda:
# self.batchgrid = self.batchgrid.cuda()
# grad_input1 = grad_input1.cuda()
grad_input1 = torch.baddbmm(grad_input1, torch.transpose(grad_output.view(-1, self.height*self.width, 2), 1,2), self.batchgrid.view(-1, self.height*self.width, 3))
return grad_input1
================================================
FILE: lib/model/roi_crop/functions/roi_crop.py
================================================
# functions/add.py
import torch
from torch.autograd import Function
from .._ext import roi_crop
import pdb
class RoICropFunction(Function):
def forward(self, input1, input2):
self.input1 = input1.clone()
self.input2 = input2.clone()
output = input2.new(input2.size()[0], input1.size()[1], input2.size()[1], input2.size()[2]).zero_()
assert output.get_device() == input1.get_device(), "output and input1 must on the same device"
assert output.get_device() == input2.get_device(), "output and input2 must on the same device"
roi_crop.BilinearSamplerBHWD_updateOutput_cuda(input1, input2, output)
return output
def backward(self, grad_output):
grad_input1 = self.input1.new(self.input1.size()).zero_()
grad_input2 = self.input2.new(self.input2.size()).zero_()
roi_crop.BilinearSamplerBHWD_updateGradInput_cuda(self.input1, self.input2, grad_input1, grad_input2, grad_output)
return grad_input1, grad_input2
================================================
FILE: lib/model/roi_crop/modules/__init__.py
================================================
================================================
FILE: lib/model/roi_crop/modules/gridgen.py
================================================
from torch.nn.modules.module import Module
import torch
from torch.autograd import Variable
import numpy as np
from ..functions.gridgen import AffineGridGenFunction
import pyximport
pyximport.install(setup_args={"include_dirs":np.get_include()},
reload_support=True)
class _AffineGridGen(Module):
def __init__(self, height, width, lr = 1, aux_loss = False):
super(_AffineGridGen, self).__init__()
self.height, self.width = height, width
self.aux_loss = aux_loss
self.f = AffineGridGenFunction(self.height, self.width, lr=lr)
self.lr = lr
def forward(self, input):
# if not self.aux_loss:
return self.f(input)
# else:
# identity = torch.from_numpy(np.array([[1,0,0], [0,1,0]], dtype=np.float32))
# batch_identity = torch.zeros([input.size(0), 2,3])
# for i in range(input.size(0)):
# batch_identity[i] = identity
# batch_identity = Variable(batch_identity)
# loss = torch.mul(input - batch_identity, input - batch_identity)
# loss = torch.sum(loss,1)
# loss = torch.sum(loss,2)
# return self.f(input), loss.view(-1,1)
class CylinderGridGen(Module):
def __init__(self, height, width, lr = 1, aux_loss = False):
super(CylinderGridGen, self).__init__()
self.height, self.width = height, width
self.aux_loss = aux_loss
self.f = CylinderGridGenFunction(self.height, self.width, lr=lr)
self.lr = lr
def forward(self, input):
if not self.aux_loss:
return self.f(input)
else:
return self.f(input), torch.mul(input, input).view(-1,1)
class AffineGridGenV2(Module):
def __init__(self, height, width, lr = 1, aux_loss = False):
super(AffineGridGenV2, self).__init__()
self.height, self.width = height, width
self.aux_loss = aux_loss
self.lr = lr
self.grid = np.zeros( [self.height, self.width, 3], dtype=np.float32)
self.grid[:,:,0] = np.expand_dims(np.repeat(np.expand_dims(np.arange(-1, 1, 2.0/self.height), 0), repeats = self.width, axis = 0).T, 0)
self.grid[:,:,1] = np.expand_dims(np.repeat(np.expand_dims(np.arange(-1, 1, 2.0/self.width), 0), repeats = self.height, axis = 0), 0)
self.grid[:,:,2] = np.ones([self.height, width])
self.grid = torch.from_numpy(self.grid.astype(np.float32))
def forward(self, input1):
self.batchgrid = torch.zeros(torch.Size([input1.size(0)]) + self.grid.size())
for i in range(input1.size(0)):
self.batchgrid[i] = self.grid
self.batchgrid = Variable(self.batchgrid)
if input1.is_cuda:
self.batchgrid = self.batchgrid.cuda()
output = torch.bmm(self.batchgrid.view(-1, self.height*self.width, 3), torch.transpose(input1, 1, 2)).view(-1, self.height, self.width, 2)
return output
class CylinderGridGenV2(Module):
def __init__(self, height, width, lr = 1):
super(CylinderGridGenV2, self).__init__()
self.height, self.width = height, width
self.lr = lr
self.grid = np.zeros( [self.height, self.width, 3], dtype=np.float32)
self.grid[:,:,0] = np.expand_dims(np.repeat(np.expand_dims(np.arange(-1, 1, 2.0/self.height), 0), repeats = self.width, axis = 0).T, 0)
self.grid[:,:,1] = np.expand_dims(np.repeat(np.expand_dims(np.arange(-1, 1, 2.0/self.width), 0), repeats = self.height, axis = 0), 0)
self.grid[:,:,2] = np.ones([self.height, width])
self.grid = torch.from_numpy(self.grid.astype(np.float32))
def forward(self, input):
self.batchgrid = torch.zeros(torch.Size([input.size(0)]) + self.grid.size() )
#print(self.batchgrid.size())
for i in range(input.size(0)):
self.batchgrid[i,:,:,:] = self.grid
self.batchgrid = Variable(self.batchgrid)
#print(self.batchgrid.size())
input_u = input.view(-1,1,1,1).repeat(1,self.height, self.width,1)
#print(input_u.requires_grad, self.batchgrid)
output0 = self.batchgrid[:,:,:,0:1]
output1 = torch.atan(torch.tan(np.pi/2.0*(self.batchgrid[:,:,:,1:2] + self.batchgrid[:,:,:,2:] * input_u[:,:,:,:]))) /(np.pi/2)
#print(output0.size(), output1.size())
output = torch.cat([output0, output1], 3)
return output
class DenseAffineGridGen(Module):
def __init__(self, height, width, lr = 1, aux_loss = False):
super(DenseAffineGridGen, self).__init__()
self.height, self.width = height, width
self.aux_loss = aux_loss
self.lr = lr
self.grid = np.zeros( [self.height, self.width, 3], dtype=np.float32)
self.grid[:,:,0] = np.expand_dims(np.repeat(np.expand_dims(np.arange(-1, 1, 2.0/self.height), 0), repeats = self.width, axis = 0).T, 0)
self.grid[:,:,1] = np.expand_dims(np.repeat(np.expand_dims(np.arange(-1, 1, 2.0/self.width), 0), repeats = self.height, axis = 0), 0)
self.grid[:,:,2] = np.ones([self.height, width])
self.grid = torch.from_numpy(self.grid.astype(np.float32))
def forward(self, input1):
self.batchgrid = torch.zeros(torch.Size([input1.size(0)]) + self.grid.size())
for i in range(input1.size(0)):
self.batchgrid[i] = self.grid
self.batchgrid = Variable(self.batchgrid)
#print self.batchgrid, input1[:,:,:,0:3]
#print self.batchgrid, input1[:,:,:,4:6]
x = torch.mul(self.batchgrid, input1[:,:,:,0:3])
y = torch.mul(self.batchgrid, input1[:,:,:,3:6])
output = torch.cat([torch.sum(x,3),torch.sum(y,3)], 3)
return output
class DenseAffine3DGridGen(Module):
def __init__(self, height, width, lr = 1, aux_loss = False):
super(DenseAffine3DGridGen, self).__init__()
self.height, self.width = height, width
self.aux_loss = aux_loss
self.lr = lr
self.grid = np.zeros( [self.height, self.width, 3], dtype=np.float32)
self.grid[:,:,0] = np.expand_dims(np.repeat(np.expand_dims(np.arange(-1, 1, 2.0/self.height), 0), repeats = self.width, axis = 0).T, 0)
self.grid[:,:,1] = np.expand_dims(np.repeat(np.expand_dims(np.arange(-1, 1, 2.0/self.width), 0), repeats = self.height, axis = 0), 0)
self.grid[:,:,2] = np.ones([self.height, width])
self.grid = torch.from_numpy(self.grid.astype(np.float32))
self.theta = self.grid[:,:,0] * np.pi/2 + np.pi/2
self.phi = self.grid[:,:,1] * np.pi
self.x = torch.sin(self.theta) * torch.cos(self.phi)
self.y = torch.sin(self.theta) * torch.sin(self.phi)
self.z = torch.cos(self.theta)
self.grid3d = torch.from_numpy(np.zeros( [self.height, self.width, 4], dtype=np.float32))
self.grid3d[:,:,0] = self.x
self.grid3d[:,:,1] = self.y
self.grid3d[:,:,2] = self.z
self.grid3d[:,:,3] = self.grid[:,:,2]
def forward(self, input1):
self.batchgrid3d = torch.zeros(torch.Size([input1.size(0)]) + self.grid3d.size())
for i in range(input1.size(0)):
self.batchgrid3d[i] = self.grid3d
self.batchgrid3d = Variable(self.batchgrid3d)
#print(self.batchgrid3d)
x = torch.sum(torch.mul(self.batchgrid3d, input1[:,:,:,0:4]), 3)
y = torch.sum(torch.mul(self.batchgrid3d, input1[:,:,:,4:8]), 3)
z = torch.sum(torch.mul(self.batchgrid3d, input1[:,:,:,8:]), 3)
#print(x)
r = torch.sqrt(x**2 + y**2 + z**2) + 1e-5
#print(r)
theta = torch.acos(z/r)/(np.pi/2) - 1
#phi = torch.atan(y/x)
phi = torch.atan(y/(x + 1e-5)) + np.pi * x.lt(0).type(torch.FloatTensor) * (y.ge(0).type(torch.FloatTensor) - y.lt(0).type(torch.FloatTensor))
phi = phi/np.pi
output = torch.cat([theta,phi], 3)
return output
class DenseAffine3DGridGen_rotate(Module):
def __init__(self, height, width, lr = 1, aux_loss = False):
super(DenseAffine3DGridGen_rotate, self).__init__()
self.height, self.width = height, width
self.aux_loss = aux_loss
self.lr = lr
self.grid = np.zeros( [self.height, self.width, 3], dtype=np.float32)
self.grid[:,:,0] = np.expand_dims(np.repeat(np.expand_dims(np.arange(-1, 1, 2.0/self.height), 0), repeats = self.width, axis = 0).T, 0)
self.grid[:,:,1] = np.expand_dims(np.repeat(np.expand_dims(np.arange(-1, 1, 2.0/self.width), 0), repeats = self.height, axis = 0), 0)
self.grid[:,:,2] = np.ones([self.height, width])
self.grid = torch.from_numpy(self.grid.astype(np.float32))
self.theta = self.grid[:,:,0] * np.pi/2 + np.pi/2
self.phi = self.grid[:,:,1] * np.pi
self.x = torch.sin(self.theta) * torch.cos(self.phi)
self.y = torch.sin(self.theta) * torch.sin(self.phi)
self.z = torch.cos(self.theta)
self.grid3d = torch.from_numpy(np.zeros( [self.height, self.width, 4], dtype=np.float32))
self.grid3d[:,:,0] = self.x
self.grid3d[:,:,1] = self.y
self.grid3d[:,:,2] = self.z
self.grid3d[:,:,3] = self.grid[:,:,2]
def forward(self, input1, input2):
self.batchgrid3d = torch.zeros(torch.Size([input1.size(0)]) + self.grid3d.size())
for i in range(input1.size(0)):
self.batchgrid3d[i] = self.grid3d
self.batchgrid3d = Variable(self.batchgrid3d)
self.batchgrid = torch.zeros(torch.Size([input1.size(0)]) + self.grid.size())
for i in range(input1.size(0)):
self.batchgrid[i] = self.grid
self.batchgrid = Variable(self.batchgrid)
#print(self.batchgrid3d)
x = torch.sum(torch.mul(self.batchgrid3d, input1[:,:,:,0:4]), 3)
y = torch.sum(torch.mul(self.batchgrid3d, input1[:,:,:,4:8]), 3)
z = torch.sum(torch.mul(self.batchgrid3d, input1[:,:,:,8:]), 3)
#print(x)
r = torch.sqrt(x**2 + y**2 + z**2) + 1e-5
#print(r)
theta = torch.acos(z/r)/(np.pi/2) - 1
#phi = torch.atan(y/x)
phi = torch.atan(y/(x + 1e-5)) + np.pi * x.lt(0).type(torch.FloatTensor) * (y.ge(0).type(torch.FloatTensor) - y.lt(0).type(torch.FloatTensor))
phi = phi/np.pi
input_u = input2.view(-1,1,1,1).repeat(1,self.height, self.width,1)
output = torch.cat([theta,phi], 3)
output1 = torch.atan(torch.tan(np.pi/2.0*(output[:,:,:,1:2] + self.batchgrid[:,:,:,2:] * input_u[:,:,:,:]))) /(np.pi/2)
output2 = torch.cat([output[:,:,:,0:1], output1], 3)
return output2
class Depth3DGridGen(Module):
def __init__(self, height, width, lr = 1, aux_loss = False):
super(Depth3DGridGen, self).__init__()
self.height, self.width = height, width
self.aux_loss = aux_loss
self.lr = lr
self.grid = np.zeros( [self.height, self.width, 3], dtype=np.float32)
self.grid[:,:,0] = np.expand_dims(np.repeat(np.expand_dims(np.arange(-1, 1, 2.0/self.height), 0), repeats = self.width, axis = 0).T, 0)
self.grid[:,:,1] = np.expand_dims(np.repeat(np.expand_dims(np.arange(-1, 1, 2.0/self.width), 0), repeats = self.height, axis = 0), 0)
self.grid[:,:,2] = np.ones([self.height, width])
self.grid = torch.from_numpy(self.grid.astype(np.float32))
self.theta = self.grid[:,:,0] * np.pi/2 + np.pi/2
self.phi = self.grid[:,:,1] * np.pi
self.x = torch.sin(self.theta) * torch.cos(self.phi)
self.y = torch.sin(self.theta) * torch.sin(self.phi)
self.z = torch.cos(self.theta)
self.grid3d = torch.from_numpy(np.zeros( [self.height, self.width, 4], dtype=np.float32))
self.grid3d[:,:,0] = self.x
self.grid3d[:,:,1] = self.y
self.grid3d[:,:,2] = self.z
self.grid3d[:,:,3] = self.grid[:,:,2]
def forward(self, depth, trans0, trans1, rotate):
self.batchgrid3d = torch.zeros(torch.Size([depth.size(0)]) + self.grid3d.size())
for i in range(depth.size(0)):
self.batchgrid3d[i] = self.grid3d
self.batchgrid3d = Variable(self.batchgrid3d)
self.batchgrid = torch.zeros(torch.Size([depth.size(0)]) + self.grid.size())
for i in range(depth.size(0)):
self.batchgrid[i] = self.grid
self.batchgrid = Variable(self.batchgrid)
x = self.batchgrid3d[:,:,:,0:1] * depth + trans0.view(-1,1,1,1).repeat(1, self.height, self.width, 1)
y = self.batchgrid3d[:,:,:,1:2] * depth + trans1.view(-1,1,1,1).repeat(1, self.height, self.width, 1)
z = self.batchgrid3d[:,:,:,2:3] * depth
#print(x.size(), y.size(), z.size())
r = torch.sqrt(x**2 + y**2 + z**2) + 1e-5
#print(r)
theta = torch.acos(z/r)/(np.pi/2) - 1
#phi = torch.atan(y/x)
phi = torch.atan(y/(x + 1e-5)) + np.pi * x.lt(0).type(torch.FloatTensor) * (y.ge(0).type(torch.FloatTensor) - y.lt(0).type(torch.FloatTensor))
phi = phi/np.pi
#print(theta.size(), phi.size())
input_u = rotate.view(-1,1,1,1).repeat(1,self.height, self.width,1)
output = torch.cat([theta,phi], 3)
#print(output.size())
output1 = torch.atan(torch.tan(np.pi/2.0*(output[:,:,:,1:2] + self.batchgrid[:,:,:,2:] * input_u[:,:,:,:]))) /(np.pi/2)
output2 = torch.cat([output[:,:,:,0:1], output1], 3)
return output2
class Depth3DGridGen_with_mask(Module):
def __init__(self, height, width, lr = 1, aux_loss = False, ray_tracing = False):
super(Depth3DGridGen_with_mask, self).__init__()
self.height, self.width = height, width
self.aux_loss = aux_loss
self.lr = lr
self.ray_tracing = ray_tracing
self.grid = np.zeros( [self.height, self.width, 3], dtype=np.float32)
self.grid[:,:,0] = np.expand_dims(np.repeat(np.expand_dims(np.arange(-1, 1, 2.0/self.height), 0), repeats = self.width, axis = 0).T, 0)
self.grid[:,:,1] = np.expand_dims(np.repeat(np.expand_dims(np.arange(-1, 1, 2.0/self.width), 0), repeats = self.height, axis = 0), 0)
self.grid[:,:,2] = np.ones([self.height, width])
self.grid = torch.from_numpy(self.grid.astype(np.float32))
self.theta = self.grid[:,:,0] * np.pi/2 + np.pi/2
self.phi = self.grid[:,:,1] * np.pi
self.x = torch.sin(self.theta) * torch.cos(self.phi)
self.y = torch.sin(self.theta) * torch.sin(self.phi)
self.z = torch.cos(self.theta)
self.grid3d = torch.from_numpy(np.zeros( [self.height, self.width, 4], dtype=np.float32))
self.grid3d[:,:,0] = self.x
self.grid3d[:,:,1] = self.y
self.grid3d[:,:,2] = self.z
self.grid3d[:,:,3] = self.grid[:,:,2]
def forward(self, depth, trans0, trans1, rotate):
self.batchgrid3d = torch.zeros(torch.Size([depth.size(0)]) + self.grid3d.size())
for i in range(depth.size(0)):
self.batchgrid3d[i] = self.grid3d
self.batchgrid3d = Variable(self.batchgrid3d)
self.batchgrid = torch.zeros(torch.Size([depth.size(0)]) + self.grid.size())
for i in range(depth.size(0)):
self.batchgrid[i] = self.grid
self.batchgrid = Variable(self.batchgrid)
if depth.is_cuda:
self.batchgrid = self.batchgrid.cuda()
self.batchgrid3d = self.batchgrid3d.cuda()
x_ = self.batchgrid3d[:,:,:,0:1] * depth + trans0.view(-1,1,1,1).repeat(1, self.height, self.width, 1)
y_ = self.batchgrid3d[:,:,:,1:2] * depth + trans1.view(-1,1,1,1).repeat(1, self.height, self.width, 1)
z = self.batchgrid3d[:,:,:,2:3] * depth
#print(x.size(), y.size(), z.size())
rotate_z = rotate.view(-1,1,1,1).repeat(1,self.height, self.width,1) * np.pi
x = x_ * torch.cos(rotate_z) - y_ * torch.sin(rotate_z)
y = x_ * torch.sin(rotate_z) + y_ * torch.cos(rotate_z)
r = torch.sqrt(x**2 + y**2 + z**2) + 1e-5
#print(r)
theta = torch.acos(z/r)/(np.pi/2) - 1
#phi = torch.atan(y/x)
if depth.is_cuda:
phi = torch.atan(y/(x + 1e-5)) + np.pi * x.lt(0).type(torch.cuda.FloatTensor) * (y.ge(0).type(torch.cuda.FloatTensor) - y.lt(0).type(torch.cuda.FloatTensor))
else:
phi = torch.atan(y/(x + 1e-5)) + np.pi * x.lt(0).type(torch.FloatTensor) * (y.ge(0).type(torch.FloatTensor) - y.lt(0).type(torch.FloatTensor))
phi = phi/np.pi
output = torch.cat([theta,phi], 3)
return output
================================================
FILE: lib/model/roi_crop/modules/roi_crop.py
================================================
from torch.nn.modules.module import Module
from ..functions.roi_crop import RoICropFunction
class _RoICrop(Module):
def __init__(self, layout = 'BHWD'):
super(_RoICrop, self).__init__()
def forward(self, input1, input2):
return RoICropFunction()(input1, input2)
================================================
FILE: lib/model/roi_crop/src/roi_crop.c
================================================
#include <TH/TH.h>
#include <stdbool.h>
#include <stdio.h>
#define real float
int BilinearSamplerBHWD_updateOutput(THFloatTensor *inputImages, THFloatTensor *grids, THFloatTensor *output)
{
int batchsize = inputImages->size[0];
int inputImages_height = inputImages->size[1];
int inputImages_width = inputImages->size[2];
int output_height = output->size[1];
int output_width = output->size[2];
int inputImages_channels = inputImages->size[3];
int output_strideBatch = output->stride[0];
int output_strideHeight = output->stride[1];
int output_strideWidth = output->stride[2];
int inputImages_strideBatch = inputImages->stride[0];
int inputImages_strideHeight = inputImages->stride[1];
int inputImages_strideWidth = inputImages->stride[2];
int grids_strideBatch = grids->stride[0];
int grids_strideHeight = grids->stride[1];
int grids_strideWidth = grids->stride[2];
real *inputImages_data, *output_data, *grids_data;
inputImages_data = THFloatTensor_data(inputImages);
output_data = THFloatTensor_data(output);
grids_data = THFloatTensor_data(grids);
int b, yOut, xOut;
for(b=0; b < batchsize; b++)
{
for(yOut=0; yOut < output_height; yOut++)
{
for(xOut=0; xOut < output_width; xOut++)
{
//read the grid
real yf = grids_data[b*grids_strideBatch + yOut*grids_strideHeight + xOut*grids_strideWidth];
real xf = grids_data[b*grids_strideBatch + yOut*grids_strideHeight + xOut*grids_strideWidth + 1];
// get the weights for interpolation
int yInTopLeft, xInTopLeft;
real yWeightTopLeft, xWeightTopLeft;
real xcoord = (xf + 1) * (inputImages_width - 1) / 2;
xInTopLeft = floor(xcoord);
xWeightTopLeft = 1 - (xcoord - xInTopLeft);
real ycoord = (yf + 1) * (inputImages_height - 1) / 2;
yInTopLeft = floor(ycoord);
yWeightTopLeft = 1 - (ycoord - yInTopLeft);
const int outAddress = output_strideBatch * b + output_strideHeight * yOut + output_strideWidth * xOut;
const int inTopLeftAddress = inputImages_strideBatch * b + inputImages_strideHeight * yInTopLeft + inputImages_strideWidth * xInTopLeft;
const int inTopRightAddress = inTopLeftAddress + inputImages_strideWidth;
const int inBottomLeftAddress = inTopLeftAddress + inputImages_strideHeight;
const int inBottomRightAddress = inBottomLeftAddress + inputImages_strideWidth;
real v=0;
real inTopLeft=0;
real inTopRight=0;
real inBottomLeft=0;
real inBottomRight=0;
// we are careful with the boundaries
bool topLeftIsIn = xInTopLeft >= 0 && xInTopLeft <= inputImages_width-1 && yInTopLeft >= 0 && yInTopLeft <= inputImages_height-1;
bool topRightIsIn = xInTopLeft+1 >= 0 && xInTopLeft+1 <= inputImages_width-1 && yInTopLeft >= 0 && yInTopLeft <= inputImages_height-1;
bool bottomLeftIsIn = xInTopLeft >= 0 && xInTopLeft <= inputImages_width-1 && yInTopLeft+1 >= 0 && yInTopLeft+1 <= inputImages_height-1;
bool bottomRightIsIn = xInTopLeft+1 >= 0 && xInTopLeft+1 <= inputImages_width-1 && yInTopLeft+1 >= 0 && yInTopLeft+1 <= inputImages_height-1;
int t;
// interpolation happens here
for(t=0; t<inputImages_channels; t++)
{
if(topLeftIsIn) inTopLeft = inputImages_data[inTopLeftAddress + t];
if(topRightIsIn) inTopRight = inputImages_data[inTopRightAddress + t];
if(bottomLeftIsIn) inBottomLeft = inputImages_data[inBottomLeftAddress + t];
if(bottomRightIsIn) inBottomRight = inputImages_data[inBottomRightAddress + t];
v = xWeightTopLeft * yWeightTopLeft * inTopLeft
+ (1 - xWeightTopLeft) * yWeightTopLeft * inTopRight
+ xWeightTopLeft * (1 - yWeightTopLeft) * inBottomLeft
+ (1 - xWeightTopLeft) * (1 - yWeightTopLeft) * inBottomRight;
output_data[outAddress + t] = v;
}
}
}
}
return 1;
}
int BilinearSamplerBHWD_updateGradInput(THFloatTensor *inputImages, THFloatTensor *grids, THFloatTensor *gradInputImages,
THFloatTensor *gradGrids, THFloatTensor *gradOutput)
{
bool onlyGrid=false;
int batchsize = inputImages->size[0];
int inputImages_height = inputImages->size[1];
int inputImages_width = inputImages->size[2];
int gradOutput_height = gradOutput->size[1];
int gradOutput_width = gradOutput->size[2];
int inputImages_channels = inputImages->size[3];
int gradOutput_strideBatch = gradOutput->stride[0];
int gradOutput_strideHeight = gradOutput->stride[1];
int gradOutput_strideWidth = gradOutput->stride[2];
int inputImages_strideBatch = inputImages->stride[0];
int inputImages_strideHeight = inputImages->stride[1];
int inputImages_strideWidth = inputImages->stride[2];
int gradInputImages_strideBatch = gradInputImages->stride[0];
int gradInputImages_strideHeight = gradInputImages->stride[1];
int gradInputImages_strideWidth = gradInputImages->stride[2];
int grids_strideBatch = grids->stride[0];
int grids_strideHeight = grids->stride[1];
int grids_strideWidth = grids->stride[2];
int gradGrids_strideBatch = gradGrids->stride[0];
int gradGrids_strideHeight = gradGrids->stride[1];
int gradGrids_strideWidth = gradGrids->stride[2];
real *inputImages_data, *gradOutput_data, *grids_data, *gradGrids_data, *gradInputImages_data;
inputImages_data = THFloatTensor_data(inputImages);
gradOutput_data = THFloatTensor_data(gradOutput);
grids_data = THFloatTensor_data(grids);
gradGrids_data = THFloatTensor_data(gradGrids);
gradInputImages_data = THFloatTensor_data(gradInputImages);
int b, yOut, xOut;
for(b=0; b < batchsize; b++)
{
for(yOut=0; yOut < gradOutput_height; yOut++)
{
for(xOut=0; xOut < gradOutput_width; xOut++)
{
//read the grid
real yf = grids_data[b*grids_strideBatch + yOut*grids_strideHeight + xOut*grids_strideWidth];
real xf = grids_data[b*grids_strideBatch + yOut*grids_strideHeight + xOut*grids_strideWidth + 1];
// get the weights for interpolation
int yInTopLeft, xInTopLeft;
real yWeightTopLeft, xWeightTopLeft;
real xcoord = (xf + 1) * (inputImages_width - 1) / 2;
xInTopLeft = floor(xcoord);
xWeightTopLeft = 1 - (xcoord - xInTopLeft);
real ycoord = (yf + 1) * (inputImages_height - 1) / 2;
yInTopLeft = floor(ycoord);
yWeightTopLeft = 1 - (ycoord - yInTopLeft);
const int inTopLeftAddress = inputImages_strideBatch * b + inputImages_strideHeight * yInTopLeft + inputImages_strideWidth * xInTopLeft;
const int inTopRightAddress = inTopLeftAddress + inputImages_strideWidth;
const int inBottomLeftAddress = inTopLeftAddress + inputImages_strideHeight;
const int inBottomRightAddress = inBottomLeftAddress + inputImages_strideWidth;
const int gradInputImagesTopLeftAddress = gradInputImages_strideBatch * b + gradInputImages_strideHeight * yInTopLeft + gradInputImages_strideWidth * xInTopLeft;
const int gradInputImagesTopRightAddress = gradInputImagesTopLeftAddress + gradInputImages_strideWidth;
const int gradInputImagesBottomLeftAddress = gradInputImagesTopLeftAddress + gradInputImages_strideHeight;
const int gradInputImagesBottomRightAddress = gradInputImagesBottomLeftAddress + gradInputImages_strideWidth;
const int gradOutputAddress = gradOutput_strideBatch * b + gradOutput_strideHeight * yOut + gradOutput_strideWidth * xOut;
real topLeftDotProduct = 0;
real topRightDotProduct = 0;
real bottomLeftDotProduct = 0;
real bottomRightDotProduct = 0;
real v=0;
real inTopLeft=0;
real inTopRight=0;
real inBottomLeft=0;
real inBottomRight=0;
// we are careful with the boundaries
bool topLeftIsIn = xInTopLeft >= 0 && xInTopLeft <= inputImages_width-1 && yInTopLeft >= 0 && yInTopLeft <= inputImages_height-1;
bool topRightIsIn = xInTopLeft+1 >= 0 && xInTopLeft+1 <= inputImages_width-1 && yInTopLeft >= 0 && yInTopLeft <= inputImages_height-1;
bool bottomLeftIsIn = xInTopLeft >= 0 && xInTopLeft <= inputImages_width-1 && yInTopLeft+1 >= 0 && yInTopLeft+1 <= inputImages_height-1;
bool bottomRightIsIn = xInTopLeft+1 >= 0 && xInTopLeft+1 <= inputImages_width-1 && yInTopLeft+1 >= 0 && yInTopLeft+1 <= inputImages_height-1;
int t;
for(t=0; t<inputImages_channels; t++)
{
real gradOutValue = gradOutput_data[gradOutputAddress + t];
if(topLeftIsIn)
{
real inTopLeft = inputImages_data[inTopLeftAddress + t];
topLeftDotProduct += inTopLeft * gradOutValue;
if(!onlyGrid) gradInputImages_data[gradInputImagesTopLeftAddress + t] += xWeightTopLeft * yWeightTopLeft * gradOutValue;
}
if(topRightIsIn)
{
real inTopRight = inputImages_data[inTopRightAddress + t];
topRightDotProduct += inTopRight * gradOutValue;
if(!onlyGrid) gradInputImages_data[gradInputImagesTopRightAddress + t] += (1 - xWeightTopLeft) * yWeightTopLeft * gradOutValue;
}
if(bottomLeftIsIn)
{
real inBottomLeft = inputImages_data[inBottomLeftAddress + t];
bottomLeftDotProduct += inBottomLeft * gradOutValue;
if(!onlyGrid) gradInputImages_data[gradInputImagesBottomLeftAddress + t] += xWeightTopLeft * (1 - yWeightTopLeft) * gradOutValue;
}
if(bottomRightIsIn)
{
real inBottomRight = inputImages_data[inBottomRightAddress + t];
bottomRightDotProduct += inBottomRight * gradOutValue;
if(!onlyGrid) gradInputImages_data[gradInputImagesBottomRightAddress + t] += (1 - xWeightTopLeft) * (1 - yWeightTopLeft) * gradOutValue;
}
}
yf = - xWeightTopLeft * topLeftDotProduct + xWeightTopLeft * bottomLeftDotProduct - (1-xWeightTopLeft) * topRightDotProduct + (1-xWeightTopLeft) * bottomRightDotProduct;
xf = - yWeightTopLeft * topLeftDotProduct + yWeightTopLeft * topRightDotProduct - (1-yWeightTopLeft) * bottomLeftDotProduct + (1-yWeightTopLeft) * bottomRightDotProduct;
gradGrids_data[b*gradGrids_strideBatch + yOut*gradGrids_strideHeight + xOut*gradGrids_strideWidth] = yf * (inputImages_height-1) / 2;
gradGrids_data[b*gradGrids_strideBatch + yOut*gradGrids_strideHeight + xOut*gradGrids_strideWidth + 1] = xf * (inputImages_width-1) / 2;
}
}
}
return 1;
}
int BilinearSamplerBCHW_updateOutput(THFloatTensor *inputImages, THFloatTensor *grids, THFloatTensor *output)
{
int batchsize = inputImages->size[0];
int inputImages_height = inputImages->size[2];
int inputImages_width = inputImages->size[3];
int output_height = output->size[2];
int output_width = output->size[3];
int inputImages_channels = inputImages->size[1];
int output_strideBatch = output->stride[0];
int output_strideHeight = output->stride[2];
int output_strideWidth = output->stride[3];
int output_strideChannel = output->stride[1];
int inputImages_strideBatch = inputImages->stride[0];
int inputImages_strideHeight = inputImages->stride[2];
int inputImages_strideWidth = inputImages->stride[3];
int inputImages_strideChannel = inputImages->stride[1];
int grids_strideBatch = grids->stride[0];
int grids_strideHeight = grids->stride[2];
int grids_strideWidth = grids->stride[3];
int grids_strideChannel = grids->stride[1];
real *inputImages_data, *output_data, *grids_data;
inputImages_data = THFloatTensor_data(inputImages);
output_data = THFloatTensor_data(output);
grids_data = THFloatTensor_data(grids);
int b, yOut, xOut;
for(b=0; b < batchsize; b++)
{
for(yOut=0; yOut < output_height; yOut++)
{
for(xOut=0; xOut < output_width; xOut++)
{
//read the grid
real xf = grids_data[b*grids_strideBatch + yOut*grids_strideHeight + xOut*grids_strideWidth + grids_strideChannel];
real yf = grids_data[b*grids_strideBatch + yOut*grids_strideHeight + xOut*grids_strideWidth];
// get the weights for interpolation
int yInTopLeft, xInTopLeft;
real yWeightTopLeft, xWeightTopLeft;
real xcoord = (xf + 1) * (inputImages_width - 1) / 2;
xInTopLeft = floor(xcoord);
xWeightTopLeft = 1 - (xcoord - xInTopLeft);
real ycoord = (yf + 1) * (inputImages_height - 1) / 2;
yInTopLeft = floor(ycoord);
yWeightTopLeft = 1 - (ycoord - yInTopLeft);
const int outAddress = output_strideBatch * b + output_strideHeight * yOut + output_strideWidth * xOut;
const int inTopLeftAddress = inputImages_strideBatch * b + inputImages_strideHeight * yInTopLeft + inputImages_strideWidth * xInTopLeft;
const int inTopRightAddress = inTopLeftAddress + inputImages_strideWidth;
const int inBottomLeftAddress = inTopLeftAddress + inputImages_strideHeight;
const int inBottomRightAddress = inBottomLeftAddress + inputImages_strideWidth;
real v=0;
real inTopLeft=0;
real inTopRight=0;
real inBottomLeft=0;
real inBottomRight=0;
// we are careful with the boundaries
bool topLeftIsIn = xInTopLeft >= 0 && xInTopLeft <= inputImages_width-1 && yInTopLeft >= 0 && yInTopLeft <= inputImages_height-1;
bool topRightIsIn = xInTopLeft+1 >= 0 && xInTopLeft+1 <= inputImages_width-1 && yInTopLeft >= 0 && yInTopLeft <= inputImages_height-1;
bool bottomLeftIsIn = xInTopLeft >= 0 && xInTopLeft <= inputImages_width-1 && yInTopLeft+1 >= 0 && yInTopLeft+1 <= inputImages_height-1;
bool bottomRightIsIn = xInTopLeft+1 >= 0 && xInTopLeft+1 <= inputImages_width-1 && yInTopLeft+1 >= 0 && yInTopLeft+1 <= inputImages_height-1;
int t;
// interpolation happens here
for(t=0; t<inputImages_channels; t++)
{
if(topLeftIsIn) inTopLeft = inputImages_data[inTopLeftAddress + t * inputImages_s
gitextract_940a232f/ ├── .gitignore ├── README.md ├── _init_paths.py ├── cfgs/ │ ├── res101_ms.yml │ └── res50.yml ├── lib/ │ ├── datasets/ │ │ ├── VOCdevkit-matlab-wrapper/ │ │ │ ├── get_voc_opts.m │ │ │ ├── voc_eval.m │ │ │ └── xVOCap.m │ │ ├── __init__.py │ │ ├── coco.py │ │ ├── ds_utils.py │ │ ├── factory.py │ │ ├── imdb.py │ │ ├── metadata.py │ │ ├── pascal_voc.py │ │ ├── pascal_voc_rbg.py │ │ ├── tools/ │ │ │ ├── compute_prior.py │ │ │ └── mcg_munge.py │ │ └── voc_eval.py │ ├── model/ │ │ ├── __init__.py │ │ ├── faster_rcnn/ │ │ │ ├── __init__.py │ │ │ ├── faster_rcnn.py │ │ │ ├── resnet.py │ │ │ ├── trail.py │ │ │ └── vgg16.py │ │ ├── nms/ │ │ │ ├── .gitignore │ │ │ ├── __init__.py │ │ │ ├── _ext/ │ │ │ │ ├── __init__.py │ │ │ │ └── nms/ │ │ │ │ └── __init__.py │ │ │ ├── build.py │ │ │ ├── nms_cpu.py │ │ │ ├── nms_gpu.py │ │ │ ├── nms_kernel.cu │ │ │ ├── nms_wrapper.py │ │ │ └── src/ │ │ │ ├── nms_cuda.h │ │ │ ├── nms_cuda_kernel.cu │ │ │ ├── nms_cuda_kernel.cu.o │ │ │ └── nms_cuda_kernel.h │ │ ├── roi_align/ │ │ │ ├── __init__.py │ │ │ ├── _ext/ │ │ │ │ ├── __init__.py │ │ │ │ └── roi_align/ │ │ │ │ └── __init__.py │ │ │ ├── build.py │ │ │ ├── functions/ │ │ │ │ ├── __init__.py │ │ │ │ └── roi_align.py │ │ │ ├── modules/ │ │ │ │ ├── __init__.py │ │ │ │ └── roi_align.py │ │ │ └── src/ │ │ │ ├── roi_align_cuda.c │ │ │ ├── roi_align_cuda.h │ │ │ ├── roi_align_kernel.cu │ │ │ ├── roi_align_kernel.cu.o │ │ │ └── roi_align_kernel.h │ │ ├── roi_crop/ │ │ │ ├── __init__.py │ │ │ ├── _ext/ │ │ │ │ ├── __init__.py │ │ │ │ └── roi_crop/ │ │ │ │ └── __init__.py │ │ │ ├── build.py │ │ │ ├── functions/ │ │ │ │ ├── __init__.py │ │ │ │ ├── crop_resize.py │ │ │ │ ├── gridgen.py │ │ │ │ └── roi_crop.py │ │ │ ├── modules/ │ │ │ │ ├── __init__.py │ │ │ │ ├── gridgen.py │ │ │ │ └── roi_crop.py │ │ │ └── src/ │ │ │ ├── roi_crop.c │ │ │ ├── roi_crop.h │ │ │ ├── roi_crop_cuda.c │ │ │ ├── roi_crop_cuda.h │ │ │ ├── roi_crop_cuda_kernel.cu │ │ │ ├── roi_crop_cuda_kernel.cu.o │ │ │ └── roi_crop_cuda_kernel.h │ │ ├── roi_pooling/ │ │ │ ├── __init__.py │ │ │ ├── _ext/ │ │ │ │ ├── __init__.py │ │ │ │ └── roi_pooling/ │ │ │ │ └── __init__.py │ │ │ ├── build.py │ │ │ ├── functions/ │ │ │ │ ├── __init__.py │ │ │ │ └── roi_pool.py │ │ │ ├── modules/ │ │ │ │ ├── __init__.py │ │ │ │ └── roi_pool.py │ │ │ └── src/ │ │ │ ├── roi_pooling.c │ │ │ ├── roi_pooling.cu.o │ │ │ ├── roi_pooling.h │ │ │ ├── roi_pooling_cuda.c │ │ │ ├── roi_pooling_cuda.h │ │ │ ├── roi_pooling_kernel.cu │ │ │ └── roi_pooling_kernel.h │ │ ├── rpn/ │ │ │ ├── __init__.py │ │ │ ├── anchor_target_layer.py │ │ │ ├── bbox_transform.py │ │ │ ├── generate_anchors.py │ │ │ ├── proposal_layer.py │ │ │ ├── proposal_layer_region.py │ │ │ ├── proposal_target_layer_cascade.py │ │ │ ├── proposal_target_layer_cascade_region.py │ │ │ ├── rpn.py │ │ │ └── rpn_region.py │ │ └── utils/ │ │ ├── .gitignore │ │ ├── __init__.py │ │ ├── bbox.pyx │ │ ├── blob.py │ │ ├── config.py │ │ └── net_utils.py │ ├── pycocotools/ │ │ ├── UPSTREAM_REV │ │ ├── __init__.py │ │ ├── _mask.c │ │ ├── _mask.pyx │ │ ├── coco.py │ │ ├── cocoeval.py │ │ ├── license.txt │ │ ├── mask.py │ │ ├── maskApi.c │ │ └── maskApi.h │ ├── roi_data_layer/ │ │ ├── __init__.py │ │ ├── minibatch.py │ │ ├── roibatchLoader.py │ │ └── roidb.py │ └── setup.py ├── test_metarcnn.py └── train_metarcnn.py
SYMBOL INDEX (722 symbols across 59 files)
FILE: _init_paths.py
function add_path (line 8) | def add_path(path):
FILE: lib/datasets/coco.py
class coco (line 27) | class coco(imdb):
method __init__ (line 28) | def __init__(self, image_set, year):
method _get_ann_file (line 68) | def _get_ann_file(self):
method _load_image_set_index (line 74) | def _load_image_set_index(self):
method _get_widths (line 81) | def _get_widths(self):
method image_path_at (line 86) | def image_path_at(self, i):
method image_id_at (line 92) | def image_id_at(self, i):
method image_path_from_index (line 98) | def image_path_from_index(self, index):
method gt_roidb (line 115) | def gt_roidb(self):
method _load_coco_annotation (line 135) | def _load_coco_annotation(self, index):
method _get_widths (line 193) | def _get_widths(self):
method append_flipped_images (line 196) | def append_flipped_images(self):
method _get_box_file (line 217) | def _get_box_file(self, index):
method _print_detection_eval_metrics (line 224) | def _print_detection_eval_metrics(self, coco_eval):
method _do_detection_eval (line 257) | def _do_detection_eval(self, res_file, output_dir):
method _coco_results_one_category (line 270) | def _coco_results_one_category(self, boxes, cat_id):
method _write_coco_results_file (line 288) | def _write_coco_results_file(self, all_boxes, res_file):
method evaluate_detections (line 306) | def evaluate_detections(self, all_boxes, output_dir):
method competition_mode (line 322) | def competition_mode(self, on):
FILE: lib/datasets/ds_utils.py
function unique_boxes (line 13) | def unique_boxes(boxes, scale=1.0):
function xywh_to_xyxy (line 21) | def xywh_to_xyxy(boxes):
function xyxy_to_xywh (line 26) | def xyxy_to_xywh(boxes):
function validate_boxes (line 31) | def validate_boxes(boxes, width=0, height=0):
function filter_small_boxes (line 45) | def filter_small_boxes(boxes, min_size):
FILE: lib/datasets/factory.py
function get_imdb (line 35) | def get_imdb(name):
function list_imdbs (line 42) | def list_imdbs():
FILE: lib/datasets/imdb.py
class imdb (line 22) | class imdb(object):
method __init__ (line 25) | def __init__(self, name, classes=None):
method name (line 40) | def name(self):
method num_classes (line 44) | def num_classes(self):
method set_classes (line 47) | def set_classes(self,classes):
method classes (line 51) | def classes(self):
method image_index (line 55) | def image_index(self):
method roidb_handler (line 59) | def roidb_handler(self):
method roidb_handler (line 63) | def roidb_handler(self, val):
method set_proposal_method (line 66) | def set_proposal_method(self, method):
method roidb (line 71) | def roidb(self):
method set_roidb (line 82) | def set_roidb(self,roidb):
method cache_path (line 88) | def cache_path(self):
method num_images (line 95) | def num_images(self):
method image_path_at (line 98) | def image_path_at(self, i):
method image_id_at (line 101) | def image_id_at(self, i):
method default_roidb (line 104) | def default_roidb(self):
method evaluate_detections (line 107) | def evaluate_detections(self, all_boxes, output_dir=None):
method _get_widths (line 118) | def _get_widths(self):
method append_flipped_images (line 122) | def append_flipped_images(self):
method evaluate_recall (line 141) | def evaluate_recall(self, candidate_boxes=None, thresholds=None,
method create_roidb_from_box_list (line 231) | def create_roidb_from_box_list(self, box_list, gt_roidb):
method merge_roidbs (line 261) | def merge_roidbs(a, b):
method competition_mode (line 273) | def competition_mode(self, on):
FILE: lib/datasets/metadata.py
class MetaDataset (line 20) | class MetaDataset(data.Dataset):
method __init__ (line 32) | def __init__(self, root, image_sets, metaclass, img_size, shots=1, shu...
method __getitem__ (line 71) | def __getitem__(self, index):
method get_prndata (line 74) | def get_prndata(self):
method __len__ (line 128) | def __len__(self):
FILE: lib/datasets/pascal_voc.py
class pascal_voc (line 40) | class pascal_voc(imdb):
method __init__ (line 41) | def __init__(self, image_set, year, devkit_path=None):
method image_path_at (line 80) | def image_path_at(self, i):
method image_id_at (line 86) | def image_id_at(self, i):
method image_path_from_index (line 92) | def image_path_from_index(self, index):
method _load_image_set_index (line 103) | def _load_image_set_index(self):
method _get_default_path (line 117) | def _get_default_path(self):
method gt_roidb (line 123) | def gt_roidb(self):
method selective_search_roidb (line 144) | def selective_search_roidb(self):
method rpn_roidb (line 172) | def rpn_roidb(self):
method _load_rpn_roidb (line 182) | def _load_rpn_roidb(self, gt_roidb):
method _load_selective_search_roidb (line 191) | def _load_selective_search_roidb(self, gt_roidb):
method _load_pascal_annotation (line 210) | def _load_pascal_annotation(self, index):
method _get_comp_id (line 264) | def _get_comp_id(self):
method _get_voc_results_file_template (line 269) | def _get_voc_results_file_template(self):
method _write_voc_results_file (line 278) | def _write_voc_results_file(self, all_boxes):
method _do_python_eval (line 296) | def _do_python_eval(self, output_dir='output', **kwargs):
method _do_matlab_eval (line 370) | def _do_matlab_eval(self, output_dir='output'):
method evaluate_detections (line 385) | def evaluate_detections(self, all_boxes, output_dir, **kwargs):
method competition_mode (line 397) | def competition_mode(self, on):
FILE: lib/datasets/pascal_voc_rbg.py
class VOCDetection (line 31) | class VOCDetection(data.Dataset):
method __init__ (line 48) | def __init__(self, root, image_sets, img_size, preproc=None, target_tr...
method __getitem__ (line 65) | def __getitem__(self, index):
method __len__ (line 97) | def __len__(self):
class pascal_voc (line 101) | class pascal_voc(imdb):
method __init__ (line 102) | def __init__(self, image_set, year, devkit_path=None):
method image_path_at (line 137) | def image_path_at(self, i):
method image_path_from_index (line 143) | def image_path_from_index(self, index):
method _load_image_set_index (line 153) | def _load_image_set_index(self):
method _get_default_path (line 168) | def _get_default_path(self):
method gt_roidb (line 174) | def gt_roidb(self):
method rpn_roidb (line 198) | def rpn_roidb(self):
method _load_rpn_roidb (line 208) | def _load_rpn_roidb(self, gt_roidb):
method _load_pascal_annotation (line 217) | def _load_pascal_annotation(self, index):
method _get_comp_id (line 263) | def _get_comp_id(self):
method _get_voc_results_file_template (line 268) | def _get_voc_results_file_template(self):
method _write_voc_results_file (line 279) | def _write_voc_results_file(self, all_boxes):
method _do_python_eval (line 297) | def _do_python_eval(self, output_dir='output'):
method _do_matlab_eval (line 342) | def _do_matlab_eval(self, output_dir='output'):
method evaluate_detections (line 357) | def evaluate_detections(self, all_boxes, output_dir):
method competition_mode (line 370) | def competition_mode(self, on):
FILE: lib/datasets/tools/compute_prior.py
function cout_w (line 7) | def cout_w(prob, num=NUM_ATTR_REL,dim=1):
function cp_kl (line 13) | def cp_kl(a, b):
function compute_js (line 22) | def compute_js(attr_prob):
FILE: lib/datasets/tools/mcg_munge.py
function munge (line 16) | def munge(src_dir):
FILE: lib/datasets/voc_eval.py
function parse_rec (line 15) | def parse_rec(filename):
function voc_ap (line 35) | def voc_ap(rec, prec, use_07_metric=False):
function voc_eval (line 69) | def voc_eval(detpath,
FILE: lib/model/faster_rcnn/faster_rcnn.py
class _fasterRCNN (line 25) | class _fasterRCNN(nn.Module):
method __init__ (line 28) | def __init__(self, classes, class_agnostic, meta_train, meta_test=None...
method forward (line 50) | def forward(self, im_data_list, im_info_list, gt_boxes_list, num_boxes...
method _init_weights (line 231) | def _init_weights(self):
method create_architecture (line 249) | def create_architecture(self):
FILE: lib/model/faster_rcnn/resnet.py
function conv3x3 (line 33) | def conv3x3(in_planes, out_planes, stride=1):
function init_conv (line 38) | def init_conv(conv,glu=True):
function init_linear (line 43) | def init_linear(linear):
class BasicBlock (line 47) | class BasicBlock(nn.Module):
method __init__ (line 50) | def __init__(self, inplanes, planes, stride=1, downsample=None):
method forward (line 60) | def forward(self, x):
class Bottleneck (line 79) | class Bottleneck(nn.Module):
method __init__ (line 82) | def __init__(self, inplanes, planes, stride=1, downsample=None):
method forward (line 95) | def forward(self, x):
class ResNet (line 118) | class ResNet(nn.Module):
method __init__ (line 119) | def __init__(self, block, layers, num_classes=1000):
method _make_layer (line 144) | def _make_layer(self, block, planes, blocks, stride=1):
method forward (line 161) | def forward(self, x):
function resnet18 (line 179) | def resnet18(pretrained=False):
function resnet34 (line 190) | def resnet34(pretrained=False):
function resnet50 (line 201) | def resnet50(pretrained=False):
function resnet101 (line 212) | def resnet101(pretrained=False):
function resnet152 (line 223) | def resnet152(pretrained=False):
class resnet (line 233) | class resnet(_fasterRCNN):
method __init__ (line 234) | def __init__(self, classes, num_layers=101, pretrained=False, class_ag...
method _init_modules (line 245) | def _init_modules(self):
method train (line 299) | def train(self, mode=True):
method _head_to_tail (line 318) | def _head_to_tail(self, pool5):
method prn_network (line 322) | def prn_network(self,im_data):
FILE: lib/model/faster_rcnn/trail.py
function cout_w (line 34) | def cout_w(prob):
function cout_a (line 44) | def cout_a(prob, NUM_ATTR_REL):
function parse_args (line 68) | def parse_args():
class sampler (line 159) | class sampler(Sampler):
method __init__ (line 160) | def __init__(self, train_size, batch_size):
method __iter__ (line 170) | def __iter__(self):
method __len__ (line 180) | def __len__(self):
FILE: lib/model/faster_rcnn/vgg16.py
class vgg16 (line 19) | class vgg16(_fasterRCNN):
method __init__ (line 20) | def __init__(self, classes, pretrained=False, class_agnostic=False):
method _init_modules (line 28) | def _init_modules(self):
method _head_to_tail (line 56) | def _head_to_tail(self, pool5):
FILE: lib/model/nms/_ext/nms/__init__.py
function _import_symbols (line 6) | def _import_symbols(locals):
FILE: lib/model/nms/nms_cpu.py
function nms_cpu (line 6) | def nms_cpu(dets, thresh):
function nms_cpu_np (line 39) | def nms_cpu_np(dets, thresh):
function soft_nms_cpu (line 70) | def soft_nms_cpu(dets, threshold=0.001, Nt=0.3, method=1):
function nms_domain (line 164) | def nms_domain(dets, dets_small, thresh_small=0.85, thresh_big=0.5):
FILE: lib/model/nms/nms_gpu.py
function nms_gpu (line 7) | def nms_gpu(dets, thresh):
FILE: lib/model/nms/nms_wrapper.py
function nms (line 13) | def nms(dets, thresh, force_cpu=False):
FILE: lib/model/roi_align/_ext/roi_align/__init__.py
function _import_symbols (line 6) | def _import_symbols(locals):
FILE: lib/model/roi_align/functions/roi_align.py
class RoIAlignFunction (line 7) | class RoIAlignFunction(Function):
method __init__ (line 8) | def __init__(self, aligned_height, aligned_width, spatial_scale):
method forward (line 15) | def forward(self, features, rois):
method backward (line 33) | def backward(self, grad_output):
FILE: lib/model/roi_align/modules/roi_align.py
class RoIAlign (line 6) | class RoIAlign(Module):
method __init__ (line 7) | def __init__(self, aligned_height, aligned_width, spatial_scale):
method forward (line 14) | def forward(self, features, rois):
class RoIAlignAvg (line 18) | class RoIAlignAvg(Module):
method __init__ (line 19) | def __init__(self, aligned_height, aligned_width, spatial_scale):
method forward (line 26) | def forward(self, features, rois):
class RoIAlignMax (line 31) | class RoIAlignMax(Module):
method __init__ (line 32) | def __init__(self, aligned_height, aligned_width, spatial_scale):
method forward (line 39) | def forward(self, features, rois):
FILE: lib/model/roi_align/src/roi_align_cuda.c
function roi_align_forward_cuda (line 7) | int roi_align_forward_cuda(int aligned_height, int aligned_width, float ...
function roi_align_backward_cuda (line 42) | int roi_align_backward_cuda(int aligned_height, int aligned_width, float...
FILE: lib/model/roi_crop/_ext/roi_crop/__init__.py
function _import_symbols (line 6) | def _import_symbols(locals):
FILE: lib/model/roi_crop/functions/crop_resize.py
class RoICropFunction (line 8) | class RoICropFunction(Function):
method forward (line 9) | def forward(self, input1, input2):
method backward (line 27) | def backward(self, grad_output):
FILE: lib/model/roi_crop/functions/gridgen.py
class AffineGridGenFunction (line 7) | class AffineGridGenFunction(Function):
method __init__ (line 8) | def __init__(self, height, width,lr=1):
method forward (line 21) | def forward(self, input1):
method backward (line 37) | def backward(self, grad_output):
FILE: lib/model/roi_crop/functions/roi_crop.py
class RoICropFunction (line 7) | class RoICropFunction(Function):
method forward (line 8) | def forward(self, input1, input2):
method backward (line 17) | def backward(self, grad_output):
FILE: lib/model/roi_crop/modules/gridgen.py
class _AffineGridGen (line 12) | class _AffineGridGen(Module):
method __init__ (line 13) | def __init__(self, height, width, lr = 1, aux_loss = False):
method forward (line 19) | def forward(self, input):
class CylinderGridGen (line 34) | class CylinderGridGen(Module):
method __init__ (line 35) | def __init__(self, height, width, lr = 1, aux_loss = False):
method forward (line 41) | def forward(self, input):
class AffineGridGenV2 (line 49) | class AffineGridGenV2(Module):
method __init__ (line 50) | def __init__(self, height, width, lr = 1, aux_loss = False):
method forward (line 63) | def forward(self, input1):
class CylinderGridGenV2 (line 78) | class CylinderGridGenV2(Module):
method __init__ (line 79) | def __init__(self, height, width, lr = 1):
method forward (line 88) | def forward(self, input):
class DenseAffineGridGen (line 108) | class DenseAffineGridGen(Module):
method __init__ (line 109) | def __init__(self, height, width, lr = 1, aux_loss = False):
method forward (line 122) | def forward(self, input1):
class DenseAffine3DGridGen (line 140) | class DenseAffine3DGridGen(Module):
method __init__ (line 141) | def __init__(self, height, width, lr = 1, aux_loss = False):
method forward (line 168) | def forward(self, input1):
class DenseAffine3DGridGen_rotate (line 198) | class DenseAffine3DGridGen_rotate(Module):
method __init__ (line 199) | def __init__(self, height, width, lr = 1, aux_loss = False):
method forward (line 226) | def forward(self, input1, input2):
class Depth3DGridGen (line 265) | class Depth3DGridGen(Module):
method __init__ (line 266) | def __init__(self, height, width, lr = 1, aux_loss = False):
method forward (line 293) | def forward(self, depth, trans0, trans1, rotate):
class Depth3DGridGen_with_mask (line 338) | class Depth3DGridGen_with_mask(Module):
method __init__ (line 339) | def __init__(self, height, width, lr = 1, aux_loss = False, ray_tracin...
method forward (line 367) | def forward(self, depth, trans0, trans1, rotate):
FILE: lib/model/roi_crop/modules/roi_crop.py
class _RoICrop (line 4) | class _RoICrop(Module):
method __init__ (line 5) | def __init__(self, layout = 'BHWD'):
method forward (line 7) | def forward(self, input1, input2):
FILE: lib/model/roi_crop/src/roi_crop.c
function BilinearSamplerBHWD_updateOutput (line 7) | int BilinearSamplerBHWD_updateOutput(THFloatTensor *inputImages, THFloat...
function BilinearSamplerBHWD_updateGradInput (line 105) | int BilinearSamplerBHWD_updateGradInput(THFloatTensor *inputImages, THFl...
function BilinearSamplerBCHW_updateOutput (line 246) | int BilinearSamplerBCHW_updateOutput(THFloatTensor *inputImages, THFloat...
function BilinearSamplerBCHW_updateGradInput (line 350) | int BilinearSamplerBCHW_updateGradInput(THFloatTensor *inputImages, THFl...
FILE: lib/model/roi_crop/src/roi_crop_cuda.c
function BilinearSamplerBHWD_updateOutput_cuda (line 15) | int BilinearSamplerBHWD_updateOutput_cuda(THCudaTensor *inputImages, THC...
function BilinearSamplerBHWD_updateGradInput_cuda (line 54) | int BilinearSamplerBHWD_updateGradInput_cuda(THCudaTensor *inputImages, ...
FILE: lib/model/roi_pooling/_ext/roi_pooling/__init__.py
function _import_symbols (line 6) | def _import_symbols(locals):
FILE: lib/model/roi_pooling/functions/roi_pool.py
class RoIPoolFunction (line 6) | class RoIPoolFunction(Function):
method __init__ (line 7) | def __init__(ctx, pooled_height, pooled_width, spatial_scale):
method forward (line 13) | def forward(ctx, features, rois):
method backward (line 30) | def backward(ctx, grad_output):
FILE: lib/model/roi_pooling/modules/roi_pool.py
class _RoIPooling (line 5) | class _RoIPooling(Module):
method __init__ (line 6) | def __init__(self, pooled_height, pooled_width, spatial_scale):
method forward (line 13) | def forward(self, features, rois):
FILE: lib/model/roi_pooling/src/roi_pooling.c
function roi_pooling_forward (line 4) | int roi_pooling_forward(int pooled_height, int pooled_width, float spati...
FILE: lib/model/roi_pooling/src/roi_pooling_cuda.c
function roi_pooling_forward_cuda (line 7) | int roi_pooling_forward_cuda(int pooled_height, int pooled_width, float ...
function roi_pooling_backward_cuda (line 49) | int roi_pooling_backward_cuda(int pooled_height, int pooled_width, float...
FILE: lib/model/rpn/anchor_target_layer.py
class _AnchorTargetLayer (line 31) | class _AnchorTargetLayer(nn.Module):
method __init__ (line 36) | def __init__(self, feat_stride, scales, ratios):
method forward (line 48) | def forward(self, input):
method backward (line 194) | def backward(self, top, propagate_down, bottom):
method reshape (line 198) | def reshape(self, bottom, top):
function _unmap (line 202) | def _unmap(data, count, inds, batch_size, fill=0):
function _compute_targets_batch (line 215) | def _compute_targets_batch(ex_rois, gt_rois):
FILE: lib/model/rpn/bbox_transform.py
function bbox_transform (line 15) | def bbox_transform(ex_rois, gt_rois):
function bbox_transform_batch (line 36) | def bbox_transform_batch(ex_rois, gt_rois):
function bbox_transform_inv (line 77) | def bbox_transform_inv(boxes, deltas, batch_size):
function clip_boxes_batch (line 105) | def clip_boxes_batch(boxes, im_shape, batch_size):
function clip_boxes (line 125) | def clip_boxes(boxes, im_shape, batch_size):
function bbox_overlaps (line 136) | def bbox_overlaps(anchors, gt_boxes):
function bbox_overlaps_batch (line 168) | def bbox_overlaps_batch(anchors, gt_boxes):
FILE: lib/model/rpn/generate_anchors.py
function generate_anchors (line 45) | def generate_anchors(base_size=16, ratios=[0.5, 1, 2],
function _whctrs (line 58) | def _whctrs(anchor):
function _mkanchors (line 69) | def _mkanchors(ws, hs, x_ctr, y_ctr):
function _ratio_enum (line 83) | def _ratio_enum(anchor, ratios):
function _scale_enum (line 96) | def _scale_enum(anchor, scales):
FILE: lib/model/rpn/proposal_layer.py
class _ProposalLayer (line 26) | class _ProposalLayer(nn.Module):
method __init__ (line 32) | def __init__(self, feat_stride, scales, ratios):
method forward (line 49) | def forward(self, input):
method backward (line 163) | def backward(self, top, propagate_down, bottom):
method reshape (line 167) | def reshape(self, bottom, top):
method _filter_boxes (line 171) | def _filter_boxes(self, boxes, min_size):
FILE: lib/model/rpn/proposal_layer_region.py
class _ProposalLayer (line 26) | class _ProposalLayer(nn.Module):
method __init__ (line 32) | def __init__(self, feat_stride, scales, ratios):
method forward (line 49) | def forward(self, input):
method backward (line 166) | def backward(self, top, propagate_down, bottom):
method reshape (line 170) | def reshape(self, bottom, top):
method _filter_boxes (line 174) | def _filter_boxes(self, boxes, min_size):
FILE: lib/model/rpn/proposal_target_layer_cascade.py
class _ProposalTargetLayer (line 20) | class _ProposalTargetLayer(nn.Module):
method __init__ (line 26) | def __init__(self, nclasses):
method forward (line 33) | def forward(self, all_rois, gt_boxes, num_boxes):
method backward (line 52) | def backward(self, top, propagate_down, bottom):
method reshape (line 56) | def reshape(self, bottom, top):
method _get_bbox_regression_labels_pytorch (line 60) | def _get_bbox_regression_labels_pytorch(self, bbox_target_data, labels...
method _compute_targets_pytorch (line 90) | def _compute_targets_pytorch(self, ex_rois, gt_rois):
method _sample_rois_pytorch (line 110) | def _sample_rois_pytorch(self, all_rois, gt_boxes, fg_rois_per_image, ...
FILE: lib/model/rpn/proposal_target_layer_cascade_region.py
class _ProposalTargetLayer (line 20) | class _ProposalTargetLayer(nn.Module):
method __init__ (line 26) | def __init__(self, nclasses):
method forward (line 33) | def forward(self, all_rois, gt_boxes, num_boxes,output_cls_score):
method backward (line 60) | def backward(self, top, propagate_down, bottom):
method reshape (line 64) | def reshape(self, bottom, top):
method _get_bbox_regression_labels_pytorch (line 68) | def _get_bbox_regression_labels_pytorch(self, bbox_target_data, labels...
method _compute_targets_pytorch (line 98) | def _compute_targets_pytorch(self, ex_rois, gt_rois):
method _sample_rois_pytorch (line 118) | def _sample_rois_pytorch(self, all_rois, gt_boxes, fg_rois_per_image, ...
FILE: lib/model/rpn/rpn.py
class _RPN (line 17) | class _RPN(nn.Module):
method __init__ (line 19) | def __init__(self, din):
method reshape (line 48) | def reshape(x, d):
method forward (line 58) | def forward(self, base_feat, im_info, gt_boxes, num_boxes):
class _RPN_out_pred_label (line 113) | class _RPN_out_pred_label(nn.Module):
method __init__ (line 115) | def __init__(self, din):
method reshape (line 144) | def reshape(x, d):
method forward (line 154) | def forward(self, base_feat, im_info, gt_boxes, num_boxes):
FILE: lib/model/rpn/rpn_region.py
class _RPN (line 17) | class _RPN(nn.Module):
method __init__ (line 19) | def __init__(self, din):
method reshape (line 48) | def reshape(x, d):
method forward (line 58) | def forward(self, base_feat, im_info, gt_boxes, num_boxes):
FILE: lib/model/utils/blob.py
function im_list_to_blob (line 20) | def im_list_to_blob(ims):
function prep_im_for_blob (line 35) | def prep_im_for_blob(im, pixel_means, target_size, max_size):
FILE: lib/model/utils/config.py
function get_output_dir (line 337) | def get_output_dir(imdb, weights_filename):
function get_output_tb_dir (line 353) | def get_output_tb_dir(imdb, weights_filename):
function _merge_a_into_b (line 369) | def _merge_a_into_b(a, b):
function cfg_from_file (line 402) | def cfg_from_file(filename):
function cfg_from_list (line 411) | def cfg_from_list(cfg_list):
FILE: lib/model/utils/net_utils.py
function save_net (line 13) | def save_net(fname, net):
function load_net (line 19) | def load_net(fname, net):
function weights_normal_init (line 26) | def weights_normal_init(model, dev=0.01):
function clip_gradient (line 38) | def clip_gradient(model, clip_norm):
function vis_detections (line 52) | def vis_detections(im, class_name, dets, thresh=0.8):
function vis_detections_label_only (line 69) | def vis_detections_label_only(im, class_name, dets, thresh=0.8):
function adjust_learning_rate (line 88) | def adjust_learning_rate(optimizer, decay=0.1):
function save_checkpoint (line 94) | def save_checkpoint(state, filename):
function _smooth_l1_loss (line 97) | def _smooth_l1_loss(bbox_pred, bbox_targets, bbox_inside_weights, bbox_o...
function _crop_pool_layer (line 113) | def _crop_pool_layer(bottom, rois, max_pool=True):
function _affine_grid_gen (line 167) | def _affine_grid_gen(rois, input_size, grid_size):
function _affine_theta (line 191) | def _affine_theta(rois, input_size):
function compare_grid_sample (line 222) | def compare_grid_sample():
FILE: lib/pycocotools/_mask.c
type PyObject (line 341) | typedef PyObject *(*__Pyx_PyCFunctionFast) (PyObject *self, PyObject *co...
type PyObject (line 342) | typedef PyObject *(*__Pyx_PyCFunctionFastWithKeywords) (PyObject *self, ...
type Py_tss_t (line 383) | typedef int Py_tss_t;
function CYTHON_INLINE (line 384) | static CYTHON_INLINE int PyThread_tss_create(Py_tss_t *key) {
function CYTHON_INLINE (line 388) | static CYTHON_INLINE Py_tss_t * PyThread_tss_alloc(void) {
function CYTHON_INLINE (line 393) | static CYTHON_INLINE void PyThread_tss_free(Py_tss_t *key) {
function CYTHON_INLINE (line 396) | static CYTHON_INLINE int PyThread_tss_is_created(Py_tss_t *key) {
function CYTHON_INLINE (line 399) | static CYTHON_INLINE void PyThread_tss_delete(Py_tss_t *key) {
function CYTHON_INLINE (line 403) | static CYTHON_INLINE int PyThread_tss_set(Py_tss_t *key, void *value) {
function CYTHON_INLINE (line 406) | static CYTHON_INLINE void * PyThread_tss_get(Py_tss_t *key) {
type Py_hash_t (line 530) | typedef long Py_hash_t;
type __Pyx_PyAsyncMethodsStruct (line 553) | typedef struct {
function CYTHON_INLINE (line 567) | static CYTHON_INLINE float __PYX_NAN() {
type __Pyx_StringTabEntry (line 610) | typedef struct {PyObject **p; const char *s; const Py_ssize_t n; const c...
function CYTHON_INLINE (line 631) | static CYTHON_INLINE int __Pyx_is_valid_index(Py_ssize_t i, Py_ssize_t l...
function CYTHON_INLINE (line 680) | static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u) {
function __Pyx_init_sys_getdefaultencoding_params (line 712) | static int __Pyx_init_sys_getdefaultencoding_params(void) {
function __Pyx_init_sys_getdefaultencoding_params (line 762) | static int __Pyx_init_sys_getdefaultencoding_params(void) {
function CYTHON_INLINE (line 794) | static CYTHON_INLINE void __Pyx_pretend_to_initialize(void* ptr) { (void...
type __Pyx_StructField_ (line 839) | struct __Pyx_StructField_
type __Pyx_TypeInfo (line 841) | typedef struct {
type __Pyx_StructField (line 851) | typedef struct __Pyx_StructField_ {
type __Pyx_BufFmt_StackElem (line 856) | typedef struct {
type __Pyx_BufFmt_Context (line 860) | typedef struct {
type npy_int8 (line 881) | typedef npy_int8 __pyx_t_5numpy_int8_t;
type npy_int16 (line 890) | typedef npy_int16 __pyx_t_5numpy_int16_t;
type npy_int32 (line 899) | typedef npy_int32 __pyx_t_5numpy_int32_t;
type npy_int64 (line 908) | typedef npy_int64 __pyx_t_5numpy_int64_t;
type npy_uint8 (line 917) | typedef npy_uint8 __pyx_t_5numpy_uint8_t;
type npy_uint16 (line 926) | typedef npy_uint16 __pyx_t_5numpy_uint16_t;
type npy_uint32 (line 935) | typedef npy_uint32 __pyx_t_5numpy_uint32_t;
type npy_uint64 (line 944) | typedef npy_uint64 __pyx_t_5numpy_uint64_t;
type npy_float32 (line 953) | typedef npy_float32 __pyx_t_5numpy_float32_t;
type npy_float64 (line 962) | typedef npy_float64 __pyx_t_5numpy_float64_t;
type npy_long (line 971) | typedef npy_long __pyx_t_5numpy_int_t;
type npy_longlong (line 980) | typedef npy_longlong __pyx_t_5numpy_long_t;
type npy_longlong (line 989) | typedef npy_longlong __pyx_t_5numpy_longlong_t;
type npy_ulong (line 998) | typedef npy_ulong __pyx_t_5numpy_uint_t;
type npy_ulonglong (line 1007) | typedef npy_ulonglong __pyx_t_5numpy_ulong_t;
type npy_ulonglong (line 1016) | typedef npy_ulonglong __pyx_t_5numpy_ulonglong_t;
type npy_intp (line 1025) | typedef npy_intp __pyx_t_5numpy_intp_t;
type npy_uintp (line 1034) | typedef npy_uintp __pyx_t_5numpy_uintp_t;
type npy_double (line 1043) | typedef npy_double __pyx_t_5numpy_float_t;
type npy_double (line 1052) | typedef npy_double __pyx_t_5numpy_double_t;
type npy_longdouble (line 1061) | typedef npy_longdouble __pyx_t_5numpy_longdouble_t;
type std (line 1065) | typedef ::std::complex< float > __pyx_t_float_complex;
type __pyx_t_float_complex (line 1067) | typedef float _Complex __pyx_t_float_complex;
type __pyx_t_float_complex (line 1070) | typedef struct { float real, imag; } __pyx_t_float_complex;
type std (line 1077) | typedef ::std::complex< double > __pyx_t_double_complex;
type __pyx_t_double_complex (line 1079) | typedef double _Complex __pyx_t_double_complex;
type __pyx_t_double_complex (line 1082) | typedef struct { double real, imag; } __pyx_t_double_complex;
type __pyx_obj_11pycocotools_5_mask_RLEs (line 1088) | struct __pyx_obj_11pycocotools_5_mask_RLEs
type __pyx_obj_11pycocotools_5_mask_Masks (line 1089) | struct __pyx_obj_11pycocotools_5_mask_Masks
type npy_cfloat (line 1098) | typedef npy_cfloat __pyx_t_5numpy_cfloat_t;
type npy_cdouble (line 1107) | typedef npy_cdouble __pyx_t_5numpy_cdouble_t;
type npy_clongdouble (line 1116) | typedef npy_clongdouble __pyx_t_5numpy_clongdouble_t;
type npy_cdouble (line 1125) | typedef npy_cdouble __pyx_t_5numpy_complex_t;
type __pyx_obj_11pycocotools_5_mask_RLEs (line 1134) | struct __pyx_obj_11pycocotools_5_mask_RLEs {
type __pyx_obj_11pycocotools_5_mask_Masks (line 1148) | struct __pyx_obj_11pycocotools_5_mask_Masks {
type __Pyx_RefNannyAPIStruct (line 1163) | typedef struct {
function CYTHON_INLINE (line 1354) | static CYTHON_INLINE int __Pyx_PyList_Append(PyObject* list, PyObject* x) {
function CYTHON_INLINE (line 1486) | static CYTHON_INLINE int __Pyx_ListComp_Append(PyObject* list, PyObject*...
type __pyx_CyFunctionObject (line 1517) | typedef struct {
type __Pyx_ImportType_CheckSize (line 1629) | enum __Pyx_ImportType_CheckSize {
type __Pyx_ImportType_CheckSize (line 1634) | enum __Pyx_ImportType_CheckSize
type __Pyx_CodeObjectCacheEntry (line 1648) | typedef struct {
type __Pyx_CodeObjectCache (line 1652) | struct __Pyx_CodeObjectCache {
type __Pyx_CodeObjectCache (line 1657) | struct __Pyx_CodeObjectCache
type __Pyx_Buf_DimInfo (line 1667) | typedef struct {
type __Pyx_Buffer (line 1670) | typedef struct {
type __Pyx_LocalBuf_ND (line 1674) | typedef struct {
type NPY_TYPES (line 1800) | enum NPY_TYPES
type __pyx_obj_11pycocotools_5_mask_RLEs (line 2106) | struct __pyx_obj_11pycocotools_5_mask_RLEs
type __pyx_obj_11pycocotools_5_mask_RLEs (line 2107) | struct __pyx_obj_11pycocotools_5_mask_RLEs
type __pyx_obj_11pycocotools_5_mask_RLEs (line 2108) | struct __pyx_obj_11pycocotools_5_mask_RLEs
type __pyx_obj_11pycocotools_5_mask_Masks (line 2111) | struct __pyx_obj_11pycocotools_5_mask_Masks
type __pyx_obj_11pycocotools_5_mask_Masks (line 2112) | struct __pyx_obj_11pycocotools_5_mask_Masks
type __pyx_obj_11pycocotools_5_mask_RLEs (line 2115) | struct __pyx_obj_11pycocotools_5_mask_RLEs
type __pyx_obj_11pycocotools_5_mask_RLEs (line 2122) | struct __pyx_obj_11pycocotools_5_mask_RLEs
type __pyx_obj_11pycocotools_5_mask_RLEs (line 2122) | struct __pyx_obj_11pycocotools_5_mask_RLEs
function __pyx_pw_11pycocotools_5_mask_4RLEs_1__cinit__ (line 2200) | static int __pyx_pw_11pycocotools_5_mask_4RLEs_1__cinit__(PyObject *__py...
function __pyx_pf_11pycocotools_5_mask_4RLEs___cinit__ (line 2257) | static int __pyx_pf_11pycocotools_5_mask_4RLEs___cinit__(struct __pyx_ob...
function __pyx_pw_11pycocotools_5_mask_4RLEs_3__dealloc__ (line 2304) | static void __pyx_pw_11pycocotools_5_mask_4RLEs_3__dealloc__(PyObject *_...
function __pyx_pf_11pycocotools_5_mask_4RLEs_2__dealloc__ (line 2313) | static void __pyx_pf_11pycocotools_5_mask_4RLEs_2__dealloc__(struct __py...
function PyObject (line 2394) | static PyObject *__pyx_pw_11pycocotools_5_mask_4RLEs_5__getattr__(PyObje...
function PyObject (line 2405) | static PyObject *__pyx_pf_11pycocotools_5_mask_4RLEs_4__getattr__(struct...
function PyObject (line 2485) | static PyObject *__pyx_pw_11pycocotools_5_mask_4RLEs_7__reduce_cython__(...
function PyObject (line 2496) | static PyObject *__pyx_pf_11pycocotools_5_mask_4RLEs_6__reduce_cython__(...
function PyObject (line 2539) | static PyObject *__pyx_pw_11pycocotools_5_mask_4RLEs_9__setstate_cython_...
function PyObject (line 2550) | static PyObject *__pyx_pf_11pycocotools_5_mask_4RLEs_8__setstate_cython_...
function __pyx_pw_11pycocotools_5_mask_5Masks_1__cinit__ (line 2594) | static int __pyx_pw_11pycocotools_5_mask_5Masks_1__cinit__(PyObject *__p...
function __pyx_pf_11pycocotools_5_mask_5Masks___cinit__ (line 2664) | static int __pyx_pf_11pycocotools_5_mask_5Masks___cinit__(struct __pyx_o...
function PyObject (line 2758) | static PyObject *__pyx_pw_11pycocotools_5_mask_5Masks_3__array__(PyObjec...
function PyObject (line 2769) | static PyObject *__pyx_pf_11pycocotools_5_mask_5Masks_2__array__(struct ...
function PyObject (line 2889) | static PyObject *__pyx_pw_11pycocotools_5_mask_5Masks_5__reduce_cython__...
function PyObject (line 2900) | static PyObject *__pyx_pf_11pycocotools_5_mask_5Masks_4__reduce_cython__...
function PyObject (line 2943) | static PyObject *__pyx_pw_11pycocotools_5_mask_5Masks_7__setstate_cython...
function PyObject (line 2954) | static PyObject *__pyx_pf_11pycocotools_5_mask_5Masks_6__setstate_cython...
function PyObject (line 2999) | static PyObject *__pyx_pw_11pycocotools_5_mask_1_toString(PyObject *__py...
function PyObject (line 3015) | static PyObject *__pyx_pf_11pycocotools_5_mask__toString(CYTHON_UNUSED P...
function PyObject (line 3191) | static PyObject *__pyx_pw_11pycocotools_5_mask_3_frString(PyObject *__py...
function PyObject (line 3202) | static PyObject *__pyx_pf_11pycocotools_5_mask_2_frString(CYTHON_UNUSED ...
function PyObject (line 3420) | static PyObject *__pyx_pw_11pycocotools_5_mask_5encode(PyObject *__pyx_s...
function PyObject (line 3436) | static PyObject *__pyx_pf_11pycocotools_5_mask_4encode(CYTHON_UNUSED PyO...
function PyObject (line 3583) | static PyObject *__pyx_pw_11pycocotools_5_mask_7decode(PyObject *__pyx_s...
function PyObject (line 3594) | static PyObject *__pyx_pf_11pycocotools_5_mask_6decode(CYTHON_UNUSED PyO...
function PyObject (line 3759) | static PyObject *__pyx_pw_11pycocotools_5_mask_9merge(PyObject *__pyx_se...
function PyObject (line 3825) | static PyObject *__pyx_pf_11pycocotools_5_mask_8merge(CYTHON_UNUSED PyOb...
function PyObject (line 3962) | static PyObject *__pyx_pw_11pycocotools_5_mask_11area(PyObject *__pyx_se...
function PyObject (line 3973) | static PyObject *__pyx_pf_11pycocotools_5_mask_10area(CYTHON_UNUSED PyOb...
function PyObject (line 4153) | static PyObject *__pyx_pw_11pycocotools_5_mask_13iou(PyObject *__pyx_sel...
function PyObject (line 4234) | static PyObject *__pyx_pw_11pycocotools_5_mask_3iou_1_preproc(PyObject *...
function PyObject (line 4245) | static PyObject *__pyx_pf_11pycocotools_5_mask_3iou__preproc(CYTHON_UNUS...
function PyObject (line 4966) | static PyObject *__pyx_pw_11pycocotools_5_mask_3iou_3_rleIou(PyObject *_...
function PyObject (line 5077) | static PyObject *__pyx_pf_11pycocotools_5_mask_3iou_2_rleIou(CYTHON_UNUS...
function PyObject (line 5155) | static PyObject *__pyx_pw_11pycocotools_5_mask_3iou_5_bbIou(PyObject *__...
function PyObject (line 5266) | static PyObject *__pyx_pf_11pycocotools_5_mask_3iou_4_bbIou(CYTHON_UNUSE...
function PyObject (line 5370) | static PyObject *__pyx_pw_11pycocotools_5_mask_3iou_7_len(PyObject *__py...
function PyObject (line 5381) | static PyObject *__pyx_pf_11pycocotools_5_mask_3iou_6_len(CYTHON_UNUSED ...
function PyObject (line 5529) | static PyObject *__pyx_pf_11pycocotools_5_mask_12iou(CYTHON_UNUSED PyObj...
function PyObject (line 6120) | static PyObject *__pyx_pw_11pycocotools_5_mask_15toBbox(PyObject *__pyx_...
function PyObject (line 6131) | static PyObject *__pyx_pf_11pycocotools_5_mask_14toBbox(CYTHON_UNUSED Py...
function PyObject (line 6358) | static PyObject *__pyx_pw_11pycocotools_5_mask_17frBbox(PyObject *__pyx_...
function PyObject (line 6433) | static PyObject *__pyx_pf_11pycocotools_5_mask_16frBbox(CYTHON_UNUSED Py...
function PyObject (line 6570) | static PyObject *__pyx_pw_11pycocotools_5_mask_19frPoly(PyObject *__pyx_...
function PyObject (line 6640) | static PyObject *__pyx_pf_11pycocotools_5_mask_18frPoly(CYTHON_UNUSED Py...
function PyObject (line 6922) | static PyObject *__pyx_pw_11pycocotools_5_mask_21frUncompressedRLE(PyObj...
type __pyx_obj_11pycocotools_5_mask_RLEs (line 6999) | struct __pyx_obj_11pycocotools_5_mask_RLEs
type __pyx_obj_11pycocotools_5_mask_RLEs (line 7074) | struct __pyx_obj_11pycocotools_5_mask_RLEs
function PyObject (line 7310) | static PyObject *__pyx_pw_11pycocotools_5_mask_23frPyObjects(PyObject *_...
function PyObject (line 7380) | static PyObject *__pyx_pf_11pycocotools_5_mask_22frPyObjects(CYTHON_UNUS...
function CYTHON_UNUSED (line 7835) | static CYTHON_UNUSED int __pyx_pw_5numpy_7ndarray_1__getbuffer__(PyObjec...
function __pyx_pf_5numpy_7ndarray___getbuffer__ (line 7846) | static int __pyx_pf_5numpy_7ndarray___getbuffer__(PyArrayObject *__pyx_v...
function CYTHON_UNUSED (line 8599) | static CYTHON_UNUSED void __pyx_pw_5numpy_7ndarray_3__releasebuffer__(Py...
function __pyx_pf_5numpy_7ndarray_2__releasebuffer__ (line 8608) | static void __pyx_pf_5numpy_7ndarray_2__releasebuffer__(PyArrayObject *_...
function CYTHON_INLINE (line 8689) | static CYTHON_INLINE PyObject *__pyx_f_5numpy_PyArray_MultiIterNew1(PyOb...
function CYTHON_INLINE (line 8736) | static CYTHON_INLINE PyObject *__pyx_f_5numpy_PyArray_MultiIterNew2(PyOb...
function CYTHON_INLINE (line 8783) | static CYTHON_INLINE PyObject *__pyx_f_5numpy_PyArray_MultiIterNew3(PyOb...
function CYTHON_INLINE (line 8830) | static CYTHON_INLINE PyObject *__pyx_f_5numpy_PyArray_MultiIterNew4(PyOb...
function CYTHON_INLINE (line 8877) | static CYTHON_INLINE PyObject *__pyx_f_5numpy_PyArray_MultiIterNew5(PyOb...
function CYTHON_INLINE (line 8924) | static CYTHON_INLINE PyObject *__pyx_f_5numpy_PyDataType_SHAPE(PyArray_D...
function CYTHON_INLINE (line 8998) | static CYTHON_INLINE char *__pyx_f_5numpy__util_dtypestring(PyArray_Desc...
function CYTHON_INLINE (line 9744) | static CYTHON_INLINE void __pyx_f_5numpy_set_array_base(PyArrayObject *_...
function CYTHON_INLINE (line 9786) | static CYTHON_INLINE PyObject *__pyx_f_5numpy_get_array_base(PyArrayObje...
function CYTHON_INLINE (line 9867) | static CYTHON_INLINE int __pyx_f_5numpy_import_array(void) {
function CYTHON_INLINE (line 9996) | static CYTHON_INLINE int __pyx_f_5numpy_import_umath(void) {
function CYTHON_INLINE (line 10125) | static CYTHON_INLINE int __pyx_f_5numpy_import_ufunc(void) {
function PyObject (line 10243) | static PyObject *__pyx_tp_new_11pycocotools_5_mask_RLEs(PyTypeObject *t,...
function __pyx_tp_dealloc_11pycocotools_5_mask_RLEs (line 10258) | static void __pyx_tp_dealloc_11pycocotools_5_mask_RLEs(PyObject *o) {
function PyObject (line 10275) | static PyObject *__pyx_tp_getattro_11pycocotools_5_mask_RLEs(PyObject *o...
type __pyx_obj_11pycocotools_5_mask_RLEs (line 10294) | struct __pyx_obj_11pycocotools_5_mask_RLEs
function PyObject (line 10352) | static PyObject *__pyx_tp_new_11pycocotools_5_mask_Masks(PyTypeObject *t...
function __pyx_tp_dealloc_11pycocotools_5_mask_Masks (line 10367) | static void __pyx_tp_dealloc_11pycocotools_5_mask_Masks(PyObject *o) {
type __pyx_obj_11pycocotools_5_mask_Masks (line 10386) | struct __pyx_obj_11pycocotools_5_mask_Masks
type PyModuleDef (line 10459) | struct PyModuleDef
function CYTHON_SMALL_CODE (line 10603) | static CYTHON_SMALL_CODE int __Pyx_InitCachedBuiltins(void) {
function CYTHON_SMALL_CODE (line 10616) | static CYTHON_SMALL_CODE int __Pyx_InitCachedConstants(void) {
function CYTHON_SMALL_CODE (line 11009) | static CYTHON_SMALL_CODE int __Pyx_InitGlobals(void) {
function __Pyx_modinit_global_init_code (line 11027) | static int __Pyx_modinit_global_init_code(void) {
function __Pyx_modinit_variable_export_code (line 11035) | static int __Pyx_modinit_variable_export_code(void) {
function __Pyx_modinit_function_export_code (line 11043) | static int __Pyx_modinit_function_export_code(void) {
function __Pyx_modinit_type_init_code (line 11051) | static int __Pyx_modinit_type_init_code(void) {
function __Pyx_modinit_type_import_code (line 11079) | static int __Pyx_modinit_type_import_code(void) {
function __Pyx_modinit_variable_import_code (line 11116) | static int __Pyx_modinit_variable_import_code(void) {
function __Pyx_modinit_function_import_code (line 11124) | static int __Pyx_modinit_function_import_code(void) {
function __Pyx_PyMODINIT_FUNC (line 11153) | __Pyx_PyMODINIT_FUNC PyInit__mask(void)
function CYTHON_SMALL_CODE (line 11158) | static CYTHON_SMALL_CODE int __Pyx_check_single_interpreter(void) {
function CYTHON_SMALL_CODE (line 11181) | static CYTHON_SMALL_CODE int __Pyx_copy_spec_to_module(PyObject *spec, P...
function CYTHON_SMALL_CODE (line 11196) | static CYTHON_SMALL_CODE PyObject* __pyx_pymod_create(PyObject *spec, CY...
function __Pyx_RefNannyAPIStruct (line 11549) | static __Pyx_RefNannyAPIStruct *__Pyx_RefNannyImportAPI(const char *modn...
function CYTHON_INLINE (line 11566) | static CYTHON_INLINE PyObject* __Pyx_PyObject_GetAttrStr(PyObject* obj, ...
function PyObject (line 11579) | static PyObject *__Pyx_GetBuiltinName(PyObject *name) {
function __Pyx_RaiseDoubleKeywordsError (line 11593) | static void __Pyx_RaiseDoubleKeywordsError(
function __Pyx_ParseOptionalKeywords (line 11607) | static int __Pyx_ParseOptionalKeywords(
function __Pyx_RaiseArgtupleInvalid (line 11709) | static void __Pyx_RaiseArgtupleInvalid(
function CYTHON_INLINE (line 11735) | static CYTHON_INLINE int __Pyx_PyBytes_Equals(PyObject* s1, PyObject* s2...
function CYTHON_INLINE (line 11782) | static CYTHON_INLINE int __Pyx_PyUnicode_Equals(PyObject* s1, PyObject* ...
function CYTHON_INLINE (line 11885) | static CYTHON_INLINE PyObject * __Pyx_PyCFunction_FastCall(PyObject *fun...
function PyObject (line 11908) | static PyObject* __Pyx_PyFunction_FastCallNoKw(PyCodeObject *co, PyObjec...
function CYTHON_INLINE (line 12027) | static CYTHON_INLINE PyObject* __Pyx_PyObject_Call(PyObject *func, PyObj...
function CYTHON_INLINE (line 12047) | static CYTHON_INLINE PyObject* __Pyx_PyObject_CallMethO(PyObject *func, ...
function PyObject (line 12067) | static PyObject* __Pyx__PyObject_CallOneArg(PyObject *func, PyObject *ar...
function CYTHON_INLINE (line 12077) | static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func,...
function CYTHON_INLINE (line 12095) | static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func,...
function CYTHON_INLINE (line 12107) | static CYTHON_INLINE void __Pyx_ErrRestoreInState(PyThreadState *tstate,...
function CYTHON_INLINE (line 12119) | static CYTHON_INLINE void __Pyx_ErrFetchInState(PyThreadState *tstate, P...
function __Pyx_Raise (line 12131) | static void __Pyx_Raise(PyObject *type, PyObject *value, PyObject *tb,
function __Pyx_Raise (line 12182) | static void __Pyx_Raise(PyObject *type, PyObject *value, PyObject *tb, P...
function CYTHON_INLINE (line 12289) | static CYTHON_INLINE int __Pyx_TypeTest(PyObject *obj, PyTypeObject *typ...
function __Pyx__ArgTypeTest (line 12302) | static int __Pyx__ArgTypeTest(PyObject *obj, PyTypeObject *type, const c...
function PyObject (line 12324) | static PyObject* __Pyx_PyInt_AddObjC(PyObject *op1, PyObject *op2, CYTHO...
function PyObject (line 12448) | static PyObject *__Pyx_PyDict_GetItem(PyObject *d, PyObject* key) {
function PyObject (line 12471) | static PyObject *__Pyx_GetItemInt_Generic(PyObject *o, PyObject* j) {
function CYTHON_INLINE (line 12478) | static CYTHON_INLINE PyObject *__Pyx_GetItemInt_List_Fast(PyObject *o, P...
function CYTHON_INLINE (line 12496) | static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Tuple_Fast(PyObject *o, ...
function CYTHON_INLINE (line 12514) | static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssi...
function CYTHON_INLINE (line 12558) | static CYTHON_INLINE int __Pyx_Is_Little_Endian(void)
function __Pyx_BufFmt_Init (line 12569) | static void __Pyx_BufFmt_Init(__Pyx_BufFmt_Context* ctx,
function __Pyx_BufFmt_ParseNumber (line 12596) | static int __Pyx_BufFmt_ParseNumber(const char** ts) {
function __Pyx_BufFmt_ExpectNumber (line 12611) | static int __Pyx_BufFmt_ExpectNumber(const char **ts) {
function __Pyx_BufFmt_RaiseUnexpectedChar (line 12618) | static void __Pyx_BufFmt_RaiseUnexpectedChar(char ch) {
function __Pyx_BufFmt_TypeCharToStandardSize (line 12646) | static size_t __Pyx_BufFmt_TypeCharToStandardSize(char ch, int is_comple...
function __Pyx_BufFmt_TypeCharToNativeSize (line 12664) | static size_t __Pyx_BufFmt_TypeCharToNativeSize(char ch, int is_complex) {
type __Pyx_st_short (line 12683) | typedef struct { char c; short x; } __Pyx_st_short;
type __Pyx_st_int (line 12684) | typedef struct { char c; int x; } __Pyx_st_int;
type __Pyx_st_long (line 12685) | typedef struct { char c; long x; } __Pyx_st_long;
type __Pyx_st_float (line 12686) | typedef struct { char c; float x; } __Pyx_st_float;
type __Pyx_st_double (line 12687) | typedef struct { char c; double x; } __Pyx_st_double;
type __Pyx_st_longdouble (line 12688) | typedef struct { char c; long double x; } __Pyx_st_longdouble;
type __Pyx_st_void_p (line 12689) | typedef struct { char c; void *x; } __Pyx_st_void_p;
type __Pyx_st_longlong (line 12691) | typedef struct { char c; PY_LONG_LONG x; } __Pyx_st_longlong;
function __Pyx_BufFmt_TypeCharToAlignment (line 12693) | static size_t __Pyx_BufFmt_TypeCharToAlignment(char ch, CYTHON_UNUSED in...
type __Pyx_pad_short (line 12715) | typedef struct { short x; char c; } __Pyx_pad_short;
type __Pyx_pad_int (line 12716) | typedef struct { int x; char c; } __Pyx_pad_int;
type __Pyx_pad_long (line 12717) | typedef struct { long x; char c; } __Pyx_pad_long;
type __Pyx_pad_float (line 12718) | typedef struct { float x; char c; } __Pyx_pad_float;
type __Pyx_pad_double (line 12719) | typedef struct { double x; char c; } __Pyx_pad_double;
type __Pyx_pad_longdouble (line 12720) | typedef struct { long double x; char c; } __Pyx_pad_longdouble;
type __Pyx_pad_void_p (line 12721) | typedef struct { void *x; char c; } __Pyx_pad_void_p;
type __Pyx_pad_longlong (line 12723) | typedef struct { PY_LONG_LONG x; char c; } __Pyx_pad_longlong;
function __Pyx_BufFmt_TypeCharToPadding (line 12725) | static size_t __Pyx_BufFmt_TypeCharToPadding(char ch, CYTHON_UNUSED int ...
function __Pyx_BufFmt_TypeCharToGroup (line 12743) | static char __Pyx_BufFmt_TypeCharToGroup(char ch, int is_complex) {
function __Pyx_BufFmt_RaiseExpected (line 12764) | static void __Pyx_BufFmt_RaiseExpected(__Pyx_BufFmt_Context* ctx) {
function __Pyx_BufFmt_ProcessTypeChunk (line 12788) | static int __Pyx_BufFmt_ProcessTypeChunk(__Pyx_BufFmt_Context* ctx) {
function PyObject (line 12890) | static PyObject *
function CYTHON_INLINE (line 13071) | static CYTHON_INLINE void __Pyx_SafeReleaseBuffer(Py_buffer* info) {
function __Pyx_ZeroBuffer (line 13076) | static void __Pyx_ZeroBuffer(Py_buffer* buf) {
function __Pyx__GetBufferAndValidate (line 13083) | static int __Pyx__GetBufferAndValidate(
function CYTHON_INLINE (line 13119) | static CYTHON_INLINE PY_UINT64_T __Pyx_get_tp_dict_version(PyObject *obj) {
function CYTHON_INLINE (line 13123) | static CYTHON_INLINE PY_UINT64_T __Pyx_get_object_dict_version(PyObject ...
function CYTHON_INLINE (line 13135) | static CYTHON_INLINE int __Pyx_object_dict_version_matches(PyObject* obj...
function CYTHON_INLINE (line 13147) | static CYTHON_INLINE PyObject *__Pyx__GetModuleGlobalName(PyObject *name)
function CYTHON_UNUSED (line 13179) | static CYTHON_UNUSED PyObject* __Pyx_PyObject_Call2Args(PyObject* functi...
function CYTHON_INLINE (line 13208) | static CYTHON_INLINE PyObject* __Pyx_PyInt_EqObjC(PyObject *op1, PyObjec...
function PyTypeObject (line 13275) | static PyTypeObject* __Pyx_FetchCommonType(PyTypeObject* type) {
function PyObject (line 13315) | static PyObject *
function __Pyx_CyFunction_set_doc (line 13335) | static int
function PyObject (line 13347) | static PyObject *
function __Pyx_CyFunction_set_name (line 13362) | static int
function PyObject (line 13382) | static PyObject *
function __Pyx_CyFunction_set_qualname (line 13388) | static int
function PyObject (line 13408) | static PyObject *
function PyObject (line 13418) | static PyObject *
function __Pyx_CyFunction_set_dict (line 13429) | static int
function PyObject (line 13449) | static PyObject *
function PyObject (line 13455) | static PyObject *
function PyObject (line 13461) | static PyObject *
function __Pyx_CyFunction_init_defaults (line 13468) | static int
function __Pyx_CyFunction_set_defaults (line 13490) | static int
function PyObject (line 13506) | static PyObject *
function __Pyx_CyFunction_set_kwdefaults (line 13520) | static int
function PyObject (line 13536) | static PyObject *
function __Pyx_CyFunction_set_annotations (line 13550) | static int
function PyObject (line 13566) | static PyObject *
function PyObject (line 13602) | static PyObject *
function PyObject (line 13620) | static PyObject *__Pyx_CyFunction_New(PyTypeObject *type, PyMethodDef *m...
function __Pyx_CyFunction_clear (line 13652) | static int
function __Pyx__CyFunction_dealloc (line 13677) | static void __Pyx__CyFunction_dealloc(__pyx_CyFunctionObject *m)
function __Pyx_CyFunction_dealloc (line 13684) | static void __Pyx_CyFunction_dealloc(__pyx_CyFunctionObject *m)
function __Pyx_CyFunction_traverse (line 13689) | static int __Pyx_CyFunction_traverse(__pyx_CyFunctionObject *m, visitpro...
function PyObject (line 13710) | static PyObject *__Pyx_CyFunction_descr_get(PyObject *func, PyObject *ob...
function PyObject (line 13726) | static PyObject*
function PyObject (line 13737) | static PyObject * __Pyx_CyFunction_CallMethod(PyObject *func, PyObject *...
function CYTHON_INLINE (line 13791) | static CYTHON_INLINE PyObject *__Pyx_CyFunction_Call(PyObject *func, PyO...
function PyObject (line 13794) | static PyObject *__Pyx_CyFunction_CallAsMethod(PyObject *func, PyObject ...
function __pyx_CyFunction_init (line 13880) | static int __pyx_CyFunction_init(void) {
function CYTHON_INLINE (line 13887) | static CYTHON_INLINE void *__Pyx_CyFunction_InitDefaults(PyObject *func,...
function CYTHON_INLINE (line 13896) | static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsTuple(PyObject *fu...
function CYTHON_INLINE (line 13901) | static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsKwDict(PyObject *f...
function CYTHON_INLINE (line 13906) | static CYTHON_INLINE void __Pyx_CyFunction_SetAnnotationsDict(PyObject *...
function __Pyx_RaiseBufferFallbackError (line 13913) | static void __Pyx_RaiseBufferFallbackError(void) {
function CYTHON_INLINE (line 13919) | static CYTHON_INLINE Py_ssize_t __Pyx_div_Py_ssize_t(Py_ssize_t a, Py_ss...
function __Pyx_RaiseBufferIndexError (line 13927) | static void __Pyx_RaiseBufferIndexError(int axis) {
function CYTHON_INLINE (line 13933) | static CYTHON_INLINE void __Pyx_RaiseTooManyValuesError(Py_ssize_t expec...
function CYTHON_INLINE (line 13939) | static CYTHON_INLINE void __Pyx_RaiseNeedMoreValuesError(Py_ssize_t inde...
function CYTHON_INLINE (line 13946) | static CYTHON_INLINE void __Pyx_RaiseNoneNotIterableError(void) {
function _PyErr_StackItem (line 13952) | static _PyErr_StackItem *
function CYTHON_INLINE (line 13967) | static CYTHON_INLINE void __Pyx__ExceptionSave(PyThreadState *tstate, Py...
function CYTHON_INLINE (line 13982) | static CYTHON_INLINE void __Pyx__ExceptionReset(PyThreadState *tstate, P...
function __Pyx_PyErr_ExceptionMatchesTuple (line 14008) | static int __Pyx_PyErr_ExceptionMatchesTuple(PyObject *exc_type, PyObjec...
function CYTHON_INLINE (line 14021) | static CYTHON_INLINE int __Pyx_PyErr_ExceptionMatchesInState(PyThreadSta...
function __Pyx_GetException (line 14035) | static int __Pyx_GetException(PyObject **type, PyObject **value, PyObjec...
function PyObject (line 14107) | static PyObject *__Pyx_RaiseGenericGetAttributeError(PyTypeObject *tp, P...
function CYTHON_INLINE (line 14118) | static CYTHON_INLINE PyObject* __Pyx_PyObject_GenericGetAttrNoDict(PyObj...
function PyObject (line 14147) | static PyObject* __Pyx_PyObject_GenericGetAttr(PyObject* obj, PyObject* ...
function __Pyx_setup_reduce_is_named (line 14156) | static int __Pyx_setup_reduce_is_named(PyObject* meth, PyObject* name) {
function __Pyx_setup_reduce (line 14172) | static int __Pyx_setup_reduce(PyObject* type_obj) {
function PyTypeObject (line 14234) | static PyTypeObject *__Pyx_ImportType(PyObject *module, const char *modu...
function PyObject (line 14293) | static PyObject *__Pyx_Import(PyObject *name, PyObject *from_list, int l...
function __Pyx_CLineForTraceback (line 14359) | static int __Pyx_CLineForTraceback(PyThreadState *tstate, int c_line) {
function __pyx_bisect_code_objects (line 14400) | static int __pyx_bisect_code_objects(__Pyx_CodeObjectCacheEntry* entries...
function PyCodeObject (line 14421) | static PyCodeObject *__pyx_find_code_object(int code_line) {
function __pyx_insert_code_object (line 14435) | static void __pyx_insert_code_object(int code_line, PyCodeObject* code_o...
function PyCodeObject (line 14483) | static PyCodeObject* __Pyx_CreateCodeObjectForTraceback(
function __Pyx_AddTraceback (line 14535) | static void __Pyx_AddTraceback(const char *funcname, int c_line,
function __Pyx_GetBuffer (line 14565) | static int __Pyx_GetBuffer(PyObject *obj, Py_buffer *view, int flags) {
function __Pyx_ReleaseBuffer (line 14571) | static void __Pyx_ReleaseBuffer(Py_buffer *view) {
function CYTHON_INLINE (line 14609) | static CYTHON_INLINE PyObject* __Pyx_PyInt_From_siz(siz value) {
function CYTHON_INLINE (line 14640) | static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) {
function CYTHON_INLINE (line 14671) | static CYTHON_INLINE PyObject* __Pyx_PyInt_From_Py_intptr_t(Py_intptr_t ...
function CYTHON_INLINE (line 14704) | static CYTHON_INLINE __pyx_t_float_complex __pyx_t_float_complex_from_pa...
function CYTHON_INLINE (line 14708) | static CYTHON_INLINE __pyx_t_float_complex __pyx_t_float_complex_from_pa...
function CYTHON_INLINE (line 14713) | static CYTHON_INLINE __pyx_t_float_complex __pyx_t_float_complex_from_pa...
function CYTHON_INLINE (line 14724) | static CYTHON_INLINE int __Pyx_c_eq_float(__pyx_t_float_complex a, __pyx...
function CYTHON_INLINE (line 14727) | static CYTHON_INLINE __pyx_t_float_complex __Pyx_c_sum_float(__pyx_t_flo...
function CYTHON_INLINE (line 14733) | static CYTHON_INLINE __pyx_t_float_complex __Pyx_c_diff_float(__pyx_t_fl...
function CYTHON_INLINE (line 14739) | static CYTHON_INLINE __pyx_t_float_complex __Pyx_c_prod_float(__pyx_t_fl...
function CYTHON_INLINE (line 14746) | static CYTHON_INLINE __pyx_t_float_complex __Pyx_c_quot_float(__pyx_t_fl...
function CYTHON_INLINE (line 14766) | static CYTHON_INLINE __pyx_t_float_complex __Pyx_c_quot_float(__pyx_t_fl...
function CYTHON_INLINE (line 14777) | static CYTHON_INLINE __pyx_t_float_complex __Pyx_c_neg_float(__pyx_t_flo...
function CYTHON_INLINE (line 14783) | static CYTHON_INLINE int __Pyx_c_is_zero_float(__pyx_t_float_complex a) {
function CYTHON_INLINE (line 14786) | static CYTHON_INLINE __pyx_t_float_complex __Pyx_c_conj_float(__pyx_t_fl...
function CYTHON_INLINE (line 14793) | static CYTHON_INLINE float __Pyx_c_abs_float(__pyx_t_float_complex z) {
function CYTHON_INLINE (line 14800) | static CYTHON_INLINE __pyx_t_float_complex __Pyx_c_pow_float(__pyx_t_flo...
function CYTHON_INLINE (line 14859) | static CYTHON_INLINE __pyx_t_double_complex __pyx_t_double_complex_from_...
function CYTHON_INLINE (line 14863) | static CYTHON_INLINE __pyx_t_double_complex __pyx_t_double_complex_from_...
function CYTHON_INLINE (line 14868) | static CYTHON_INLINE __pyx_t_double_complex __pyx_t_double_complex_from_...
function CYTHON_INLINE (line 14879) | static CYTHON_INLINE int __Pyx_c_eq_double(__pyx_t_double_complex a, __p...
function CYTHON_INLINE (line 14882) | static CYTHON_INLINE __pyx_t_double_complex __Pyx_c_sum_double(__pyx_t_d...
function CYTHON_INLINE (line 14888) | static CYTHON_INLINE __pyx_t_double_complex __Pyx_c_diff_double(__pyx_t_...
function CYTHON_INLINE (line 14894) | static CYTHON_INLINE __pyx_t_double_complex __Pyx_c_prod_double(__pyx_t_...
function CYTHON_INLINE (line 14901) | static CYTHON_INLINE __pyx_t_double_complex __Pyx_c_quot_double(__pyx_t_...
function CYTHON_INLINE (line 14921) | static CYTHON_INLINE __pyx_t_double_complex __Pyx_c_quot_double(__pyx_t_...
function CYTHON_INLINE (line 14932) | static CYTHON_INLINE __pyx_t_double_complex __Pyx_c_neg_double(__pyx_t_d...
function CYTHON_INLINE (line 14938) | static CYTHON_INLINE int __Pyx_c_is_zero_double(__pyx_t_double_complex a) {
function CYTHON_INLINE (line 14941) | static CYTHON_INLINE __pyx_t_double_complex __Pyx_c_conj_double(__pyx_t_...
function CYTHON_INLINE (line 14948) | static CYTHON_INLINE double __Pyx_c_abs_double(__pyx_t_double_complex z) {
function CYTHON_INLINE (line 14955) | static CYTHON_INLINE __pyx_t_double_complex __Pyx_c_pow_double(__pyx_t_d...
function CYTHON_INLINE (line 15012) | static CYTHON_INLINE PyObject* __Pyx_PyInt_From_int(int value) {
function CYTHON_INLINE (line 15043) | static CYTHON_INLINE PyObject* __Pyx_PyInt_From_enum__NPY_TYPES(enum NPY...
function __Pyx_InBases (line 15831) | static int __Pyx_InBases(PyTypeObject *a, PyTypeObject *b) {
function CYTHON_INLINE (line 15839) | static CYTHON_INLINE int __Pyx_IsSubtype(PyTypeObject *a, PyTypeObject *...
function __Pyx_inner_PyErr_GivenExceptionMatches2 (line 15855) | static int __Pyx_inner_PyErr_GivenExceptionMatches2(PyObject *err, PyObj...
function CYTHON_INLINE (line 15877) | static CYTHON_INLINE int __Pyx_inner_PyErr_GivenExceptionMatches2(PyObje...
function __Pyx_PyErr_GivenExceptionMatchesTuple (line 15885) | static int __Pyx_PyErr_GivenExceptionMatchesTuple(PyObject *exc_type, Py...
function CYTHON_INLINE (line 15906) | static CYTHON_INLINE int __Pyx_PyErr_GivenExceptionMatches(PyObject *err...
function CYTHON_INLINE (line 15918) | static CYTHON_INLINE int __Pyx_PyErr_GivenExceptionMatches2(PyObject *er...
function __Pyx_check_binary_version (line 15930) | static int __Pyx_check_binary_version(void) {
function __Pyx_InitStrings (line 15946) | static int __Pyx_InitStrings(__Pyx_StringTabEntry *t) {
function CYTHON_INLINE (line 15978) | static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_...
function CYTHON_INLINE (line 15981) | static CYTHON_INLINE const char* __Pyx_PyObject_AsString(PyObject* o) {
function CYTHON_INLINE (line 16008) | static CYTHON_INLINE const char* __Pyx_PyUnicode_AsStringAndSize(PyObjec...
function CYTHON_INLINE (line 16050) | static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject* x) {
function CYTHON_INLINE (line 16055) | static CYTHON_INLINE int __Pyx_PyObject_IsTrueAndDecref(PyObject* x) {
function PyObject (line 16062) | static PyObject* __Pyx_PyNumber_IntOrLongWrongResultType(PyObject* resul...
function CYTHON_INLINE (line 16131) | static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) {
function CYTHON_INLINE (line 16193) | static CYTHON_INLINE PyObject * __Pyx_PyBool_FromLong(long b) {
function CYTHON_INLINE (line 16196) | static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) {
FILE: lib/pycocotools/coco.py
class COCO (line 70) | class COCO:
method __init__ (line 71) | def __init__(self, annotation_file=None):
method createIndex (line 93) | def createIndex(self):
method info (line 131) | def info(self):
method getAnnIds (line 139) | def getAnnIds(self, imgIds=[], catIds=[], areaRng=[], iscrowd=None):
method getCatIds (line 169) | def getCatIds(self, catNms=[], supNms=[], catIds=[]):
method getImgIds (line 191) | def getImgIds(self, imgIds=[], catIds=[]):
method loadAnns (line 212) | def loadAnns(self, ids=[]):
method loadCats (line 223) | def loadCats(self, ids=[]):
method loadImgs (line 234) | def loadImgs(self, ids=[]):
method showAnns (line 245) | def showAnns(self, anns):
method loadRes (line 291) | def loadRes(self, resFile):
method download (line 339) | def download(self, tarDir=None, imgIds=[]):
FILE: lib/pycocotools/cocoeval.py
class COCOeval (line 8) | class COCOeval:
method __init__ (line 58) | def __init__(self, cocoGt=None, cocoDt=None, iouType='segm'):
method _prepare (line 83) | def _prepare(self):
method evaluate (line 120) | def evaluate(self):
method computeIoU (line 162) | def computeIoU(self, imgId, catId):
method computeOks (line 191) | def computeOks(self, imgId, catId):
method evaluateImg (line 234) | def evaluateImg(self, imgId, catId, aRng, maxDet):
method accumulate (line 314) | def accumulate(self, p = None):
method summarize (line 415) | def summarize(self):
method __str__ (line 488) | def __str__(self):
class Params (line 491) | class Params:
method setDetParams (line 495) | def setDetParams(self):
method setKpParams (line 506) | def setKpParams(self):
method __init__ (line 517) | def __init__(self, iouType='segm'):
FILE: lib/pycocotools/maskApi.c
function uint (line 11) | uint umin( uint a, uint b ) { return (a<b) ? a : b; }
function uint (line 12) | uint umax( uint a, uint b ) { return (a>b) ? a : b; }
function rleInit (line 14) | void rleInit( RLE *R, siz h, siz w, siz m, uint *cnts ) {
function rleFree (line 19) | void rleFree( RLE *R ) {
function rlesInit (line 23) | void rlesInit( RLE **R, siz n ) {
function rlesFree (line 28) | void rlesFree( RLE **R, siz n ) {
function rleEncode (line 32) | void rleEncode( RLE *R, const byte *M, siz h, siz w, siz n ) {
function rleDecode (line 43) | void rleDecode( const RLE *R, byte *M, siz n ) {
function rleMerge (line 49) | void rleMerge( const RLE *R, RLE *M, siz n, bool intersect ) {
function rleArea (line 72) | void rleArea( const RLE *R, siz n, uint *a ) {
function rleIou (line 77) | void rleIou( RLE *dt, RLE *gt, siz m, siz n, byte *iscrowd, double *o ) {
function bbIou (line 98) | void bbIou( BB dt, BB gt, siz m, siz n, byte *iscrowd, double *o ) {
function rleToBbox (line 111) | void rleToBbox( const RLE *R, BB bb, siz n ) {
function rleFrBbox (line 126) | void rleFrBbox( RLE *R, const BB bb, siz h, siz w, siz n ) {
function uintCompare (line 135) | int uintCompare(const void *a, const void *b) {
function rleFrPoly (line 139) | void rleFrPoly( RLE *R, const double *xy, siz k, siz h, siz w ) {
function rleFrString (line 195) | void rleFrString( RLE *R, char *s, siz h, siz w ) {
FILE: lib/pycocotools/maskApi.h
type uint (line 10) | typedef unsigned int uint;
type siz (line 11) | typedef unsigned long siz;
type byte (line 12) | typedef unsigned char byte;
type RLE (line 14) | typedef struct { siz h, w, m; uint *cnts; } RLE;
FILE: lib/roi_data_layer/minibatch.py
function get_minibatch (line 19) | def get_minibatch(roidb, num_classes, random_scale_inds):
function _get_image_blob (line 56) | def _get_image_blob(roidb, scale_inds):
FILE: lib/roi_data_layer/roibatchLoader.py
class roibatchLoader (line 23) | class roibatchLoader(data.Dataset):
method __init__ (line 24) | def __init__(self, roidb, ratio_list, ratio_index, batch_size, num_cla...
method __getitem__ (line 58) | def __getitem__(self, index):
method __len__ (line 217) | def __len__(self):
FILE: lib/roi_data_layer/roidb.py
function prepare_roidb (line 14) | def prepare_roidb(imdb):
function update_keyvalue (line 49) | def update_keyvalue(rdb, idx):
function filter_class_roidb (line 61) | def filter_class_roidb(roidb, shot, imdb):
function rank_roidb_ratio (line 133) | def rank_roidb_ratio(roidb):
function filter_roidb (line 159) | def filter_roidb(roidb):
function combined_roidb (line 172) | def combined_roidb(imdb_names, training=True):
FILE: lib/setup.py
function find_in_path (line 17) | def find_in_path(name, path):
function customize_compiler_for_nvcc (line 69) | def customize_compiler_for_nvcc(self):
class custom_build_ext (line 109) | class custom_build_ext(build_ext):
method build_extensions (line 110) | def build_extensions(self):
FILE: test_metarcnn.py
function parse_args (line 45) | def parse_args():
FILE: train_metarcnn.py
function parse_args (line 34) | def parse_args():
class sampler (line 127) | class sampler(Sampler):
method __init__ (line 128) | def __init__(self, train_size, batch_size):
method __iter__ (line 138) | def __iter__(self):
method __len__ (line 148) | def __len__(self):
Condensed preview — 117 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (1,197K chars).
[
{
"path": ".gitignore",
"chars": 81,
"preview": "attent*\nlog*\n*.pth\ncheckpoint\n1st\nallmetaclass\nbase*\noutput*\nsplit*\nresult*\n*.sh\n"
},
{
"path": "README.md",
"chars": 4603,
"preview": "## Meta R-CNN : Towards General Solver for Instance-level Low-shot Learning.\r\n\r\nCode for reproducing the results in the "
},
{
"path": "_init_paths.py",
"chars": 451,
"preview": "import os.path as osp\nimport sys\nimport os\n\nif os.listdir('data/cache/'):\n os.system('rm data/cache/*')\n\ndef add_path"
},
{
"path": "cfgs/res101_ms.yml",
"chars": 411,
"preview": "EXP_DIR: res101\nTRAIN:\n HAS_RPN: True\n BBOX_NORMALIZE_TARGETS_PRECOMPUTED: True\n RPN_POSITIVE_OVERLAP: 0.7\n RPN_BATC"
},
{
"path": "cfgs/res50.yml",
"chars": 410,
"preview": "EXP_DIR: res50\nTRAIN:\n HAS_RPN: True\n BBOX_NORMALIZE_TARGETS_PRECOMPUTED: True\n RPN_POSITIVE_OVERLAP: 0.7\n RPN_BATCH"
},
{
"path": "lib/datasets/VOCdevkit-matlab-wrapper/get_voc_opts.m",
"chars": 231,
"preview": "function VOCopts = get_voc_opts(path)\n\ntmp = pwd;\ncd(path);\ntry\n addpath('VOCcode');\n VOCinit;\ncatch\n rmpath('VOCcode"
},
{
"path": "lib/datasets/VOCdevkit-matlab-wrapper/voc_eval.m",
"chars": 1328,
"preview": "function res = voc_eval(path, comp_id, test_set, output_dir)\n\nVOCopts = get_voc_opts(path);\nVOCopts.testset = test_set;\n"
},
{
"path": "lib/datasets/VOCdevkit-matlab-wrapper/xVOCap.m",
"chars": 258,
"preview": "function ap = xVOCap(rec,prec)\r\n% From the PASCAL VOC 2011 devkit\r\n\r\nmrec=[0 ; rec ; 1];\r\nmpre=[0 ; prec ; 0];\r\nfor i=nu"
},
{
"path": "lib/datasets/__init__.py",
"chars": 247,
"preview": "# --------------------------------------------------------\n# Fast R-CNN\n# Copyright (c) 2015 Microsoft\n# Licensed under "
},
{
"path": "lib/datasets/coco.py",
"chars": 12182,
"preview": "# --------------------------------------------------------\n# Fast/er R-CNN\n# Licensed under The MIT License [see LICENSE"
},
{
"path": "lib/datasets/ds_utils.py",
"chars": 1402,
"preview": "# --------------------------------------------------------\n# Fast/er R-CNN\n# Licensed under The MIT License [see LICENSE"
},
{
"path": "lib/datasets/factory.py",
"chars": 1444,
"preview": "# --------------------------------------------------------\n# Fast R-CNN\n# Copyright (c) 2015 Microsoft\n# Licensed under "
},
{
"path": "lib/datasets/imdb.py",
"chars": 9229,
"preview": "# --------------------------------------------------------\n# Fast R-CNN\n# Copyright (c) 2015 Microsoft\n# Licensed under "
},
{
"path": "lib/datasets/metadata.py",
"chars": 5103,
"preview": "# --------------------------------------------------------\n# Pytorch Meta R-CNN\n# Written by Anny Xu, Xiaopeng Yan, base"
},
{
"path": "lib/datasets/pascal_voc.py",
"chars": 16365,
"preview": "from __future__ import print_function\nfrom __future__ import absolute_import\n# -----------------------------------------"
},
{
"path": "lib/datasets/pascal_voc_rbg.py",
"chars": 13812,
"preview": "# --------------------------------------------------------\n# Fast R-CNN\n# Copyright (c) 2015 Microsoft\n# Licensed under "
},
{
"path": "lib/datasets/tools/compute_prior.py",
"chars": 2778,
"preview": "import numpy as np\nimport pickle\nimport os\nimport sys\n\nNUM_ATTR_REL = 200\ndef cout_w(prob, num=NUM_ATTR_REL,dim=1):\n "
},
{
"path": "lib/datasets/tools/mcg_munge.py",
"chars": 1490,
"preview": "from __future__ import print_function\nimport os\nimport sys\n\n\"\"\"Hacky tool to convert file system layout of MCG boxes dow"
},
{
"path": "lib/datasets/voc_eval.py",
"chars": 6729,
"preview": "# --------------------------------------------------------\n# Fast/er R-CNN\n# Licensed under The MIT License [see LICENSE"
},
{
"path": "lib/model/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "lib/model/faster_rcnn/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "lib/model/faster_rcnn/faster_rcnn.py",
"chars": 12711,
"preview": "# --------------------------------------------------------\n# Pytorch Meta R-CNN\n# Written by Anny Xu, Xiaopeng Yan, base"
},
{
"path": "lib/model/faster_rcnn/resnet.py",
"chars": 10141,
"preview": "# --------------------------------------------------------\n# Pytorch Meta R-CNN\n# Written by Anny Xu, Xiaopeng Yan, base"
},
{
"path": "lib/model/faster_rcnn/trail.py",
"chars": 15360,
"preview": "# --------------------------------------------------------\n# Pytorch multi-GPU Faster R-CNN\n# Licensed under The MIT Lic"
},
{
"path": "lib/model/faster_rcnn/vgg16.py",
"chars": 2018,
"preview": "# --------------------------------------------------------\n# Tensorflow Faster R-CNN\n# Licensed under The MIT License [s"
},
{
"path": "lib/model/nms/.gitignore",
"chars": 15,
"preview": "*.c\n*.cpp\n*.so\n"
},
{
"path": "lib/model/nms/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "lib/model/nms/_ext/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "lib/model/nms/_ext/nms/__init__.py",
"chars": 377,
"preview": "\nfrom torch.utils.ffi import _wrap_function\nfrom ._nms import lib as _lib, ffi as _ffi\n\n__all__ = []\ndef _import_symbols"
},
{
"path": "lib/model/nms/build.py",
"chars": 850,
"preview": "from __future__ import print_function\nimport os\nimport torch\nfrom torch.utils.ffi import create_extension\n\n#this_file = "
},
{
"path": "lib/model/nms/nms_cpu.py",
"chars": 5777,
"preview": "from __future__ import absolute_import\n\nimport numpy as np\nimport torch\n\ndef nms_cpu(dets, thresh):\n dets = dets.cpu("
},
{
"path": "lib/model/nms/nms_gpu.py",
"chars": 299,
"preview": "from __future__ import absolute_import\nimport torch\nimport numpy as np\nfrom ._ext import nms\nimport pdb\n\ndef nms_gpu(det"
},
{
"path": "lib/model/nms/nms_kernel.cu",
"chars": 5064,
"preview": "// ------------------------------------------------------------------\n// Faster R-CNN\n// Copyright (c) 2015 Microsoft\n//"
},
{
"path": "lib/model/nms/nms_wrapper.py",
"chars": 757,
"preview": "# --------------------------------------------------------\n# Fast R-CNN\n# Copyright (c) 2015 Microsoft\n# Licensed under "
},
{
"path": "lib/model/nms/src/nms_cuda.h",
"chars": 272,
"preview": "// int nms_cuda(THCudaTensor *keep_out, THCudaTensor *num_out,\n// THCudaTensor *boxes_host, THCudaTensor *nm"
},
{
"path": "lib/model/nms/src/nms_cuda_kernel.cu",
"chars": 5623,
"preview": "// ------------------------------------------------------------------\n// Faster R-CNN\n// Copyright (c) 2015 Microsoft\n//"
},
{
"path": "lib/model/nms/src/nms_cuda_kernel.h",
"chars": 206,
"preview": "#ifdef __cplusplus\nextern \"C\" {\n#endif\n\nvoid nms_cuda_compute(int* keep_out, int *num_out, float* boxes_host, int boxes_"
},
{
"path": "lib/model/roi_align/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "lib/model/roi_align/_ext/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "lib/model/roi_align/_ext/roi_align/__init__.py",
"chars": 383,
"preview": "\nfrom torch.utils.ffi import _wrap_function\nfrom ._roi_align import lib as _lib, ffi as _ffi\n\n__all__ = []\ndef _import_s"
},
{
"path": "lib/model/roi_align/build.py",
"chars": 872,
"preview": "from __future__ import print_function\nimport os\nimport torch\nfrom torch.utils.ffi import create_extension\n\n# sources = ["
},
{
"path": "lib/model/roi_align/functions/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "lib/model/roi_align/functions/roi_align.py",
"chars": 1760,
"preview": "import torch\nfrom torch.autograd import Function\nfrom .._ext import roi_align\n\n\n# TODO use save_for_backward instead\ncla"
},
{
"path": "lib/model/roi_align/modules/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "lib/model/roi_align/modules/roi_align.py",
"chars": 1672,
"preview": "from torch.nn.modules.module import Module\nfrom torch.nn.functional import avg_pool2d, max_pool2d\nfrom ..functions.roi_a"
},
{
"path": "lib/model/roi_align/src/roi_align_cuda.c",
"chars": 2425,
"preview": "#include <THC/THC.h>\n#include <math.h>\n#include \"roi_align_kernel.h\"\n\nextern THCState *state;\n\nint roi_align_forward_cud"
},
{
"path": "lib/model/roi_align/src/roi_align_cuda.h",
"chars": 369,
"preview": "int roi_align_forward_cuda(int aligned_height, int aligned_width, float spatial_scale,\n THCudaTen"
},
{
"path": "lib/model/roi_align/src/roi_align_kernel.cu",
"chars": 7732,
"preview": "#ifdef __cplusplus\nextern \"C\" {\n#endif\n\n#include <stdio.h>\n#include <math.h>\n#include <float.h>\n#include \"roi_align_kern"
},
{
"path": "lib/model/roi_align/src/roi_align_kernel.h",
"chars": 1263,
"preview": "#ifndef _ROI_ALIGN_KERNEL\n#define _ROI_ALIGN_KERNEL\n\n#ifdef __cplusplus\nextern \"C\" {\n#endif\n\n__global__ void ROIAlignFor"
},
{
"path": "lib/model/roi_crop/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "lib/model/roi_crop/_ext/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "lib/model/roi_crop/_ext/roi_crop/__init__.py",
"chars": 382,
"preview": "\nfrom torch.utils.ffi import _wrap_function\nfrom ._roi_crop import lib as _lib, ffi as _ffi\n\n__all__ = []\ndef _import_sy"
},
{
"path": "lib/model/roi_crop/build.py",
"chars": 881,
"preview": "from __future__ import print_function\nimport os\nimport torch\nfrom torch.utils.ffi import create_extension\n\n#this_file = "
},
{
"path": "lib/model/roi_crop/functions/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "lib/model/roi_crop/functions/crop_resize.py",
"chars": 1545,
"preview": "# functions/add.py\nimport torch\nfrom torch.autograd import Function\nfrom .._ext import roi_crop\nfrom cffi import FFI\nffi"
},
{
"path": "lib/model/roi_crop/functions/gridgen.py",
"chars": 2233,
"preview": "# functions/add.py\nimport torch\nfrom torch.autograd import Function\nimport numpy as np\n\n\nclass AffineGridGenFunction(Fun"
},
{
"path": "lib/model/roi_crop/functions/roi_crop.py",
"chars": 1002,
"preview": "# functions/add.py\nimport torch\nfrom torch.autograd import Function\nfrom .._ext import roi_crop\nimport pdb\n\nclass RoICro"
},
{
"path": "lib/model/roi_crop/modules/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "lib/model/roi_crop/modules/gridgen.py",
"chars": 16532,
"preview": "from torch.nn.modules.module import Module\nimport torch\nfrom torch.autograd import Variable\nimport numpy as np\nfrom ..fu"
},
{
"path": "lib/model/roi_crop/modules/roi_crop.py",
"chars": 287,
"preview": "from torch.nn.modules.module import Module\nfrom ..functions.roi_crop import RoICropFunction\n\nclass _RoICrop(Module):\n "
},
{
"path": "lib/model/roi_crop/src/roi_crop.c",
"chars": 22258,
"preview": "#include <TH/TH.h>\n#include <stdbool.h>\n#include <stdio.h>\n\n#define real float\n\nint BilinearSamplerBHWD_updateOutput(THF"
},
{
"path": "lib/model/roi_crop/src/roi_crop.h",
"chars": 659,
"preview": "int BilinearSamplerBHWD_updateOutput(THFloatTensor *inputImages, THFloatTensor *grids, THFloatTensor *output);\n\nint Bili"
},
{
"path": "lib/model/roi_crop/src/roi_crop_cuda.c",
"chars": 6735,
"preview": "#include <THC/THC.h>\n#include <stdbool.h>\n#include <stdio.h>\n#include \"roi_crop_cuda_kernel.h\"\n\n#define real float\n\n// t"
},
{
"path": "lib/model/roi_crop/src/roi_crop_cuda.h",
"chars": 481,
"preview": "// Bilinear sampling is done in BHWD (coalescing is not obvious in BDHW)\n// we assume BHWD format in inputImages\n// we a"
},
{
"path": "lib/model/roi_crop/src/roi_crop_cuda_kernel.cu",
"chars": 17168,
"preview": "#include <stdbool.h>\n#include <stdio.h>\n#include \"roi_crop_cuda_kernel.h\"\n\n#define real float\n\n// Bilinear sampling is d"
},
{
"path": "lib/model/roi_crop/src/roi_crop_cuda_kernel.h",
"chars": 2816,
"preview": "#ifdef __cplusplus\nextern \"C\" {\n#endif\n\n\nint BilinearSamplerBHWD_updateOutput_cuda_kernel(/*output->size[3]*/int oc,\n "
},
{
"path": "lib/model/roi_pooling/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "lib/model/roi_pooling/_ext/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "lib/model/roi_pooling/_ext/roi_pooling/__init__.py",
"chars": 385,
"preview": "\nfrom torch.utils.ffi import _wrap_function\nfrom ._roi_pooling import lib as _lib, ffi as _ffi\n\n__all__ = []\ndef _import"
},
{
"path": "lib/model/roi_pooling/build.py",
"chars": 848,
"preview": "from __future__ import print_function\nimport os\nimport torch\nfrom torch.utils.ffi import create_extension\n\n\nsources = ['"
},
{
"path": "lib/model/roi_pooling/functions/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "lib/model/roi_pooling/functions/roi_pool.py",
"chars": 1773,
"preview": "import torch\nfrom torch.autograd import Function\nfrom .._ext import roi_pooling\nimport pdb\n\nclass RoIPoolFunction(Functi"
},
{
"path": "lib/model/roi_pooling/modules/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "lib/model/roi_pooling/modules/roi_pool.py",
"chars": 524,
"preview": "from torch.nn.modules.module import Module\nfrom ..functions.roi_pool import RoIPoolFunction\n\n\nclass _RoIPooling(Module):"
},
{
"path": "lib/model/roi_pooling/src/roi_pooling.c",
"chars": 4104,
"preview": "#include <TH/TH.h>\n#include <math.h>\n\nint roi_pooling_forward(int pooled_height, int pooled_width, float spatial_scale,\n"
},
{
"path": "lib/model/roi_pooling/src/roi_pooling.h",
"chars": 178,
"preview": "int roi_pooling_forward(int pooled_height, int pooled_width, float spatial_scale,\n THFloatTensor "
},
{
"path": "lib/model/roi_pooling/src/roi_pooling_cuda.c",
"chars": 2836,
"preview": "#include <THC/THC.h>\n#include <math.h>\n#include \"roi_pooling_kernel.h\"\n\nextern THCState *state;\n\nint roi_pooling_forward"
},
{
"path": "lib/model/roi_pooling/src/roi_pooling_cuda.h",
"chars": 420,
"preview": "int roi_pooling_forward_cuda(int pooled_height, int pooled_width, float spatial_scale,\n THCudaTen"
},
{
"path": "lib/model/roi_pooling/src/roi_pooling_kernel.cu",
"chars": 9573,
"preview": "// #ifdef __cplusplus\n// extern \"C\" {\n// #endif\n\n#include <stdio.h>\n#include <vector>\n#include <math.h>\n#include <float."
},
{
"path": "lib/model/roi_pooling/src/roi_pooling_kernel.h",
"chars": 767,
"preview": "#ifndef _ROI_POOLING_KERNEL\n#define _ROI_POOLING_KERNEL\n\n#ifdef __cplusplus\nextern \"C\" {\n#endif\n\nint ROIPoolForwardLauch"
},
{
"path": "lib/model/rpn/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "lib/model/rpn/anchor_target_layer.py",
"chars": 8903,
"preview": "from __future__ import absolute_import\n# --------------------------------------------------------\n# Faster R-CNN\n# Copyr"
},
{
"path": "lib/model/rpn/bbox_transform.py",
"chars": 9288,
"preview": "# --------------------------------------------------------\n# Fast R-CNN\n# Copyright (c) 2015 Microsoft\n# Licensed under "
},
{
"path": "lib/model/rpn/generate_anchors.py",
"chars": 3248,
"preview": "from __future__ import print_function\n# --------------------------------------------------------\n# Faster R-CNN\n# Copyri"
},
{
"path": "lib/model/rpn/proposal_layer.py",
"chars": 7001,
"preview": "from __future__ import absolute_import\n# --------------------------------------------------------\n# Faster R-CNN\n# Copyr"
},
{
"path": "lib/model/rpn/proposal_layer_region.py",
"chars": 7199,
"preview": "from __future__ import absolute_import\n# --------------------------------------------------------\n# Faster R-CNN\n# Copyr"
},
{
"path": "lib/model/rpn/proposal_target_layer_cascade.py",
"chars": 9274,
"preview": "from __future__ import absolute_import\n# --------------------------------------------------------\n# Faster R-CNN\n# Copyr"
},
{
"path": "lib/model/rpn/proposal_target_layer_cascade_region.py",
"chars": 9977,
"preview": "from __future__ import absolute_import\n# --------------------------------------------------------\n# Faster R-CNN\n# Copyr"
},
{
"path": "lib/model/rpn/rpn.py",
"chars": 8248,
"preview": "from __future__ import absolute_import\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.aut"
},
{
"path": "lib/model/rpn/rpn_region.py",
"chars": 4338,
"preview": "from __future__ import absolute_import\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.aut"
},
{
"path": "lib/model/utils/.gitignore",
"chars": 15,
"preview": "*.c\n*.cpp\n*.so\n"
},
{
"path": "lib/model/utils/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "lib/model/utils/bbox.pyx",
"chars": 3431,
"preview": "# --------------------------------------------------------\n# Fast R-CNN\n# Copyright (c) 2015 Microsoft\n# Licensed under "
},
{
"path": "lib/model/utils/blob.py",
"chars": 1642,
"preview": "# --------------------------------------------------------\n# Fast R-CNN\n# Copyright (c) 2015 Microsoft\n# Licensed under "
},
{
"path": "lib/model/utils/config.py",
"chars": 13585,
"preview": "from __future__ import absolute_import\nfrom __future__ import division\nfrom __future__ import print_function\n\nimport os\n"
},
{
"path": "lib/model/utils/net_utils.py",
"chars": 8904,
"preview": "import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.autograd import Variable\nimport numpy as n"
},
{
"path": "lib/pycocotools/UPSTREAM_REV",
"chars": 80,
"preview": "https://github.com/pdollar/coco/commit/3ac47c77ebd5a1ed4254a98b7fbf2ef4765a3574\n"
},
{
"path": "lib/pycocotools/__init__.py",
"chars": 21,
"preview": "__author__ = 'tylin'\n"
},
{
"path": "lib/pycocotools/_mask.c",
"chars": 685570,
"preview": "/* Generated by Cython 0.29.12 */\n\n#define PY_SSIZE_T_CLEAN\n#include \"Python.h\"\n#ifndef Py_PYTHON_H\n #error Python he"
},
{
"path": "lib/pycocotools/_mask.pyx",
"chars": 10709,
"preview": "# distutils: language = c\n# distutils: sources = ../MatlabAPI/private/maskApi.c\n\n#**************************************"
},
{
"path": "lib/pycocotools/coco.py",
"chars": 15091,
"preview": "from __future__ import print_function\nfrom __future__ import absolute_import\n\n__author__ = 'tylin'\n__version__ = '1.0.1'"
},
{
"path": "lib/pycocotools/cocoeval.py",
"chars": 23832,
"preview": "import numpy as np\nimport datetime\nimport time\nfrom collections import defaultdict\nfrom . import mask as maskUtils\nimpor"
},
{
"path": "lib/pycocotools/license.txt",
"chars": 1533,
"preview": "Copyright (c) 2014, Piotr Dollar and Tsung-Yi Lin\nAll rights reserved.\n\nRedistribution and use in source and binary form"
},
{
"path": "lib/pycocotools/mask.py",
"chars": 4044,
"preview": "__author__ = 'tsungyi'\n\nfrom . import _mask\n\n# Interface for manipulating masks stored in RLE format.\n#\n# RLE is a simpl"
},
{
"path": "lib/pycocotools/maskApi.c",
"chars": 7704,
"preview": "/**************************************************************************\n* Microsoft COCO Toolbox. version 2.0\n*"
},
{
"path": "lib/pycocotools/maskApi.h",
"chars": 1928,
"preview": "/**************************************************************************\n* Microsoft COCO Toolbox. version 2.0\n*"
},
{
"path": "lib/roi_data_layer/__init__.py",
"chars": 248,
"preview": "# --------------------------------------------------------\n# Fast R-CNN\n# Copyright (c) 2015 Microsoft\n# Licensed under "
},
{
"path": "lib/roi_data_layer/minibatch.py",
"chars": 2946,
"preview": "# --------------------------------------------------------\n# Fast R-CNN\n# Copyright (c) 2015 Microsoft\n# Licensed under "
},
{
"path": "lib/roi_data_layer/roibatchLoader.py",
"chars": 8898,
"preview": "\n\"\"\"The data layer used during training to train a Fast R-CNN network.\n\"\"\"\n\nfrom __future__ import absolute_import\nfrom "
},
{
"path": "lib/roi_data_layer/roidb.py",
"chars": 7415,
"preview": "\"\"\"Transform a roidb into a trainable roidb by adding a bunch of metadata.\"\"\"\nfrom __future__ import absolute_import\nfro"
},
{
"path": "lib/setup.py",
"chars": 4802,
"preview": "from __future__ import print_function\n# --------------------------------------------------------\n# Fast R-CNN\n# Copyrigh"
},
{
"path": "test_metarcnn.py",
"chars": 16199,
"preview": "# --------------------------------------------------------\n# Pytorch Meta R-CNN\n# Written by Anny Xu, Xiaopeng Yan, base"
},
{
"path": "train_metarcnn.py",
"chars": 22335,
"preview": "# --------------------------------------------------------\n# Pytorch Meta R-CNN\n# Written by Anny Xu, Xiaopeng Yan, base"
}
]
// ... and 4 more files (download for full content)
About this extraction
This page contains the full source code of the yanxp/MetaR-CNN GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 117 files (1.1 MB), approximately 363.1k tokens, and a symbol index with 722 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.