[ { "path": "LICENSE", "content": "MIT License\n\nCopyright (c) 2021 Sartorius AG\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, { "path": "README.md", "content": "# LIVECell dataset\n\nThis document contains instructions of how to access the data associated with the submitted\nmanuscript \"LIVECell - A large-scale dataset for label-free live cell segmentation\" by Edlund et. al. 2021.\n\n## Background\nLight microscopy is a cheap, accessible, non-invasive modality that when combined with well-established\nprotocols of two-dimensional cell culture facilitates high-throughput quantitative imaging to study biological\nphenomena. Accurate segmentation of individual cells enables exploration of complex biological questions, but\nthis requires sophisticated imaging processing pipelines due to the low contrast and high object density.\nDeep learning-based methods are considered state-of-the-art for most computer vision problems but require vast\namounts of annotated data, for which there is no suitable resource available in the field of label-free cellular\nimaging. To address this gap we present LIVECell, a high-quality, manually annotated and expert-validated dataset\nthat is the largest of its kind to date, consisting of over 1.6 million cells from a diverse set of cell morphologies\nand culture densities. To further demonstrate its utility, we provide convolutional neural network-based models\ntrained and evaluated on LIVECell.\n\n## How to access LIVECell\n\nAll images in LIVECell are available following [this link](http://livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/images.zip) (requires 1.3 GB). Annotations for the different experiments are linked below.\nTo see a more details regarding benchmarks and how to use our models, see [this link](model/README.md).\n\n### LIVECell-wide train and evaluate\n\n| Annotation set | URL |\n| -------------------------- |:-------------:|\n| Training set | [link](http://livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell/livecell_coco_train.json) |\n| Validation set | [link](http://livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell/livecell_coco_val.json) |\n| Test set | [link](http://livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell/livecell_coco_test.json) |\n\n### Single cell-type experiments\n\n\n| Cell Type | Training set | Validation set | Test set |\n| ---------------|:-------------:|:--------------:|:--------:|\n| A172 | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/a172/train.json) | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/a172/val.json) | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/a172/test.json) |\n| BT474 | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/bt474/train.json) | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/bt474/val.json) | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/bt474/test.json) |\n| BV-2 | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/bv2/train.json) | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/bv2/val.json) | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/bv2/test.json) |\n| Huh7 | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/huh7/train.json) | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/huh7/val.json) | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/huh7/test.json) |\n| MCF7 | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/mcf7/train.json) | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/mcf7/val.json) | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/mcf7/test.json) |\n| SH-SHY5Y | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/shsy5y/train.json) | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/shsy5y/val.json) | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/shsy5y/test.json) |\n| SkBr3 | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/skbr3/train.json) | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/skbr3/val.json) | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/skbr3/test.json) |\n| SK-OV-3 | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/skov3/train.json) | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/skov3/val.json) | [link](//livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_single_cells/skov3/test.json) |\n\n\n### Dataset size experiments\n\n| Split | URL |\n| ---------- |:-----:|\n| 2 % | [link](http://livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_dataset_size_split/0_train2percent.json) |\n| 4 % | [link](http://livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_dataset_size_split/1_train4percent.json)|\n| 5 % | [link](http://livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_dataset_size_split/2_train5percent.json)|\n| 25 % | [link](http://livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_dataset_size_split/3_train25percent.json)|\n| 50 % | [link](http://livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/annotations/LIVECell_dataset_size_split/4_train50percent.json)|\n\n\n### Comparison to fluorescence-based object counts\nThe images and corresponding json-file with object count per image is available together with the raw fluorescent \nimages the counts is based on.\n\n| Cell Type | Images | Counts | Fluorescent images\n| ------------ |:------:|:----------:| :-----: |\n| A549 | [link](http://livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/nuclear_count_benchmark/A549.zip) | [link](http://livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/nuclear_count_benchmark/A549_counts.json) | [link](http://livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/nuclear_count_benchmark/A549_fluorescent_images.zip) \n| A172 | [link](http://livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/nuclear_count_benchmark/A172.zip) | [link](http://livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/nuclear_count_benchmark/A172_counts.json) | [link](http://livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/nuclear_count_benchmark/A172_fluorescent_images.zip) \n\n\n### Download all of LIVECell\n\nThe LIVECell-dataset and trained models is stored in an Amazon Web Services (AWS) S3-bucket. It is easiest to\ndownload the dataset if you have an AWS IAM-user using the\n[AWS-CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html) in the folder\nyou would like to download the dataset to by simply:\n```\naws s3 sync s3://livecell-dataset .\n```\n\nIf you do not have an AWS IAM-user, the procedure is a little bit more involved. We can use `curl` to make an\nHTTP-request to get the S3 XML-response and save to `files.xml`:\n\n```\ncurl -H \"GET /?list-type=2 HTTP/1.1\" \\\n -H \"Host: livecell-dataset.s3.eu-central-1.amazonaws.com\" \\\n -H \"Date: 20161025T124500Z\" \\\n -H \"Content-Type: text/plain\" http://livecell-dataset.s3.eu-central-1.amazonaws.com/ > files.xml\n```\n\nWe then get the urls from files using `grep`:\n\n```\ngrep -oPm1 \"(?<=)[^<]+\" files.xml | sed -e 's/^/http:\\/\\/livecell-dataset.s3.eu-central-1.amazonaws.com\\//' > urls.txt\n```\n\nThen download the files you like using `wget`.\n\n## File structure\nThe top-level structure of the files is arranged like:\n```\n/livecell-dataset/\n ├── LIVECell_dataset_2021 \n | ├── annotations/\n | ├── models/\n | ├── nuclear_count_benchmark/\t\n | └── images.zip \n ├── README.md \n └── LICENSE\n```\n\n### LIVECell_dataset_2021/images\nThe images of the LIVECell-dataset are stored in `/livecell-dataset/LIVECell_dataset_2021/images.zip` along with\ntheir annotations in `/livecell-dataset/LIVECell_dataset_2021/annotations/`.\n\nWithin `images.zip` are the training/validation-set and test-set images are completely separate to\nfacilitate fair comparison between studies. The images require 1.3 GB disk space unzipped and are arranged like:\n```\nimages/\n ├── livecell_test_images\n | └── \n | └── _Phase____.tif\n └── livecell_train_val_images\n └── \n```\nWhere `` is each of the eight cell-types in LIVECell (A172, BT474, BV2, Huh7, MCF7, SHSY5Y, SkBr3, SKOV3).\nWells `` are the location in the 96-well plate used to culture cells, `` indicates location in the well\nwhere the image was acquired, `` the time passed since the beginning of the experiment to image acquisition\nand `` index of the crop of the original larger image. An example image name is `A172_Phase_C7_1_02d16h00m_2.tif`,\nwhich is an image of A172-cells, grown in well C7 where the image is acquired in position 1 two days and 16 hours after\nexperiment start (crop position 2).\n\n### LIVECell_dataset_2021/annotations/\nThe annotations of LIVECell are prepared for all tasks along with the training/validation/test splits used for all\nexperiments in the paper. The annotations require 2.1 GB of disk space and are arranged like:\n\n```\nannotations/\n ├── LIVECell\n | └── livecell_coco_.json\n ├── LIVECell_single_cells\n | └── \n | └── .json\n └── LIVECell_dataset_size_split\n └── _trainpercent.json\n```\n\n* `annotations/LIVECell` contains the annotations used for the LIVECell-wide train and evaluate task.\n* `annotations/LIVECell_single_cells` contains the annotations used for Single cell type train and evaluate as well\n as the Single cell type transferability tasks.\n* `annotations/LIVECell_dataset_size_split` contains the annotations used to investigate the impact of training set\n scale.\n\nAll annotations are in [Microsoft COCO Object Detection-format](https://cocodataset.org/#format-data), and can for\ninstance be parsed by the Python package [`pycocotools`](https://pypi.org/project/pycocotools/).\n\n### models/\nALL models trained and evaluated for tasks associated with LIVECell are made available for wider use. The models\nare trained using [detectron2](https://github.com/facebookresearch/detectron2), Facebook's framework for\nobject detection and instance segmentation. The models require 15 GB of disk space and are arranged like:\n\n```\nmodels/\n └── Anchor_\n ├── ALL/\n | └──.pth\n └── /\n └──.pths\n \n```\n\nWhere each `.pth` is a binary file containing the model weights.\n\n### configs/\n \nThe config files for each model can be found in the [LIVECell github repo](https://github.com/sartorius-research/LIVECell)\n\n```\nLIVECell\n └── Anchor_\n ├── livecell_config.yaml\n ├── a172_config.yaml\n ├── bt474_config.yaml\n ├── bv2_config.yaml\n ├── huh7_config.yaml\n ├── mcf7_config.yaml\n ├── shsy5y_config.yaml\n ├── skbr3_config.yaml\n └── skov3_config.yaml\n```\n\nWhere each config file can be used to reproduce the training done or in combination with our model weights for usage, \nfor more info see the [usage section](model/README.md#Usage).\n\n### nuclear_count_benchmark/\nThe images and fluorescence-based object counts are stored as the label-free images in a zip-archive\nand the corresponding counts in a json as below:\n\n```\nnuclear_count_benchmark/\n ├── A172.zip\n ├── A172_counts.json\n ├── A172_fluorescent_images.zip\n ├── A549.zip\n ├── A549_counts.json \n └── A549_fluorescent_images.zip\n```\n\nThe json files are on the following format:\n\n```\n{\n \"\": \"\"\n}\n```\nWhere `` points to one of the images in the zip-archive, and `` refers to the object count\naccording fluorescent nuclear labels.\n\n## LICENSE\nAll images, annotations and models associated with LIVECell are published under\nAttribution-NonCommercial 4.0 International ([CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/)) license.\n\nAll software source code associated associated with LIVECell are published under the MIT License.\n" }, { "path": "_config.yml", "content": "theme: jekyll-theme-cayman\ntitle: LIVECell\ndescription: A large-scale dataset for label-free live cell segmentation\n" }, { "path": "code/Fluorescence cell count evaluation.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# LIVECell Fluorescence cell count benchmark\\n\",\n \"\\n\",\n \"This notebook contains a reference implementation of the evaluation of the fluorescence cell count benchmark in \\\"LIVECell - A large-scale dataset for label-free live cell segmentation\\\" by Edlund et. al. Given data of predicted and fluorescence-based cell count, the evaluation consists of two parts:\\n\",\n \"\\n\",\n \"1. R2 between predicted and fluorescence-based counts in images with fewer than 1600 cells per image (roughly corresponding to full confluency).\\n\",\n \"2. The point which the linear relationship breaks. This test works by comparing the residuals of a linear vs. a non-linear regression model of the fluorescence-based counts as a function of the predicted ones.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 1,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import pandas as pd\\n\",\n \"import numpy as np\\n\",\n \"import ipywidgets\\n\",\n \"from IPython.core.display import display\\n\",\n \"import matplotlib.pyplot as plt\\n\",\n \"\\n\",\n \"from sklearn.metrics import r2_score\\n\",\n \"from scipy import stats\\n\",\n \"from sklearn.linear_model import LinearRegression\\n\",\n \"from sklearn.neighbors import KNeighborsRegressor\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"First we define our functions.\\n\",\n \"\\n\",\n \"1. `get_counts_from_excel_file` reads the counts from the specific Excel-file format we used for the manuscripts. This is preferrably replaced by whatever format you like.\\n\",\n \"2. `linearity_cutoff_test` contains the test for when linearity breaks.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 2,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"def get_counts_from_excel_file(sheet_name, excel_file):\\n\",\n \" \\\"\\\"\\\" Load data from Excel-file and flatten to 1D-arrays. \\\"\\\"\\\"\\n\",\n \" \\n\",\n \" sheet = excel_file.parse(sheet_name, index_col=1)\\n\",\n \" sheet = sheet.rename(columns={sheet.columns[0]: 'time'})\\n\",\n \"\\n\",\n \" nc_cols = [col for col in sheet.columns if 'Image' in col]\\n\",\n \" model_cols = [col for col in sheet.columns if not col in nc_cols and col != 'time']\\n\",\n \"\\n\",\n \" nc_flat = sheet[nc_cols].values.flatten()\\n\",\n \" model_flat = sheet[model_cols].values.flatten()\\n\",\n \"\\n\",\n \" nc_is_nan = np.isnan(nc_flat)\\n\",\n \" model_is_nan = np.isnan(model_flat)\\n\",\n \" any_is_nan = nc_is_nan | model_is_nan\\n\",\n \"\\n\",\n \" nc_flat = nc_flat[~any_is_nan]\\n\",\n \" model_flat = model_flat[~any_is_nan]\\n\",\n \" return nc_flat, model_flat\\n\",\n \"\\n\",\n \"\\n\",\n \"def linearity_cutoff_test(\\n\",\n \" fluorescence_counts,\\n\",\n \" prediction_counts,\\n\",\n \" start_threshold = 500,\\n\",\n \" increment = 1,\\n\",\n \" p_cutoff = 1e-5, \\n\",\n \" n_neighbors=5\\n\",\n \"):\\n\",\n \" \\\"\\\"\\\" Test when linearity breaks. \\n\",\n \" \\n\",\n \" While the maximum number of objects per image is increased incrementally, \\n\",\n \" the fluorescence-based counts are regressed as a function of the predicted\\n\",\n \" counts using linear regression and KNN-regression (default 5 neighbors). \\n\",\n \" \\n\",\n \" Then the null hypothesis of equally sized residuals is tested using a \\n\",\n \" Levene's test. If the null hypothesis is rejected, the fit is considered\\n\",\n \" non-linear. \\n\",\n \" \\n\",\n \" Parameters\\n\",\n \" ----------\\n\",\n \" fluorescence_counts : array\\n\",\n \" 1D-array of ints containing fluorescence-based counts\\n\",\n \" prediction_counts : array\\n\",\n \" 1D-array ints containing predicted counts\\n\",\n \" start_threshold : int\\n\",\n \" Maximum number of objects per image to start incrementing from (default 500)\\n\",\n \" increment : int\\n\",\n \" Number of objects per image to increment with (default 1)\\n\",\n \" p_cutoff : float\\n\",\n \" p-value cutoff to reject null hypothesis (default 1E-5)\\n\",\n \" n_neighbors : int\\n\",\n \" Number of neighbors in KNN-regression.\\n\",\n \" \\n\",\n \" Returns\\n\",\n \" -------\\n\",\n \" int\\n\",\n \" Number of objects per image where null hypothesis was first rejected.\\n\",\n \" \\\"\\\"\\\"\\n\",\n \"\\n\",\n \" for test_threshold in range(start_threshold, int(nc_flat.max()), increment):\\n\",\n \" below_test_threshold = fluorescence_counts < test_threshold\\n\",\n \" y = fluorescence_counts[below_test_threshold]\\n\",\n \"\\n\",\n \" prediction_counts_2d = np.atleast_2d(prediction_counts[below_test_threshold]).T\\n\",\n \" linear_model = LinearRegression().fit(prediction_counts_2d, y)\\n\",\n \" knn_model = KNeighborsRegressor(n_neighbors).fit(prediction_counts_2d, y)\\n\",\n \" linear_pred_nc = linear_model.predict(prediction_counts_2d)\\n\",\n \" knn_pred_nc = knn_model.predict(prediction_counts_2d)\\n\",\n \"\\n\",\n \" knn_residal = (y - knn_pred_nc)\\n\",\n \" linear_residual = (y - linear_pred_nc)\\n\",\n \" test_result = stats.levene(knn_residal, linear_residual)\\n\",\n \" if test_result.pvalue < p_cutoff:\\n\",\n \" break\\n\",\n \" \\n\",\n \" return test_threshold\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Pick file to analyze.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 3,\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"application/vnd.jupyter.widget-view+json\": {\n \"model_id\": \"596c505bc775437995efc34f96ed561d\",\n \"version_major\": 2,\n \"version_minor\": 0\n },\n \"text/plain\": [\n \"FileUpload(value={}, accept='.xlsx', description='Upload')\"\n ]\n },\n \"metadata\": {},\n \"output_type\": \"display_data\"\n }\n ],\n \"source\": [\n \"uploader = ipywidgets.FileUpload(accept='.xlsx', multiple=False)\\n\",\n \"display(uploader)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Run tests\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 4,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"A549 - Anchor-free model\\n\",\n \"R2 below 1600 objects = 0.980\\n\",\n \"Linearity break, n objects = 2031\\n\",\n \"\\n\",\n \"A549 - Anchor-based model\\n\",\n \"R2 below 1600 objects = 0.985\\n\",\n \"Linearity break, n objects = 1403\\n\",\n \"\\n\",\n \"A172 - Anchor-free model\\n\",\n \"R2 below 1600 objects = 0.942\\n\",\n \"Linearity break, n objects = 1948\\n\",\n \"\\n\",\n \"A172 - Anchor-based model\\n\",\n \"R2 below 1600 objects = 0.977\\n\",\n \"Linearity break, n objects = 1328\\n\",\n \"\\n\"\n ]\n }\n ],\n \"source\": [\n \"if not uploader.value:\\n\",\n \" print('Pick file using file-picker first')\\n\",\n \"else:\\n\",\n \" first_key = next(key for key in uploader.value)\\n\",\n \" excel_file = pd.ExcelFile(uploader.value[first_key]['content'], engine='openpyxl')\\n\",\n \" sheet_names = excel_file.sheet_names\\n\",\n \"\\n\",\n \" threshold = 1600\\n\",\n \"\\n\",\n \" for sheet_name in sheet_names:\\n\",\n \" cell_type, model_name = sheet_name.split('-', 1)\\n\",\n \" print(f'{cell_type} - {model_name} model')\\n\",\n \" nc_flat, model_flat = get_counts_from_excel_file(sheet_name, excel_file)\\n\",\n \"\\n\",\n \" below_threshold = nc_flat < threshold\\n\",\n \" r2 = r2_score(nc_flat[below_threshold], model_flat[below_threshold])\\n\",\n \" linearity_cutoff = linearity_cutoff_test(nc_flat, model_flat)\\n\",\n \" print(f'R2 below {threshold} objects = {r2:.3f}')\\n\",\n \" print(f'Linearity break, n objects = {linearity_cutoff}')\\n\",\n \" print()\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.6.7\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 2\n}\n" }, { "path": "code/coco_evaluation.py", "content": "# Copyright (c) Facebook, Inc. and its affiliates.\n# Modified by Sangrok Lee and Youngwan Lee (ETRI), 2020.\n# We modify COCOEvaluator for adopting mask_score in mask evalaution.\n# Modified by Christoffer Edlund (Sartorius), 2022. All Rights Reserved.\n# Modfied COCOEvaluator to support more than 100 detections in the evaluation and added\n# evlaution of multiple IoU Levels.\n\nimport types\nimport contextlib\nimport copy\nimport io\nimport itertools\nimport json\nimport logging\nimport numpy as np\nimport os\nimport pickle\nfrom collections import OrderedDict\nimport pycocotools.mask as mask_util\nimport torch\nfrom pycocotools.coco import COCO\nfrom pycocotools.cocoeval import COCOeval\nfrom tabulate import tabulate\n\nimport detectron2.utils.comm as comm\nfrom detectron2.config import CfgNode\nfrom detectron2.data import MetadataCatalog\nfrom detectron2.data.datasets.coco import convert_to_coco_json\nfrom detectron2.evaluation.fast_eval_api import COCOeval_opt\nfrom detectron2.structures import Boxes, BoxMode, pairwise_iou\nfrom detectron2.utils.file_io import PathManager\nfrom detectron2.utils.logger import create_small_table\n\nfrom detectron2.evaluation.evaluator import DatasetEvaluator\n\n\nclass COCOEvaluator(DatasetEvaluator):\n \"\"\"\n Evaluate AR for object proposals, AP for instance detection/segmentation, AP\n for keypoint detection outputs using COCO's metrics.\n See http://cocodataset.org/#detection-eval and\n http://cocodataset.org/#keypoints-eval to understand its metrics.\n In addition to COCO, this evaluator is able to support any bounding box detection,\n instance segmentation, or keypoint detection dataset.\n \"\"\"\n\n def __init__(\n self,\n dataset_name,\n tasks=None,\n distributed=True,\n output_dir=None,\n *,\n use_fast_impl=True,\n kpt_oks_sigmas=(),\n ):\n \"\"\"\n Args:\n dataset_name (str): name of the dataset to be evaluated.\n It must have either the following corresponding metadata:\n \"json_file\": the path to the COCO format annotation\n Or it must be in detectron2's standard dataset format\n so it can be converted to COCO format automatically.\n tasks (tuple[str]): tasks that can be evaluated under the given\n configuration. A task is one of \"bbox\", \"segm\", \"keypoints\".\n By default, will infer this automatically from predictions.\n distributed (True): if True, will collect results from all ranks and run evaluation\n in the main process.\n Otherwise, will only evaluate the results in the current process.\n output_dir (str): optional, an output directory to dump all\n results predicted on the dataset. The dump contains two files:\n 1. \"instances_predictions.pth\" a file in torch serialization\n format that contains all the raw original predictions.\n 2. \"coco_instances_results.json\" a json file in COCO's result\n format.\n use_fast_impl (bool): use a fast but **unofficial** implementation to compute AP.\n Although the results should be very close to the official implementation in COCO\n API, it is still recommended to compute results with the official API for use in\n papers. The faster implementation also uses more RAM.\n kpt_oks_sigmas (list[float]): The sigmas used to calculate keypoint OKS.\n See http://cocodataset.org/#keypoints-eval\n When empty, it will use the defaults in COCO.\n Otherwise it should be the same length as ROI_KEYPOINT_HEAD.NUM_KEYPOINTS.\n \"\"\"\n self._logger = logging.getLogger(__name__)\n self._distributed = distributed\n self._output_dir = output_dir\n self._use_fast_impl = use_fast_impl\n\n if tasks is not None and isinstance(tasks, CfgNode):\n kpt_oks_sigmas = (\n tasks.TEST.KEYPOINT_OKS_SIGMAS if not kpt_oks_sigmas else kpt_oks_sigmas\n )\n self._logger.warn(\n \"COCO Evaluator instantiated using config, this is deprecated behavior.\"\n \" Please pass in explicit arguments instead.\"\n )\n self._tasks = None # Infering it from predictions should be better\n else:\n self._tasks = tasks\n\n self._cpu_device = torch.device(\"cpu\")\n\n self._metadata = MetadataCatalog.get(dataset_name)\n if not hasattr(self._metadata, \"json_file\"):\n self._logger.info(\n f\"'{dataset_name}' is not registered by `register_coco_instances`.\"\n \" Therefore trying to convert it to COCO format ...\"\n )\n\n cache_path = os.path.join(output_dir, f\"{dataset_name}_coco_format.json\")\n self._metadata.json_file = cache_path\n convert_to_coco_json(dataset_name, cache_path)\n\n json_file = PathManager.get_local_path(self._metadata.json_file)\n with contextlib.redirect_stdout(io.StringIO()):\n self._coco_api = COCO(json_file)\n\n # Test set json files do not contain annotations (evaluation must be\n # performed using the COCO evaluation server).\n self._do_evaluation = \"annotations\" in self._coco_api.dataset\n if self._do_evaluation:\n self._kpt_oks_sigmas = kpt_oks_sigmas\n\n def reset(self):\n self._predictions = []\n\n def process(self, inputs, outputs):\n \"\"\"\n Args:\n inputs: the inputs to a COCO model (e.g., GeneralizedRCNN).\n It is a list of dict. Each dict corresponds to an image and\n contains keys like \"height\", \"width\", \"file_name\", \"image_id\".\n outputs: the outputs of a COCO model. It is a list of dicts with key\n \"instances\" that contains :class:`Instances`.\n \"\"\"\n for input, output in zip(inputs, outputs):\n prediction = {\"image_id\": input[\"image_id\"]}\n\n if \"instances\" in output:\n instances = output[\"instances\"].to(self._cpu_device)\n prediction[\"instances\"] = instances_to_coco_json(instances, input[\"image_id\"])\n if \"proposals\" in output:\n prediction[\"proposals\"] = output[\"proposals\"].to(self._cpu_device)\n if len(prediction) > 1:\n self._predictions.append(prediction)\n\n def evaluate(self, img_ids=None):\n \"\"\"\n Args:\n img_ids: a list of image IDs to evaluate on. Default to None for the whole dataset\n \"\"\"\n if self._distributed:\n comm.synchronize()\n predictions = comm.gather(self._predictions, dst=0)\n predictions = list(itertools.chain(*predictions))\n\n if not comm.is_main_process():\n return {}\n else:\n predictions = self._predictions\n\n if len(predictions) == 0:\n self._logger.warning(\"[COCOEvaluator] Did not receive valid predictions.\")\n return {}\n\n if self._output_dir:\n PathManager.mkdirs(self._output_dir)\n file_path = os.path.join(self._output_dir, \"instances_predictions.pth\")\n with PathManager.open(file_path, \"wb\") as f:\n torch.save(predictions, f)\n\n self._results = OrderedDict()\n if \"proposals\" in predictions[0]:\n self._eval_box_proposals(predictions)\n if \"instances\" in predictions[0]:\n self._eval_predictions(predictions, img_ids=img_ids)\n # Copy so the caller can do whatever with results\n return copy.deepcopy(self._results)\n\n def _tasks_from_predictions(self, predictions):\n \"\"\"\n Get COCO API \"tasks\" (i.e. iou_type) from COCO-format predictions.\n \"\"\"\n tasks = {\"bbox\"}\n for pred in predictions:\n if \"segmentation\" in pred:\n tasks.add(\"segm\")\n if \"keypoints\" in pred:\n tasks.add(\"keypoints\")\n return sorted(tasks)\n\n def _eval_predictions(self, predictions, img_ids=None):\n \"\"\"\n Evaluate predictions. Fill self._results with the metrics of the tasks.\n \"\"\"\n self._logger.info(\"Preparing results for COCO format ...\")\n coco_results = list(itertools.chain(*[x[\"instances\"] for x in predictions]))\n tasks = self._tasks or self._tasks_from_predictions(coco_results)\n\n # unmap the category ids for COCO\n if hasattr(self._metadata, \"thing_dataset_id_to_contiguous_id\"):\n dataset_id_to_contiguous_id = self._metadata.thing_dataset_id_to_contiguous_id\n all_contiguous_ids = list(dataset_id_to_contiguous_id.values())\n num_classes = len(all_contiguous_ids)\n assert min(all_contiguous_ids) == 0 and max(all_contiguous_ids) == num_classes - 1\n\n reverse_id_mapping = {v: k for k, v in dataset_id_to_contiguous_id.items()}\n for result in coco_results:\n category_id = result[\"category_id\"]\n assert category_id < num_classes, (\n f\"A prediction has class={category_id}, \"\n f\"but the dataset only has {num_classes} classes and \"\n f\"predicted class id should be in [0, {num_classes - 1}].\"\n )\n result[\"category_id\"] = reverse_id_mapping[category_id]\n\n if self._output_dir:\n file_path = os.path.join(self._output_dir, \"coco_instances_results.json\")\n self._logger.info(\"Saving results to {}\".format(file_path))\n with PathManager.open(file_path, \"w\") as f:\n f.write(json.dumps(coco_results))\n f.flush()\n\n if not self._do_evaluation:\n self._logger.info(\"Annotations are not available for evaluation.\")\n return\n\n self._logger.info(\n \"Evaluating predictions with {} COCO API...\".format(\n \"unofficial\" if self._use_fast_impl else \"official\"\n )\n )\n for task in sorted(tasks):\n coco_eval = (\n _evaluate_predictions_on_coco(\n self._coco_api,\n coco_results,\n task,\n kpt_oks_sigmas=self._kpt_oks_sigmas,\n use_fast_impl=self._use_fast_impl,\n img_ids=img_ids,\n )\n if len(coco_results) > 0\n else None # cocoapi does not handle empty results very well\n )\n\n res = self._derive_coco_results(\n coco_eval, task, class_names=self._metadata.get(\"thing_classes\")\n )\n self._results[task] = res\n\n def _eval_box_proposals(self, predictions):\n \"\"\"\n Evaluate the box proposals in predictions.\n Fill self._results with the metrics for \"box_proposals\" task.\n \"\"\"\n if self._output_dir:\n # Saving generated box proposals to file.\n # Predicted box_proposals are in XYXY_ABS mode.\n bbox_mode = BoxMode.XYXY_ABS.value\n ids, boxes, objectness_logits = [], [], []\n for prediction in predictions:\n ids.append(prediction[\"image_id\"])\n boxes.append(prediction[\"proposals\"].proposal_boxes.tensor.numpy())\n objectness_logits.append(prediction[\"proposals\"].objectness_logits.numpy())\n\n proposal_data = {\n \"boxes\": boxes,\n \"objectness_logits\": objectness_logits,\n \"ids\": ids,\n \"bbox_mode\": bbox_mode,\n }\n with PathManager.open(os.path.join(self._output_dir, \"box_proposals.pkl\"), \"wb\") as f:\n pickle.dump(proposal_data, f)\n\n if not self._do_evaluation:\n self._logger.info(\"Annotations are not available for evaluation.\")\n return\n\n self._logger.info(\"Evaluating bbox proposals ...\")\n res = {}\n areas = {\"all\": \"\", \"small\": \"s\", \"medium\": \"m\", \"large\": \"l\"}\n for limit in [100, 1000]:\n for area, suffix in areas.items():\n stats = _evaluate_box_proposals(predictions, self._coco_api, area=area, limit=limit)\n key = \"AR{}@{:d}\".format(suffix, limit)\n res[key] = float(stats[\"ar\"].item() * 100)\n self._logger.info(\"Proposal metrics: \\n\" + create_small_table(res))\n self._results[\"box_proposals\"] = res\n\n def _derive_coco_results(self, coco_eval, iou_type, class_names=None):\n \"\"\"\n Derive the desired score numbers from summarized COCOeval.\n Args:\n coco_eval (None or COCOEval): None represents no predictions from model.\n iou_type (str):\n class_names (None or list[str]): if provided, will use it to predict\n per-category AP.\n Returns:\n a dict of {metric name: score}\n \"\"\"\n\n metrics = {\n \"bbox\": [\"AP\", \"AP50\", \"AP75\", \"APs\", \"APm\", \"APl\"],\n \"segm\": [\"AP\", \"AP50\", \"AP75\", \"APs\", \"APm\", \"APl\"],\n \"keypoints\": [\"AP\", \"AP50\", \"AP75\", \"APm\", \"APl\"],\n }[iou_type]\n\n if coco_eval is None:\n self._logger.warn(\"No predictions from the model!\")\n return {metric: float(\"nan\") for metric in metrics}\n\n # the standard metrics\n results = {\n metric: float(coco_eval.stats[idx] * 100 if coco_eval.stats[idx] >= 0 else \"nan\")\n for idx, metric in enumerate(metrics)\n }\n self._logger.info(\n \"Evaluation results for {}: \\n\".format(iou_type) + create_small_table(results)\n )\n if not np.isfinite(sum(results.values())):\n self._logger.info(\"Some metrics cannot be computed and is shown as NaN.\")\n\n if class_names is None or len(class_names) <= 1:\n return results\n # Compute per-category AP\n # from https://github.com/facebookresearch/Detectron/blob/a6a835f5b8208c45d0dce217ce9bbda915f44df7/detectron/datasets/json_dataset_evaluator.py#L222-L252 # noqa\n precisions = coco_eval.eval[\"precision\"]\n # precision has dims (iou, recall, cls, area range, max dets)\n assert len(class_names) == precisions.shape[2]\n\n results_per_category = []\n for idx, name in enumerate(class_names):\n # area range index 0: all area ranges\n # max dets index -1: typically 100 per image\n precision = precisions[:, :, idx, 0, -1]\n precision = precision[precision > -1]\n ap = np.mean(precision) if precision.size else float(\"nan\")\n results_per_category.append((\"{}\".format(name), float(ap * 100)))\n\n # tabulate it\n N_COLS = min(6, len(results_per_category) * 2)\n results_flatten = list(itertools.chain(*results_per_category))\n results_2d = itertools.zip_longest(*[results_flatten[i::N_COLS] for i in range(N_COLS)])\n table = tabulate(\n results_2d,\n tablefmt=\"pipe\",\n floatfmt=\".3f\",\n headers=[\"category\", \"AP\"] * (N_COLS // 2),\n numalign=\"left\",\n )\n self._logger.info(\"Per-category {} AP: \\n\".format(iou_type) + table)\n\n results.update({\"AP-\" + name: ap for name, ap in results_per_category})\n return results\n\n\ndef instances_to_coco_json(instances, img_id):\n \"\"\"\n Dump an \"Instances\" object to a COCO-format json that's used for evaluation.\n Args:\n instances (Instances):\n img_id (int): the image id\n Returns:\n list[dict]: list of json annotations in COCO format.\n \"\"\"\n num_instance = len(instances)\n if num_instance == 0:\n return []\n\n boxes = instances.pred_boxes.tensor.numpy()\n boxes = BoxMode.convert(boxes, BoxMode.XYXY_ABS, BoxMode.XYWH_ABS)\n boxes = boxes.tolist()\n scores = instances.scores.tolist()\n classes = instances.pred_classes.tolist()\n\n has_mask = instances.has(\"pred_masks\")\n has_mask_scores = instances.has(\"mask_scores\")\n if has_mask:\n # use RLE to encode the masks, because they are too large and takes memory\n # since this evaluator stores outputs of the entire dataset\n rles = [\n mask_util.encode(np.array(mask[:, :, None], order=\"F\", dtype=\"uint8\"))[0]\n for mask in instances.pred_masks\n ]\n for rle in rles:\n # \"counts\" is an array encoded by mask_util as a byte-stream. Python3's\n # json writer which always produces strings cannot serialize a bytestream\n # unless you decode it. Thankfully, utf-8 works out (which is also what\n # the pycocotools/_mask.pyx does).\n rle[\"counts\"] = rle[\"counts\"].decode(\"utf-8\")\n\n if has_mask_scores:\n mask_scores = instances.mask_scores.tolist()\n\n has_keypoints = instances.has(\"pred_keypoints\")\n if has_keypoints:\n keypoints = instances.pred_keypoints\n\n results = []\n for k in range(num_instance):\n result = {\n \"image_id\": img_id,\n \"category_id\": classes[k],\n \"bbox\": boxes[k],\n \"score\": scores[k],\n }\n if has_mask:\n result[\"segmentation\"] = rles[k]\n if has_mask_scores:\n result[\"mask_score\"] = mask_scores[k]\n if has_keypoints:\n # In COCO annotations,\n # keypoints coordinates are pixel indices.\n # However our predictions are floating point coordinates.\n # Therefore we subtract 0.5 to be consistent with the annotation format.\n # This is the inverse of data loading logic in `datasets/coco.py`.\n keypoints[k][:, :2] -= 0.5\n result[\"keypoints\"] = keypoints[k].flatten().tolist()\n results.append(result)\n return results\n\n\n# inspired from Detectron:\n# https://github.com/facebookresearch/Detectron/blob/a6a835f5b8208c45d0dce217ce9bbda915f44df7/detectron/datasets/json_dataset_evaluator.py#L255 # noqa\ndef _evaluate_box_proposals(dataset_predictions, coco_api, thresholds=None, area=\"all\", limit=None):\n \"\"\"\n Evaluate detection proposal recall metrics. This function is a much\n faster alternative to the official COCO API recall evaluation code. However,\n it produces slightly different results.\n \"\"\"\n # Record max overlap value for each gt box\n # Return vector of overlap values\n areas = {\n \"all\": 0,\n \"small\": 1,\n \"medium\": 2,\n \"large\": 3,\n \"96-128\": 4,\n \"128-256\": 5,\n \"256-512\": 6,\n \"512-inf\": 7,\n }\n area_ranges = [\n [0 ** 2, 1e5 ** 2], # all\n [0 ** 2, 32 ** 2], # small\n [32 ** 2, 96 ** 2], # medium\n [96 ** 2, 1e5 ** 2], # large\n [96 ** 2, 128 ** 2], # 96-128\n [128 ** 2, 256 ** 2], # 128-256\n [256 ** 2, 512 ** 2], # 256-512\n [512 ** 2, 1e5 ** 2],\n ] # 512-inf\n assert area in areas, \"Unknown area range: {}\".format(area)\n area_range = area_ranges[areas[area]]\n gt_overlaps = []\n num_pos = 0\n\n for prediction_dict in dataset_predictions:\n predictions = prediction_dict[\"proposals\"]\n\n # sort predictions in descending order\n # TODO maybe remove this and make it explicit in the documentation\n inds = predictions.objectness_logits.sort(descending=True)[1]\n predictions = predictions[inds]\n\n ann_ids = coco_api.getAnnIds(imgIds=prediction_dict[\"image_id\"])\n anno = coco_api.loadAnns(ann_ids)\n gt_boxes = [\n BoxMode.convert(obj[\"bbox\"], BoxMode.XYWH_ABS, BoxMode.XYXY_ABS)\n for obj in anno\n if obj[\"iscrowd\"] == 0\n ]\n gt_boxes = torch.as_tensor(gt_boxes).reshape(-1, 4) # guard against no boxes\n gt_boxes = Boxes(gt_boxes)\n gt_areas = torch.as_tensor([obj[\"area\"] for obj in anno if obj[\"iscrowd\"] == 0])\n\n if len(gt_boxes) == 0 or len(predictions) == 0:\n continue\n\n valid_gt_inds = (gt_areas >= area_range[0]) & (gt_areas <= area_range[1])\n gt_boxes = gt_boxes[valid_gt_inds]\n\n num_pos += len(gt_boxes)\n\n if len(gt_boxes) == 0:\n continue\n\n if limit is not None and len(predictions) > limit:\n predictions = predictions[:limit]\n\n overlaps = pairwise_iou(predictions.proposal_boxes, gt_boxes)\n\n _gt_overlaps = torch.zeros(len(gt_boxes))\n for j in range(min(len(predictions), len(gt_boxes))):\n # find which proposal box maximally covers each gt box\n # and get the iou amount of coverage for each gt box\n max_overlaps, argmax_overlaps = overlaps.max(dim=0)\n\n # find which gt box is 'best' covered (i.e. 'best' = most iou)\n gt_ovr, gt_ind = max_overlaps.max(dim=0)\n assert gt_ovr >= 0\n # find the proposal box that covers the best covered gt box\n box_ind = argmax_overlaps[gt_ind]\n # record the iou coverage of this gt box\n _gt_overlaps[j] = overlaps[box_ind, gt_ind]\n assert _gt_overlaps[j] == gt_ovr\n # mark the proposal box and the gt box as used\n overlaps[box_ind, :] = -1\n overlaps[:, gt_ind] = -1\n\n # append recorded iou coverage level\n gt_overlaps.append(_gt_overlaps)\n gt_overlaps = (\n torch.cat(gt_overlaps, dim=0) if len(gt_overlaps) else torch.zeros(0, dtype=torch.float32)\n )\n gt_overlaps, _ = torch.sort(gt_overlaps)\n\n if thresholds is None:\n step = 0.05\n thresholds = torch.arange(0.5, 0.95 + 1e-5, step, dtype=torch.float32)\n recalls = torch.zeros_like(thresholds)\n # compute recall for each iou threshold\n for i, t in enumerate(thresholds):\n recalls[i] = (gt_overlaps >= t).float().sum() / float(num_pos)\n # ar = 2 * np.trapz(recalls, thresholds)\n ar = recalls.mean()\n return {\n \"ar\": ar,\n \"recalls\": recalls,\n \"thresholds\": thresholds,\n \"gt_overlaps\": gt_overlaps,\n \"num_pos\": num_pos,\n }\n\n\ndef _evaluate_predictions_on_coco(\n coco_gt, coco_results, iou_type, kpt_oks_sigmas=None, use_fast_impl=True, img_ids=None\n):\n \"\"\"\n Evaluate the coco results using COCOEval API.\n \"\"\"\n assert len(coco_results) > 0\n\n if iou_type == \"segm\":\n coco_results = copy.deepcopy(coco_results)\n # When evaluating mask AP, if the results contain bbox, cocoapi will\n # use the box area as the area of the instance, instead of the mask area.\n # This leads to a different definition of small/medium/large.\n # We remove the bbox field to let mask AP use mask area.\n has_mask_scores = \"mask_score\" in coco_results[0]\n\n for c in coco_results:\n c.pop(\"bbox\", None)\n if has_mask_scores:\n c[\"score\"] = c[\"mask_score\"]\n del c[\"mask_score\"]\n\n coco_dt = coco_gt.loadRes(coco_results)\n coco_eval = (COCOeval_opt if use_fast_impl else COCOeval)(coco_gt, coco_dt, iou_type)\n if img_ids is not None:\n coco_eval.params.imgIds = img_ids\n\n if iou_type == \"keypoints\":\n # Use the COCO default keypoint OKS sigmas unless overrides are specified\n if kpt_oks_sigmas:\n assert hasattr(coco_eval.params, \"kpt_oks_sigmas\"), \"pycocotools is too old!\"\n coco_eval.params.kpt_oks_sigmas = np.array(kpt_oks_sigmas)\n # COCOAPI requires every detection and every gt to have keypoints, so\n # we just take the first entry from both\n num_keypoints_dt = len(coco_results[0][\"keypoints\"]) // 3\n num_keypoints_gt = len(next(iter(coco_gt.anns.values()))[\"keypoints\"]) // 3\n num_keypoints_oks = len(coco_eval.params.kpt_oks_sigmas)\n assert num_keypoints_oks == num_keypoints_dt == num_keypoints_gt, (\n f\"[COCOEvaluator] Prediction contain {num_keypoints_dt} keypoints. \"\n f\"Ground truth contains {num_keypoints_gt} keypoints. \"\n f\"The length of cfg.TEST.KEYPOINT_OKS_SIGMAS is {num_keypoints_oks}. \"\n \"They have to agree with each other. For meaning of OKS, please refer to \"\n \"http://cocodataset.org/#keypoints-eval.\"\n )\n\n\n #Insert this code to increase the number of detections possible /Christoffer :\n\n def summarize_2(self, all_prec=False):\n '''\n Compute and display summary metrics for evaluation results.\n Note this functin can *only* be applied on the default parameter setting\n '''\n\n print(\"In method\")\n def _summarize(ap=1, iouThr=None, areaRng='all', maxDets=2000):\n p = self.params\n iStr = ' {:<18} {} @[ IoU={:<9} | area={:>6s} | maxDets={:>3d} ] = {:0.3f}'\n titleStr = 'Average Precision' if ap == 1 else 'Average Recall'\n typeStr = '(AP)' if ap == 1 else '(AR)'\n iouStr = '{:0.2f}:{:0.2f}'.format(p.iouThrs[0], p.iouThrs[-1]) \\\n if iouThr is None else '{:0.2f}'.format(iouThr)\n\n aind = [i for i, aRng in enumerate(p.areaRngLbl) if aRng == areaRng]\n mind = [i for i, mDet in enumerate(p.maxDets) if mDet == maxDets]\n if ap == 1:\n # dimension of precision: [TxRxKxAxM]\n s = self.eval['precision']\n # IoU\n if iouThr is not None:\n t = np.where(iouThr == p.iouThrs)[0]\n s = s[t]\n s = s[:, :, :, aind, mind]\n else:\n # dimension of recall: [TxKxAxM]\n s = self.eval['recall']\n if iouThr is not None:\n t = np.where(iouThr == p.iouThrs)[0]\n s = s[t]\n s = s[:, :, aind, mind]\n if len(s[s > -1]) == 0:\n mean_s = -1\n else:\n mean_s = np.mean(s[s > -1])\n print(iStr.format(titleStr, typeStr, iouStr, areaRng, maxDets, mean_s))\n return mean_s\n\n def _summarizeDets():\n\n stats = np.zeros((12,))\n stats[0] = _summarize(1, maxDets=self.params.maxDets[2])\n stats[1] = _summarize(1, iouThr=.5, maxDets=self.params.maxDets[2])\n stats[2] = _summarize(1, iouThr=.75, maxDets=self.params.maxDets[2])\n stats[3] = _summarize(1, areaRng='small', maxDets=self.params.maxDets[2])\n stats[4] = _summarize(1, areaRng='medium', maxDets=self.params.maxDets[2])\n stats[5] = _summarize(1, areaRng='large', maxDets=self.params.maxDets[2])\n stats[6] = _summarize(0, maxDets=self.params.maxDets[0])\n stats[7] = _summarize(0, maxDets=self.params.maxDets[1])\n stats[8] = _summarize(0, maxDets=self.params.maxDets[2])\n stats[9] = _summarize(0, areaRng='small', maxDets=self.params.maxDets[2])\n stats[10] = _summarize(0, areaRng='medium', maxDets=self.params.maxDets[2])\n stats[11] = _summarize(0, areaRng='large', maxDets=self.params.maxDets[2])\n return stats\n\n\n def _summarizeKps():\n stats = np.zeros((10,))\n stats[0] = _summarize(1, maxDets=self.params.maxDets[2])\n stats[1] = _summarize(1, maxDets=self.params.maxDets[2], iouThr=.5)\n stats[2] = _summarize(1, maxDets=self.params.maxDets[2], iouThr=.75)\n stats[3] = _summarize(1, maxDets=self.params.maxDets[2], areaRng='medium')\n stats[4] = _summarize(1, maxDets=self.params.maxDets[2], areaRng='large')\n stats[5] = _summarize(0, maxDets=self.params.maxDets[2])\n stats[6] = _summarize(0, maxDets=self.params.maxDets[2], iouThr=.5)\n stats[7] = _summarize(0, maxDets=self.params.maxDets[2], iouThr=.75)\n stats[8] = _summarize(0, maxDets=self.params.maxDets[2], areaRng='medium')\n stats[9] = _summarize(0, maxDets=self.params.maxDets[2], areaRng='large')\n return stats\n\n if not self.eval:\n raise Exception('Please run accumulate() first')\n iouType = self.params.iouType\n if iouType == 'segm' or iouType == 'bbox':\n summarize = _summarizeDets\n elif iouType == 'keypoints':\n summarize = _summarizeKps\n self.stats = summarize()\n\n coco_eval.params.catIds = [1]\n coco_eval.params.useCats = 0\n coco_eval.params.maxDets = [100, 500, 2000]\n\n coco_eval.params.areaRng = [[0 ** 2, 1e5 ** 2], [0 ** 2, 18 ** 2], [18 ** 2, 31 ** 2], [31 ** 2, 1e5 ** 2]]\n coco_eval.params.areaRngLbl = ['all', 'small', 'medium', 'large']\n\n print(f\"Size parameters: {coco_eval.params.areaRng}\")\n\n coco_eval.summarize = types.MethodType(summarize_2, coco_eval)\n\n coco_eval.evaluate()\n coco_eval.accumulate()\n coco_eval.summarize()\n\n \"\"\"\n Added code to produce precision and recall for all iou levels / Chris\n \"\"\"\n precisions = coco_eval.eval['precision']\n recalls = coco_eval.eval['recall']\n\n # IoU threshold | instances | Categories | areas | max dets\n pre_per_iou = [precisions[iou_idx, :, :, 0, -1].mean() for iou_idx in range(precisions.shape[0])]\n rec_pre_iou = [recalls[iou_idx, :, 0, -1].mean() for iou_idx in range(recalls.shape[0])]\n\n print(f\"Precision and Recall per iou \\n IoU: {coco_eval.params.iouThrs}\")\n print(f\"Pre: {np.round(np.array(pre_per_iou), 4)}\")\n print(f\"Rec: {np.round(np.array(rec_pre_iou), 4)}\")\n\n\n return coco_eval\n" }, { "path": "code/coco_evaluation_resnest.py", "content": "# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n# Modified by Sangrok Lee and Youngwan Lee (ETRI), 2020. All Rights Reserved.\n# Modified by Christoffer Edlund (Sartorius), 2020. All Rights Reserved.\nimport types\nimport contextlib\nimport copy\nimport io\nimport itertools\nimport json\nimport logging\nimport numpy as np\nimport os\nimport pickle\nfrom collections import OrderedDict\nimport pycocotools.mask as mask_util\nimport torch\nfrom fvcore.common.file_io import PathManager\nfrom pycocotools.coco import COCO\nfrom pycocotools.cocoeval import COCOeval\nfrom tabulate import tabulate\n\nimport detectron2.utils.comm as comm\nfrom detectron2.data import MetadataCatalog\nfrom detectron2.data.datasets.coco import convert_to_coco_json\nfrom detectron2.structures import Boxes, BoxMode, pairwise_iou\nfrom detectron2.utils.logger import create_small_table\n\nfrom detectron2.evaluation.evaluator import DatasetEvaluator\n#from detectron2.evaluation.fast_eval_api import COCOeval_opt\n_use_fast_impl = True\ntry:\n from fast_coco_eval import COCOeval_fast as COCOeval_opt\nexcept ImportError:\n print(f\"Could not find fast coco implementation\")\n _use_fast_impl = False\n\nclass COCOEvaluator(DatasetEvaluator):\n \"\"\"\n Evaluate object proposal, instance detection/segmentation, keypoint detection\n outputs using COCO's metrics and APIs.\n \"\"\"\n\n def __init__(self, dataset_name, cfg, distributed, output_dir=None):\n \"\"\"\n Args:\n dataset_name (str): name of the dataset to be evaluated.\n It must have either the following corresponding metadata:\n \"json_file\": the path to the COCO format annotation\n Or it must be in detectron2's standard dataset format\n so it can be converted to COCO format automatically.\n cfg (CfgNode): config instance\n distributed (True): if True, will collect results from all ranks for evaluation.\n Otherwise, will evaluate the results in the current process.\n output_dir (str): optional, an output directory to dump all\n results predicted on the dataset. The dump contains two files:\n 1. \"instance_predictions.pth\" a file in torch serialization\n format that contains all the raw original predictions.\n 2. \"coco_instances_results.json\" a json file in COCO's result\n format.\n \"\"\"\n print(\"__init__\")\n self._tasks = self._tasks_from_config(cfg)\n self._distributed = distributed\n self._output_dir = output_dir\n\n self._cpu_device = torch.device(\"cpu\")\n self._logger = logging.getLogger(__name__)\n\n self._metadata = MetadataCatalog.get(dataset_name)\n if not hasattr(self._metadata, \"json_file\"):\n self._logger.warning(\n f\"json_file was not found in MetaDataCatalog for '{dataset_name}'.\"\n \" Trying to convert it to COCO format ...\"\n )\n\n cache_path = os.path.join(output_dir, f\"{dataset_name}_coco_format.json\")\n self._metadata.json_file = cache_path\n convert_to_coco_json(dataset_name, cache_path)\n\n json_file = PathManager.get_local_path(self._metadata.json_file)\n with contextlib.redirect_stdout(io.StringIO()):\n self._coco_api = COCO(json_file)\n\n self._kpt_oks_sigmas = cfg.TEST.KEYPOINT_OKS_SIGMAS\n # Test set json files do not contain annotations (evaluation must be\n # performed using the COCO evaluation server).\n self._do_evaluation = \"annotations\" in self._coco_api.dataset\n\n def reset(self):\n self._predictions = []\n\n def _tasks_from_config(self, cfg):\n \"\"\"\n Returns:\n tuple[str]: tasks that can be evaluated under the given configuration.\n \"\"\"\n print(\"_tasks_from_config\")\n tasks = (\"bbox\",)\n if cfg.MODEL.MASK_ON:\n tasks = tasks + (\"segm\",)\n if cfg.MODEL.KEYPOINT_ON:\n tasks = tasks + (\"keypoints\",)\n print(f\"tasks: {tasks}\")\n return tasks\n\n def process(self, inputs, outputs):\n \"\"\"\n Args:\n inputs: the inputs to a COCO model (e.g., GeneralizedRCNN).\n It is a list of dict. Each dict corresponds to an image and\n contains keys like \"height\", \"width\", \"file_name\", \"image_id\".\n outputs: the outputs of a COCO model. It is a list of dicts with key\n \"instances\" that contains :class:`Instances`.\n \"\"\"\n\n for input, output in zip(inputs, outputs):\n prediction = {\"image_id\": input[\"image_id\"]}\n\n # TODO this is ugly\n if \"instances\" in output:\n instances = output[\"instances\"].to(self._cpu_device)\n prediction[\"instances\"] = instances_to_coco_json(instances, input[\"image_id\"])\n if \"proposals\" in output:\n prediction[\"proposals\"] = output[\"proposals\"].to(self._cpu_device)\n self._predictions.append(prediction)\n\n def evaluate(self):\n\n print(\"evaluate\")\n\n if self._distributed:\n comm.synchronize()\n predictions = comm.gather(self._predictions, dst=0)\n predictions = list(itertools.chain(*predictions))\n\n if not comm.is_main_process():\n return {}\n else:\n predictions = self._predictions\n\n if len(predictions) == 0:\n self._logger.warning(\"[COCOEvaluator] Did not receive valid predictions.\")\n return {}\n\n if self._output_dir:\n PathManager.mkdirs(self._output_dir)\n file_path = os.path.join(self._output_dir, \"instances_predictions.pth\")\n with PathManager.open(file_path, \"wb\") as f:\n torch.save(predictions, f)\n\n self._results = OrderedDict()\n if \"proposals\" in predictions[0]:\n self._eval_box_proposals(predictions)\n if \"instances\" in predictions[0]:\n self._eval_predictions(set(self._tasks), predictions)\n # Copy so the caller can do whatever with results\n return copy.deepcopy(self._results)\n\n def _eval_predictions(self, tasks, predictions):\n \"\"\"\n Evaluate predictions on the given tasks.\n Fill self._results with the metrics of the tasks.\n \"\"\"\n\n print(\"_eval_predictions\")\n print(f\"use_fast_impl: {_use_fast_impl}\")\n\n self._logger.info(\"Preparing results for COCO format ...\")\n coco_results = list(itertools.chain(*[x[\"instances\"] for x in predictions]))\n\n # unmap the category ids for COCO\n if hasattr(self._metadata, \"thing_dataset_id_to_contiguous_id\"):\n reverse_id_mapping = {\n v: k for k, v in self._metadata.thing_dataset_id_to_contiguous_id.items()\n }\n for result in coco_results:\n category_id = result[\"category_id\"]\n assert (\n category_id in reverse_id_mapping\n ), \"A prediction has category_id={}, which is not available in the dataset.\".format(\n category_id\n )\n result[\"category_id\"] = reverse_id_mapping[category_id]\n\n if self._output_dir:\n file_path = os.path.join(self._output_dir, \"coco_instances_results.json\")\n self._logger.info(\"Saving results to {}\".format(file_path))\n with PathManager.open(file_path, \"w\") as f:\n f.write(json.dumps(coco_results))\n f.flush()\n\n if not self._do_evaluation:\n self._logger.info(\"Annotations are not available for evaluation.\")\n return\n\n self._logger.info(\"Evaluating predictions ...\")\n for task in sorted(tasks):\n coco_eval = (\n _evaluate_predictions_on_coco(\n self._coco_api, coco_results, task, kpt_oks_sigmas=self._kpt_oks_sigmas\n )\n if len(coco_results) > 0\n else None # cocoapi does not handle empty results very well\n )\n\n res = self._derive_coco_results(\n coco_eval, task, class_names=self._metadata.get(\"thing_classes\")\n )\n self._results[task] = res\n\n def _eval_box_proposals(self, predictions):\n \"\"\"\n Evaluate the box proposals in predictions.\n Fill self._results with the metrics for \"box_proposals\" task.\n \"\"\"\n print(\"_eval_box_proposals\")\n if self._output_dir:\n # Saving generated box proposals to file.\n # Predicted box_proposals are in XYXY_ABS mode.\n bbox_mode = BoxMode.XYXY_ABS.value\n ids, boxes, objectness_logits = [], [], []\n for prediction in predictions:\n ids.append(prediction[\"image_id\"])\n boxes.append(prediction[\"proposals\"].proposal_boxes.tensor.numpy())\n objectness_logits.append(prediction[\"proposals\"].objectness_logits.numpy())\n\n proposal_data = {\n \"boxes\": boxes,\n \"objectness_logits\": objectness_logits,\n \"ids\": ids,\n \"bbox_mode\": bbox_mode,\n }\n with PathManager.open(os.path.join(self._output_dir, \"box_proposals.pkl\"), \"wb\") as f:\n pickle.dump(proposal_data, f)\n\n if not self._do_evaluation:\n self._logger.info(\"Annotations are not available for evaluation.\")\n return\n\n self._logger.info(\"Evaluating bbox proposals ...\")\n res = {}\n areas = {\"all\": \"\", \"small\": \"s\", \"medium\": \"m\", \"large\": \"l\"}\n for limit in [100, 1000]:\n for area, suffix in areas.items():\n stats = _evaluate_box_proposals(predictions, self._coco_api, area=area, limit=limit)\n key = \"AR{}@{:d}\".format(suffix, limit)\n res[key] = float(stats[\"ar\"].item() * 100)\n self._logger.info(\"Proposal metrics: \\n\" + create_small_table(res))\n self._results[\"box_proposals\"] = res\n\n def _derive_coco_results(self, coco_eval, iou_type, class_names=None):\n \"\"\"\n Derive the desired score numbers from summarized COCOeval.\n Args:\n coco_eval (None or COCOEval): None represents no predictions from model.\n iou_type (str):\n class_names (None or list[str]): if provided, will use it to predict\n per-category AP.\n Returns:\n a dict of {metric name: score}\n \"\"\"\n print(\"_derive_coco_results\")\n\n metrics = {\n \"bbox\": [\"AP\", \"AP50\", \"AP75\", \"APs\", \"APm\", \"APl\"],\n \"segm\": [\"AP\", \"AP50\", \"AP75\", \"APs\", \"APm\", \"APl\"],\n \"keypoints\": [\"AP\", \"AP50\", \"AP75\", \"APm\", \"APl\"],\n }[iou_type]\n\n if coco_eval is None:\n self._logger.warn(\"No predictions from the model!\")\n return {metric: float(\"nan\") for metric in metrics}\n\n # the standard metrics\n results = {\n metric: float(coco_eval.stats[idx] * 100 if coco_eval.stats[idx] >= 0 else \"nan\")\n for idx, metric in enumerate(metrics)\n }\n self._logger.info(\n \"Evaluation results for {}: \\n\".format(iou_type) + create_small_table(results)\n )\n if not np.isfinite(sum(results.values())):\n self._logger.info(\"Note that some metrics cannot be computed.\")\n\n if class_names is None or len(class_names) <= 1:\n return results\n # Compute per-category AP\n # from https://github.com/facebookresearch/Detectron/blob/a6a835f5b8208c45d0dce217ce9bbda915f44df7/detectron/datasets/json_dataset_evaluator.py#L222-L252 # noqa\n precisions = coco_eval.eval[\"precision\"]\n # precision has dims (iou, recall, cls, area range, max dets)\n assert len(class_names) == precisions.shape[2]\n\n results_per_category = []\n for idx, name in enumerate(class_names):\n # area range index 0: all area ranges\n # max dets index -1: typically 100 per image\n precision = precisions[:, :, idx, 0, -1]\n precision = precision[precision > -1]\n ap = np.mean(precision) if precision.size else float(\"nan\")\n results_per_category.append((\"{}\".format(name), float(ap * 100)))\n\n # tabulate it\n N_COLS = min(6, len(results_per_category) * 2)\n results_flatten = list(itertools.chain(*results_per_category))\n results_2d = itertools.zip_longest(*[results_flatten[i::N_COLS] for i in range(N_COLS)])\n table = tabulate(\n results_2d,\n tablefmt=\"pipe\",\n floatfmt=\".3f\",\n headers=[\"category\", \"AP\"] * (N_COLS // 2),\n numalign=\"left\",\n )\n self._logger.info(\"Per-category {} AP: \\n\".format(iou_type) + table)\n\n results.update({\"AP-\" + name: ap for name, ap in results_per_category})\n return results\n\n\ndef instances_to_coco_json(instances, img_id):\n \"\"\"\n Dump an \"Instances\" object to a COCO-format json that's used for evaluation.\n Args:\n instances (Instances):\n img_id (int): the image id\n Returns:\n list[dict]: list of json annotations in COCO format.\n \"\"\"\n\n num_instance = len(instances)\n if num_instance == 0:\n return []\n\n boxes = instances.pred_boxes.tensor.numpy()\n boxes = BoxMode.convert(boxes, BoxMode.XYXY_ABS, BoxMode.XYWH_ABS)\n boxes = boxes.tolist()\n scores = instances.scores.tolist()\n classes = instances.pred_classes.tolist()\n\n has_mask = instances.has(\"pred_masks\")\n has_mask_scores = instances.has(\"mask_scores\")\n if has_mask:\n # use RLE to encode the masks, because they are too large and takes memory\n # since this evaluator stores outputs of the entire dataset\n rles = [\n mask_util.encode(np.array(mask[:, :, None], order=\"F\", dtype=\"uint8\"))[0]\n for mask in instances.pred_masks\n ]\n for rle in rles:\n # \"counts\" is an array encoded by mask_util as a byte-stream. Python3's\n # json writer which always produces strings cannot serialize a bytestream\n # unless you decode it. Thankfully, utf-8 works out (which is also what\n # the pycocotools/_mask.pyx does).\n rle[\"counts\"] = rle[\"counts\"].decode(\"utf-8\")\n\n if has_mask_scores:\n mask_scores = instances.mask_scores.tolist()\n\n has_keypoints = instances.has(\"pred_keypoints\")\n if has_keypoints:\n keypoints = instances.pred_keypoints\n\n results = []\n for k in range(num_instance):\n result = {\n \"image_id\": img_id,\n \"category_id\": classes[k],\n \"bbox\": boxes[k],\n \"score\": scores[k],\n }\n if has_mask:\n result[\"segmentation\"] = rles[k]\n if has_mask_scores:\n result[\"mask_score\"] = mask_scores[k]\n\n if has_keypoints:\n # In COCO annotations,\n # keypoints coordinates are pixel indices.\n # However our predictions are floating point coordinates.\n # Therefore we subtract 0.5 to be consistent with the annotation format.\n # This is the inverse of data loading logic in `datasets/coco.py`.\n keypoints[k][:, :2] -= 0.5\n result[\"keypoints\"] = keypoints[k].flatten().tolist()\n results.append(result)\n return results\n\n\n# inspired from Detectron:\n# https://github.com/facebookresearch/Detectron/blob/a6a835f5b8208c45d0dce217ce9bbda915f44df7/detectron/datasets/json_dataset_evaluator.py#L255 # noqa\ndef _evaluate_box_proposals(dataset_predictions, coco_api, thresholds=None, area=\"all\", limit=None):\n \"\"\"\n Evaluate detection proposal recall metrics. This function is a much\n faster alternative to the official COCO API recall evaluation code. However,\n it produces slightly different results.\n \"\"\"\n# print(\"_evaluate_box_proposals\")\n # Record max overlap value for each gt box\n # Return vector of overlap values\n areas = {\n \"all\": 0,\n \"small\": 1,\n \"medium\": 2,\n \"large\": 3,\n \"96-128\": 4,\n \"128-256\": 5,\n \"256-512\": 6,\n \"512-inf\": 7,\n }\n\n\n area_ranges = [\n [0 ** 2, 1e5 ** 2], # all\n [0 ** 2, 18 ** 2], # small org: 0 - 32\n [18 ** 2, 31 ** 2], # medium org: 32 - 96\n [31 ** 2, 1e5 ** 2], # large org: 96 - 1e5\n [31 ** 2, 128 ** 2], # org: 96-128\n [128 ** 2, 256 ** 2], # 128-256\n [256 ** 2, 512 ** 2], # 256-512\n [512 ** 2, 1e5 ** 2],\n ] # 512-inf\n\n \"\"\"\n area_ranges = [\n [0 ** 2, 1e5 ** 2], # all\n [0 ** 2, 28 ** 2], # small org: 0 - 32\n [28 ** 2, 94 ** 2], # medium org: 32 - 96\n [94 ** 2, 1e5 ** 2], # large org: 96 - 1e5 - our 64\n [94 ** 2, 128 ** 2], # org: 96-128\n [128 ** 2, 256 ** 2], # 128-256\n [256 ** 2, 512 ** 2], # 256-512\n [512 ** 2, 1e5 ** 2],\n ] # 512-inf\n \"\"\"\n assert area in areas, \"Unknown area range: {}\".format(area)\n area_range = area_ranges[areas[area]]\n gt_overlaps = []\n num_pos = 0\n\n for prediction_dict in dataset_predictions:\n predictions = prediction_dict[\"proposals\"]\n\n # sort predictions in descending order\n # TODO maybe remove this and make it explicit in the documentation\n inds = predictions.objectness_logits.sort(descending=True)[1]\n predictions = predictions[inds]\n\n ann_ids = coco_api.getAnnIds(imgIds=prediction_dict[\"image_id\"])\n anno = coco_api.loadAnns(ann_ids)\n gt_boxes = [\n BoxMode.convert(obj[\"bbox\"], BoxMode.XYWH_ABS, BoxMode.XYXY_ABS)\n for obj in anno\n if obj[\"iscrowd\"] == 0\n ]\n gt_boxes = torch.as_tensor(gt_boxes).reshape(-1, 4) # guard against no boxes\n gt_boxes = Boxes(gt_boxes)\n gt_areas = torch.as_tensor([obj[\"area\"] for obj in anno if obj[\"iscrowd\"] == 0])\n\n if len(gt_boxes) == 0 or len(predictions) == 0:\n continue\n\n valid_gt_inds = (gt_areas >= area_range[0]) & (gt_areas <= area_range[1])\n gt_boxes = gt_boxes[valid_gt_inds]\n\n num_pos += len(gt_boxes)\n\n if len(gt_boxes) == 0:\n continue\n\n if limit is not None and len(predictions) > limit:\n predictions = predictions[:limit]\n\n overlaps = pairwise_iou(predictions.proposal_boxes, gt_boxes)\n\n _gt_overlaps = torch.zeros(len(gt_boxes))\n for j in range(min(len(predictions), len(gt_boxes))):\n # find which proposal box maximally covers each gt box\n # and get the iou amount of coverage for each gt box\n max_overlaps, argmax_overlaps = overlaps.max(dim=0)\n\n # find which gt box is 'best' covered (i.e. 'best' = most iou)\n gt_ovr, gt_ind = max_overlaps.max(dim=0)\n assert gt_ovr >= 0\n # find the proposal box that covers the best covered gt box\n box_ind = argmax_overlaps[gt_ind]\n # record the iou coverage of this gt box\n _gt_overlaps[j] = overlaps[box_ind, gt_ind]\n assert _gt_overlaps[j] == gt_ovr\n # mark the proposal box and the gt box as used\n overlaps[box_ind, :] = -1\n overlaps[:, gt_ind] = -1\n\n # append recorded iou coverage level\n gt_overlaps.append(_gt_overlaps)\n gt_overlaps = torch.cat(gt_overlaps, dim=0)\n gt_overlaps, _ = torch.sort(gt_overlaps)\n\n if thresholds is None:\n step = 0.05\n thresholds = torch.arange(0.5, 0.95 + 1e-5, step, dtype=torch.float32)\n recalls = torch.zeros_like(thresholds)\n # compute recall for each iou threshold\n for i, t in enumerate(thresholds):\n recalls[i] = (gt_overlaps >= t).float().sum() / float(num_pos)\n # ar = 2 * np.trapz(recalls, thresholds)\n ar = recalls.mean()\n return {\n \"ar\": ar,\n \"recalls\": recalls,\n \"thresholds\": thresholds,\n \"gt_overlaps\": gt_overlaps,\n \"num_pos\": num_pos,\n }\n\n\ndef _evaluate_predictions_on_coco(coco_gt, coco_results, iou_type, kpt_oks_sigmas=None, use_fast_impl=False):\n \"\"\"\n Evaluate the coco results using COCOEval API.\n \"\"\"\n# print(\"_evaluate_predictions_on_coco\")\n assert len(coco_results) > 0\n\n #Insert this code to increase the number of detections possible /Christoffer :\n\n def summarize_2(self, all_prec=False):\n '''\n Compute and display summary metrics for evaluation results.\n Note this functin can *only* be applied on the default parameter setting\n '''\n\n print(\"In method\")\n def _summarize(ap=1, iouThr=None, areaRng='all', maxDets=2000):\n p = self.params\n iStr = ' {:<18} {} @[ IoU={:<9} | area={:>6s} | maxDets={:>3d} ] = {:0.3f}'\n titleStr = 'Average Precision' if ap == 1 else 'Average Recall'\n typeStr = '(AP)' if ap == 1 else '(AR)'\n iouStr = '{:0.2f}:{:0.2f}'.format(p.iouThrs[0], p.iouThrs[-1]) \\\n if iouThr is None else '{:0.2f}'.format(iouThr)\n\n aind = [i for i, aRng in enumerate(p.areaRngLbl) if aRng == areaRng]\n mind = [i for i, mDet in enumerate(p.maxDets) if mDet == maxDets]\n if ap == 1:\n # dimension of precision: [TxRxKxAxM]\n s = self.eval['precision']\n # IoU\n if iouThr is not None:\n t = np.where(iouThr == p.iouThrs)[0]\n s = s[t]\n s = s[:, :, :, aind, mind]\n else:\n # dimension of recall: [TxKxAxM]\n s = self.eval['recall']\n if iouThr is not None:\n t = np.where(iouThr == p.iouThrs)[0]\n s = s[t]\n s = s[:, :, aind, mind]\n if len(s[s > -1]) == 0:\n mean_s = -1\n else:\n mean_s = np.mean(s[s > -1])\n print(iStr.format(titleStr, typeStr, iouStr, areaRng, maxDets, mean_s))\n return mean_s\n\n def _summarizeDets():\n\n stats = np.zeros((12,))\n stats[0] = _summarize(1, maxDets=self.params.maxDets[2])\n stats[1] = _summarize(1, iouThr=.5, maxDets=self.params.maxDets[2])\n stats[2] = _summarize(1, iouThr=.75, maxDets=self.params.maxDets[2])\n stats[3] = _summarize(1, areaRng='small', maxDets=self.params.maxDets[2])\n stats[4] = _summarize(1, areaRng='medium', maxDets=self.params.maxDets[2])\n stats[5] = _summarize(1, areaRng='large', maxDets=self.params.maxDets[2])\n stats[6] = _summarize(0, maxDets=self.params.maxDets[0])\n stats[7] = _summarize(0, maxDets=self.params.maxDets[1])\n stats[8] = _summarize(0, maxDets=self.params.maxDets[2])\n stats[9] = _summarize(0, areaRng='small', maxDets=self.params.maxDets[2])\n stats[10] = _summarize(0, areaRng='medium', maxDets=self.params.maxDets[2])\n stats[11] = _summarize(0, areaRng='large', maxDets=self.params.maxDets[2])\n return stats\n\n\n def _summarizeKps():\n stats = np.zeros((10,))\n stats[0] = _summarize(1, maxDets=self.params.maxDets[2])\n stats[1] = _summarize(1, maxDets=self.params.maxDets[2], iouThr=.5)\n stats[2] = _summarize(1, maxDets=self.params.maxDets[2], iouThr=.75)\n stats[3] = _summarize(1, maxDets=self.params.maxDets[2], areaRng='medium')\n stats[4] = _summarize(1, maxDets=self.params.maxDets[2], areaRng='large')\n stats[5] = _summarize(0, maxDets=self.params.maxDets[2])\n stats[6] = _summarize(0, maxDets=self.params.maxDets[2], iouThr=.5)\n stats[7] = _summarize(0, maxDets=self.params.maxDets[2], iouThr=.75)\n stats[8] = _summarize(0, maxDets=self.params.maxDets[2], areaRng='medium')\n stats[9] = _summarize(0, maxDets=self.params.maxDets[2], areaRng='large')\n return stats\n\n if not self.eval:\n raise Exception('Please run accumulate() first')\n iouType = self.params.iouType\n if iouType == 'segm' or iouType == 'bbox':\n summarize = _summarizeDets\n elif iouType == 'keypoints':\n summarize = _summarizeKps\n self.stats = summarize()\n\n\n if iou_type == \"segm\":\n coco_results = copy.deepcopy(coco_results)\n # When evaluating mask AP, if the results contain bbox, cocoapi will\n # use the box area as the area of the instance, instead of the mask area.\n # This leads to a different definition of small/medium/large.\n # We remove the bbox field to let mask AP use mask area.\n # We also replace `score` with `mask_score` when using mask scoring.\n has_mask_scores = \"mask_score\" in coco_results[0]\n\n for c in coco_results:\n c.pop(\"bbox\", None)\n if has_mask_scores:\n c[\"score\"] = c[\"mask_score\"]\n del c[\"mask_score\"]\n\n coco_dt = coco_gt.loadRes(coco_results)\n coco_eval = (COCOeval_opt if _use_fast_impl else COCOeval)(coco_gt, coco_dt, iou_type)\n # Use the COCO default keypoint OKS sigmas unless overrides are specified\n if kpt_oks_sigmas:\n coco_eval.params.kpt_oks_sigmas = np.array(kpt_oks_sigmas)\n\n if iou_type == \"keypoints\":\n num_keypoints = len(coco_results[0][\"keypoints\"]) // 3\n assert len(coco_eval.params.kpt_oks_sigmas) == num_keypoints, (\n \"[COCOEvaluator] The length of cfg.TEST.KEYPOINT_OKS_SIGMAS (default: 17) \"\n \"must be equal to the number of keypoints. However the prediction has {} \"\n \"keypoints! For more information please refer to \"\n \"http://cocodataset.org/#keypoints-eval.\".format(num_keypoints)\n )\n\n coco_eval.params.catIds = [1]\n coco_eval.params.useCats = 0\n coco_eval.params.maxDets = [100, 500, 2000]\n\n coco_eval.params.areaRng = [[0 ** 2, 1e5 ** 2], [0 ** 2, 18 ** 2], [18 ** 2, 31 ** 2], [31 ** 2, 1e5 ** 2]]\n coco_eval.params.areaRngLbl = ['all', 'small', 'medium', 'large']\n\n print(f\"Size parameters: {coco_eval.params.areaRng}\")\n\n coco_eval.summarize = types.MethodType(summarize_2, coco_eval)\n\n coco_eval.evaluate()\n coco_eval.accumulate()\n coco_eval.summarize()\n\n \"\"\"\n Added code to produce precision and recall for all iou levels / Chris\n \"\"\"\n precisions = coco_eval.eval['precision']\n recalls = coco_eval.eval['recall']\n\n # IoU threshold | instances | Categories | areas | max dets\n pre_per_iou = [precisions[iou_idx, :, :, 0, -1].mean() for iou_idx in range(precisions.shape[0])]\n rec_pre_iou = [recalls[iou_idx, :, 0, -1].mean() for iou_idx in range(recalls.shape[0])]\n\n print(f\"Precision and Recall per iou: {coco_eval.params.iouThrs}\")\n print(np.round(np.array(pre_per_iou), 4))\n print(np.round(np.array(rec_pre_iou), 4))\n\n return coco_eval\n" }, { "path": "code/preprocessing.py", "content": "import cv2\nimport numpy as np\n\n#this function is designed to adapt images acquired with other light microscopy modalities\n#in order to enable inference with LIVECell trained models\n\n#input_image = uint8 numpy array\n#magnification_resample_factor = downsample factor needed to make image effective 10X\n# for 40x images, use 0.25\n# for 20x images, use 0.5\n# for 10x images, use 1\n\ndef preprocess(input_image, magnification_downsample_factor=1.0): \n #internal variables\n # median_radius_raw = used in the background illumination pattern estimation. \n # this radius should be larger than the radius of a single cell\n # target_median = 128 -- LIVECell phase contrast images all center around a 128 intensity\n median_radius_raw = 75\n target_median = 128.0\n \n #large median filter kernel size is dependent on resize factor, and must also be odd\n median_radius = round(median_radius_raw*magnification_downsample_factor)\n if median_radius%2==0:\n median_radius=median_radius+1\n\n #scale so mean median image intensity is 128\n input_median = np.median(input_image)\n intensity_scale = target_median/input_median\n output_image = input_image.astype('float')*intensity_scale\n\n #define dimensions of downsampled image image\n dims = input_image.shape\n y = int(dims[0]*magnification_downsample_factor)\n x = int(dims[1]*magnification_downsample_factor)\n\n #apply resizing image to account for different magnifications\n output_image = cv2.resize(output_image, (x,y), interpolation = cv2.INTER_AREA)\n \n #clip here to regular 0-255 range to avoid any odd median filter results\n output_image[output_image > 255] = 255\n output_image[output_image < 0] = 0\n\n #estimate background illumination pattern using the large median filter\n background = cv2.medianBlur(output_image.astype('uint8'), median_radius)\n output_image = output_image.astype('float')/background.astype('float')*target_median\n\n #clipping for zernike phase halo artifacts\n output_image[output_image > 180] = 180\n output_image[output_image < 70] = 70\n output_image = output_image.astype('uint8')\n\n return output_image\n\ndef preprocess_fluorescence(input_image, bInvert=True, magnification_downsample_factor=1.0): \n #invert to bring background up to 128\n img = (255-input_image)/2\n if not bInvert:\n img = 255-img\n output_image = preprocess(img, magnification_downsample_factor=magnification_downsample_factor) \n return output_image" }, { "path": "model/README.md", "content": "# Usage\n\nThe anchor free models used in this benchmark is based on the [centermask2](https://github.com/youngwanLEE/centermask2#evaluation) architecture and the anchor based models are \nbased on the [detectron2-ResNeSt](https://github.com/chongruo/detectron2-ResNeSt/blob/resnest/GETTING_STARTED.md) architectures, \nwhich is both built upon the [detectron2](https://github.com/facebookresearch/detectron2) library.\n\nThe models in the LIVECell paper was trained in on 8 Nvidia V100 GPUS.\nTo help others reproduce our results and use the models for further research, we provide pre-trained models and config files.\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
ArchitectureDatasetBox mAP%Mask mAP%download
Anchor freeLIVECell48.4547.78config | model \n
A17231.4934.57config | model \n
BT-47442.1242.60config | model \n
BV-242.6245.69config | model \n
Huh742.4445.85config | model \n
MCF736.5337.30 config | model \n
SH-SY5Y25.2023.91config | model \n
SkBr364.3565.85config | model \n
SK-OV-346.4349.39config | model \n
Anchor basedLIVECell48.4347.89config | model \n
A17236.3738.02config | model \n
BT-47443.2543.00config | model \n
BV-254.3652.60config | model \n
Huh752.7951.83config | model \n
MCF737.5337.94 config | model \n
SH-SY5Y27.8724.92config | model \n
SkBr364.4165.39config | model \n
SK-OV-353.2954.12config | model \n
\n\nThe box and mask AP presented here is derived by training on either the whole LIVECell dataset or a cell \ncell specific subset, and then evaluated on the corresponding test dataset.\n\nTo use our fully trained models download them from our S3 bucket, and use it together with appropriate config file as \ndescribed below in the [traing and evaluation section](#Training and evaluation)\n\n\n\n# Installation\n\nThe installation takes approximately 30 minutes, and it is recommended to set up different virtual environments for the anchor-free and the anchor-based model if one wants to use both of them. The anchor-based models uses a modified version of detectron2 with the same name which creates conflicts.\n\n## Requirements:\n\n- Linux or macOS with Python ≥ 3.6\n- PyTorch ≥ 1.3\n- torchvision that matches the PyTorch installation. You can install them together at pytorch.org to make sure of this.\n- OpenCV, optional, needed by demo and visualization\n- pycocotools: pip install cython; pip install -U 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'\n\n## Anchor-free model (detectron2 + centermask2)\n\nBuild from source\n````python\npython -m pip install 'git+https://github.com/facebookresearch/detectron2.git'\n````\n\nOr, to install it from a local clone:\n````python\ngit clone https://github.com/facebookresearch/detectron2.git\npython -m pip install -e detectron2\n````\n\n\nOr if you are on macOS\n````python\nCC=clang CXX=clang++ python -m pip install -e detectron2\n````\n\n\nTo install a pre-built detectron for different torch and cuda versions and further information, \nsee the detectron2 [install document](https://github.com/facebookresearch/detectron2/blob/master/INSTALL.md)\n\n### \nRetrive the centermask2 code:\n````python\ngit clone https://github.com/youngwanLEE/centermask2.git\n````\n\nFor further information, on installation and usage, see the [centermask2 documentation](https://github.com/youngwanLEE/centermask2#evaluation)\n\n## Anchor-based (detectron2-ResNeSt)\nDo not install detectron2 as outlined above, and if it is installed, uninstall it or create a new virtual environment where it is not installed.\nThis is due to the reason that the ResNeSt code has cloned detectron2, made changes to the code and still uses detectron2 as the name which \ngenerates conflicts with the original detectron2 code. \n\nRetrive and install the detectron2-ResNeSt code:\n```sh\ngit clone https://github.com/chongruo/detectron2-ResNeSt\npython -m pip install -e detectron2-ResNeSt\n```\nFor further information, on installation and usage, see the [detectron2-ResNeSt documentation](https://github.com/chongruo/detectron2-ResNeSt/blob/resnest/GETTING_STARTED.md)\n\n\n### Common Installation Issues\n- Not compiled with GPU support\" or \"Detectron2 CUDA Compiler: not available\".\n```sh\nCUDA is not found when building detectron2. You should make sure\npython -c 'import torch; from torch.utils.cpp_extension import CUDA_HOME; print(torch.cuda.is_available(), CUDA_HOME)'\nprint valid outputs at the time you build detectron2.\n```\n\n# Training and evaluation\n### Register LIVECell dataset\nUsing a custom dataset such as LIVECell together with the detectron2 code base is done by first registering the dataset\nvia the detectron2 python API. In practice this can be done adding the following code to the train_net.py file in the cloned\ncentermask2 repo:\n\nhttps://detectron2.readthedocs.io/en/latest/tutorials/datasets.html\n````python\nfrom detectron2.data.datasets import register_coco_instances\nregister_coco_instances(dataset_name, {}, /path/coco/annotations.json, path/to/image/dir)\n````\n\nWere dataset_name will be the name of your dataset and will be how you decide what dataset to use in your config file.\nPer default, the config file will point to *TRAIN* and *TEST*, so registering a test dataset as *TEST* will work directly with the\nprovided config files, for other names, make sure to update your config file accordingly.\n\n- In the config file change the dataset entries with the name used to register the dataset.\n- Set the output directory in the config file to save the models and results.\n\n### Train\nTo train a model, change the OUTPUT directory in the config file to where the models and checkpoints should be saved.\nMake sure you follow the previous step and register a TRAIN and TEST dataset, cd into \nthe cloned directory (centermask2 or detectron2-ResNeSt), and run the following code:\n\n````python\npython tools/train_net.py --num-gpus 8 --config-file your_config.yaml\n````\nTo train a model on the dataset defined in *you_config.yaml* with 8 gpus.\n\nTo fine-tune a model on your own dataset, set MODEL.WEIGTS in the config file to point at one of our weight files,\nif you want to finetune our centermask2 model for instance.\n````python\nMODEL:\n WEIGHTS: \"http://livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/models/Anchor_free/ALL/LIVECell_anchor_free_model.pth\"\n````\n \n ### Evaluate\nTo evaluate a model, make sure to register a TEST dataset and point to it in your config file and cd into \nthe cloned directory (centermask2 or detectron2-ResNeSt), then run the following code\n ````python\npython train_net.py --config-file --eval-only MODEL.WEIGHTS \n````\n\nThis will evaluate a model defined in `your_config.yaml` with the weights saved in `/path/to/checkpoint_file.pth`\n\nTo evaluate one of our models, like the centermask2 (anchor-free), you can point directly at the URI link for the weight \nfile.\n\n\n ````python\npython train_net.py --config-file livecell_config.yaml --eval-only MODEL.WEIGHTS http://livecell-dataset.s3.eu-central-1.amazonaws.com/LIVECell_dataset_2021/models/Anchor_free/ALL/LIVECell_anchor_free_model.pth\n````\n\n#### Evaluation script\nThe original evaluation script available in the centermask and detectron2 repo is based on there being no more than 100\ndetections in an image. In our case we can have thousands of annotations and thus the AP evaluation will be off. We \ntherefore provide `coco_evaluation.py` evaluation script in the [code](../code) folder. \n\nTo use this script, go into the `train_net.py` file and remove (or comment out) the current import of `COCOEvaluator`.\nThen import `COCOEvaluator` from the provided `coco_evaluator.py` file instead. This will result in AP evaluation\nsupporting for up to 2000 instances in one image.\n \nThe evaluation script will take approximately 30 minutes to run on our test dataset with a tesla V100 GPU.\nThe output of the evaluation will appear in the terminal, begining with information about the environment, data and \narchitecture used. Then it will start evaluating all the images and summerize the results in the following manner:\n \n````python\n.\n.\n.\n\u001b[32m[11/18 17:19:06 d2.evaluation.evaluator]: \u001b[0mInference done 1557/1564. 0.1733 s / img. ETA=0:00:06\n\u001b[32m[11/18 17:19:11 d2.evaluation.evaluator]: \u001b[0mInference done 1561/1564. 0.1734 s / img. ETA=0:00:02\n\u001b[32m[11/18 17:19:14 d2.evaluation.evaluator]: \u001b[0mTotal inference time: 0:22:23.437057 (0.861730 s / img per device, on 1 devices)\n\u001b[32m[11/18 17:19:14 d2.evaluation.evaluator]: \u001b[0mTotal inference pure compute time: 0:04:30 (0.173426 s / img per device, on 1 devices)\nLoading and preparing results...\nDONE (t=1.12s)\ncreating index...\nindex created!\nSize parameters: [[0, 10000000000.0], [0, 324], [324, 961], [961, 10000000000.0]]\nRunning per image evaluation...\nEvaluate annotation type *bbox*\nCOCOeval_opt.evaluate() finished in 119.67 seconds.\nAccumulating evaluation results...\nCOCOeval_opt.accumulate() finished in 5.86 seconds.\nIn method\n Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=2000 ] = 0.485\n Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=2000 ] = 0.830\n Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=2000 ] = 0.504\n Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=2000 ] = 0.483\n Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=2000 ] = 0.494\n Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=2000 ] = 0.507\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.212\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=500 ] = 0.480\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=2000 ] = 0.569\n Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=2000 ] = 0.531\n Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=2000 ] = 0.602\n Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=2000 ] = 0.672\nLoading and preparing results...\nDONE (t=11.04s)\ncreating index...\nindex created!\nSize parameters: [[0, 10000000000.0], [0, 324], [324, 961], [961, 10000000000.0]]\nRunning per image evaluation...\nEvaluate annotation type *segm*\nCOCOeval_opt.evaluate() finished in 135.80 seconds.\nAccumulating evaluation results...\nCOCOeval_opt.accumulate() finished in 5.78 seconds.\nIn method\n Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=2000 ] = 0.478\n Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=2000 ] = 0.816\n Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=2000 ] = 0.509\n Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=2000 ] = 0.451\n Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=2000 ] = 0.491\n Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=2000 ] = 0.570\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.210\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=500 ] = 0.470\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=2000 ] = 0.547\n Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=2000 ] = 0.516\n Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=2000 ] = 0.565\n Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=2000 ] = 0.649\n\u001b[32m[11/18 17:25:07 d2.engine.defaults]: \u001b[0mEvaluation results for cell_phase_test in csv format:\n\u001b[32m[11/18 17:25:07 d2.evaluation.testing]: \u001b[0mcopypaste: Task: bbox\n\u001b[32m[11/18 17:25:07 d2.evaluation.testing]: \u001b[0mcopypaste: AP,AP50,AP75,APs,APm,APl\n\u001b[32m[11/18 17:25:07 d2.evaluation.testing]: \u001b[0mcopypaste: 48.4529,82.9806,50.4426,48.3240,49.4476,50.7434\n\u001b[32m[11/18 17:25:07 d2.evaluation.testing]: \u001b[0mcopypaste: Task: segm\n\u001b[32m[11/18 17:25:07 d2.evaluation.testing]: \u001b[0mcopypaste: AP,AP50,AP75,APs,APm,APl\n\u001b[32m[11/18 17:25:07 d2.evaluation.testing]: \u001b[0mcopypaste: 47.7810,81.6260,50.8958,45.1110,49.0684,56.9874\n\n````\n \nFor further details on training, testing and inference, \nvisit the [centermask2](https://github.com/youngwanLEE/centermask2#evaluation) or \n[detectron2-ResNeSt](https://github.com/chongruo/detectron2-ResNeSt/blob/resnest/GETTING_STARTED.md) docs\n\n## One-shot usage\nFor LIVECell experiments with zero-shot learning of EVICAN and Cellpose the input images was preprocessed using the \npreprocessing-script preprocessing.py found under the [code folder](../code).\n" }, { "path": "model/anchor_based/a172_config.yaml", "content": "MODEL:\n META_ARCHITECTURE: \"GeneralizedRCNN\"\n WEIGHTS: \"https://s3.us-west-1.wasabisys.com/resnest/detectron/mask_cascade_rcnn_ResNeSt_200_FPN_dcn_syncBN_all_tricks_3x-e1901134.pth\"\n BACKBONE:\n NAME: \"build_resnet_fpn_backbone\"\n FREEZE_AT: 0\n MASK_ON: True\n RESNETS:\n OUT_FEATURES: [\"res2\", \"res3\", \"res4\", \"res5\"]\n DEPTH: 200\n STRIDE_IN_1X1: False\n RADIX: 2\n DEFORM_ON_PER_STAGE: [False, True, True, True] # on Res3,Res4,Res5\n DEFORM_MODULATED: True\n DEFORM_NUM_GROUPS: 2\n NORM: \"SyncBN\"\n\n FPN:\n NORM: \"SyncBN\"\n IN_FEATURES: [\"res2\", \"res3\", \"res4\", \"res5\"]\n ANCHOR_GENERATOR:\n SIZES: [[4], [9], [17], [31], [64], [127]] # One size for each in feature map\n ASPECT_RATIOS: [[0.25, 0.5, 1.0, 2.0, 4.0]] # Three aspect ratios (same for all in feature maps)\n ROI_HEADS:\n NUM_CLASSES: 1\n BATCH_SIZE_PER_IMAGE: 512\n NAME: CascadeROIHeads\n IN_FEATURES: [\"p2\", \"p3\", \"p4\", \"p5\"]\n ROI_BOX_HEAD:\n NAME: \"FastRCNNConvFCHead\"\n NUM_CONV: 4\n NUM_FC: 1\n NORM: \"SyncBN\"\n POOLER_RESOLUTION: 7\n CLS_AGNOSTIC_BBOX_REG: True\n ROI_MASK_HEAD:\n NUM_CONV: 8\n NORM: \"SyncBN\"\n RPN:\n IN_FEATURES: [\"p2\" ,\"p2\", \"p3\", \"p4\", \"p5\", \"p6\"]\n BATCH_SIZE_PER_IMAGE: 256\n POST_NMS_TOPK_TEST: 3000\n POST_NMS_TOPK_TRAIN: 3000\n PRE_NMS_TOPK_TEST: 6000\n PRE_NMS_TOPK_TRAIN: 12000\n RETINANET:\n NUM_CLASSES: 1\n TOPK_CANDIDATES_TEST: 3000\n PIXEL_MEAN: [128, 128, 128]\n PIXEL_STD: [11.578, 11.578, 11.578]\nSOLVER:\n IMS_PER_BATCH: 16\n BASE_LR: 0.02\n STEPS: (17500, 20000)\n MAX_ITER: 30000\n CHECKPOINT_PERIOD: 500\nDATASETS:\n TRAIN: (\"TRAIN\",) #REPLACE TRAIN WITH THE REGISTERED NAME \n TEST: (\"TEST\",) #REPLACE TRAIN WITH THE REGISTERED NAME\n PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000\nINPUT:\n MIN_SIZE_TRAIN: (440, 480, 520, 560, 580, 620)\n\n CROP:\n ENABLED: False\n FORMAT: \"BGR\"\nTEST:\n DETECTIONS_PER_IMAGE: 3000 # 1000\n EVAL_PERIOD: 500\n PRECISE_BN:\n ENABLED: False\n AUG:\n ENABLED: False\nOUTPUT_DIR: \"PATH/TO/SAVE/RESULTS\" # PATH TO SAVE THE OUTPUT RESULTS\nDATALOADER:\n NUM_WORKERS: 8\nVERSION: 2\n\n" }, { "path": "model/anchor_based/bt474_config.yaml", "content": "MODEL:\n META_ARCHITECTURE: \"GeneralizedRCNN\"\n WEIGHTS: \"https://s3.us-west-1.wasabisys.com/resnest/detectron/mask_cascade_rcnn_ResNeSt_200_FPN_dcn_syncBN_all_tricks_3x-e1901134.pth\"\n BACKBONE:\n NAME: \"build_resnet_fpn_backbone\"\n FREEZE_AT: 0\n MASK_ON: True\n RESNETS:\n OUT_FEATURES: [\"res2\", \"res3\", \"res4\", \"res5\"]\n DEPTH: 200\n STRIDE_IN_1X1: False\n RADIX: 2\n DEFORM_ON_PER_STAGE: [False, True, True, True] # on Res3,Res4,Res5\n DEFORM_MODULATED: True\n DEFORM_NUM_GROUPS: 2\n NORM: \"SyncBN\"\n\n FPN:\n NORM: \"SyncBN\"\n IN_FEATURES: [\"res2\", \"res3\", \"res4\", \"res5\"]\n ANCHOR_GENERATOR:\n SIZES: [[4], [9], [17], [31], [64], [127]] # One size for each in feature map\n ASPECT_RATIOS: [[0.25, 0.5, 1.0, 2.0, 4.0]] # Three aspect ratios (same for all in feature maps)\n ROI_HEADS:\n NUM_CLASSES: 1\n BATCH_SIZE_PER_IMAGE: 512\n NAME: CascadeROIHeads\n IN_FEATURES: [\"p2\", \"p3\", \"p4\", \"p5\"]\n ROI_BOX_HEAD:\n NAME: \"FastRCNNConvFCHead\"\n NUM_CONV: 4\n NUM_FC: 1\n NORM: \"SyncBN\"\n POOLER_RESOLUTION: 7\n CLS_AGNOSTIC_BBOX_REG: True\n ROI_MASK_HEAD:\n NUM_CONV: 8\n NORM: \"SyncBN\"\n RPN:\n IN_FEATURES: [\"p2\" ,\"p2\", \"p3\", \"p4\", \"p5\", \"p6\"]\n BATCH_SIZE_PER_IMAGE: 256\n POST_NMS_TOPK_TEST: 3000\n POST_NMS_TOPK_TRAIN: 3000\n PRE_NMS_TOPK_TEST: 6000\n PRE_NMS_TOPK_TRAIN: 12000\n RETINANET:\n NUM_CLASSES: 1\n TOPK_CANDIDATES_TEST: 3000\n PIXEL_MEAN: [128, 128, 128]\n PIXEL_STD: [11.578, 11.578, 11.578]\nSOLVER:\n IMS_PER_BATCH: 16\n BASE_LR: 0.02\n STEPS: (17500, 20000)\n MAX_ITER: 30000\n CHECKPOINT_PERIOD: 500\nDATASETS:\n TRAIN: (\"TRAIN\",) #REPLACE TRAIN WITH THE REGISTERED NAME \n TEST: (\"TEST\",) #REPLACE TRAIN WITH THE REGISTERED NAME\n PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000\nINPUT:\n MIN_SIZE_TRAIN: (440, 480, 520, 560, 580, 620)\n\n CROP:\n ENABLED: False\n FORMAT: \"BGR\"\nTEST:\n DETECTIONS_PER_IMAGE: 3000 # 1000\n EVAL_PERIOD: 500\n PRECISE_BN:\n ENABLED: False\n AUG:\n ENABLED: False\nOUTPUT_DIR: \"PATH/TO/SAVE/RESULTS\" # PATH TO SAVE THE OUTPUT RESULTS\nDATALOADER:\n NUM_WORKERS: 8\nVERSION: 2\n\n" }, { "path": "model/anchor_based/bv2_config.yaml", "content": "MODEL:\n META_ARCHITECTURE: \"GeneralizedRCNN\"\n WEIGHTS: \"https://s3.us-west-1.wasabisys.com/resnest/detectron/mask_cascade_rcnn_ResNeSt_200_FPN_dcn_syncBN_all_tricks_3x-e1901134.pth\"\n BACKBONE:\n NAME: \"build_resnet_fpn_backbone\"\n FREEZE_AT: 0\n MASK_ON: True\n RESNETS:\n OUT_FEATURES: [\"res2\", \"res3\", \"res4\", \"res5\"]\n DEPTH: 200\n STRIDE_IN_1X1: False\n RADIX: 2\n DEFORM_ON_PER_STAGE: [False, True, True, True] # on Res3,Res4,Res5\n DEFORM_MODULATED: True\n DEFORM_NUM_GROUPS: 2\n NORM: \"SyncBN\"\n\n FPN:\n NORM: \"SyncBN\"\n IN_FEATURES: [\"res2\", \"res3\", \"res4\", \"res5\"]\n ANCHOR_GENERATOR:\n SIZES: [[4], [9], [17], [31], [64], [127]] # One size for each in feature map\n ASPECT_RATIOS: [[0.25, 0.5, 1.0, 2.0, 4.0]] # Three aspect ratios (same for all in feature maps)\n ROI_HEADS:\n NUM_CLASSES: 1\n BATCH_SIZE_PER_IMAGE: 512\n NAME: CascadeROIHeads\n IN_FEATURES: [\"p2\", \"p3\", \"p4\", \"p5\"]\n ROI_BOX_HEAD:\n NAME: \"FastRCNNConvFCHead\"\n NUM_CONV: 4\n NUM_FC: 1\n NORM: \"SyncBN\"\n POOLER_RESOLUTION: 7\n CLS_AGNOSTIC_BBOX_REG: True\n ROI_MASK_HEAD:\n NUM_CONV: 8\n NORM: \"SyncBN\"\n RPN:\n IN_FEATURES: [\"p2\" ,\"p2\", \"p3\", \"p4\", \"p5\", \"p6\"]\n BATCH_SIZE_PER_IMAGE: 256\n POST_NMS_TOPK_TEST: 3000\n POST_NMS_TOPK_TRAIN: 3000\n PRE_NMS_TOPK_TEST: 6000\n PRE_NMS_TOPK_TRAIN: 12000\n RETINANET:\n NUM_CLASSES: 1\n TOPK_CANDIDATES_TEST: 3000\n PIXEL_MEAN: [128, 128, 128]\n PIXEL_STD: [11.578, 11.578, 11.578]\nSOLVER:\n IMS_PER_BATCH: 16\n BASE_LR: 0.02\n STEPS: (17500, 20000)\n MAX_ITER: 30000\n CHECKPOINT_PERIOD: 500\nDATASETS:\n TRAIN: (\"TRAIN\",) #REPLACE TRAIN WITH THE REGISTERED NAME \n TEST: (\"TEST\",) #REPLACE TRAIN WITH THE REGISTERED NAME\n PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000\nINPUT:\n MIN_SIZE_TRAIN: (440, 480, 520, 560, 580, 620)\n\n CROP:\n ENABLED: False\n FORMAT: \"BGR\"\nTEST:\n DETECTIONS_PER_IMAGE: 3000 # 1000\n EVAL_PERIOD: 500\n PRECISE_BN:\n ENABLED: False\n AUG:\n ENABLED: False\nOUTPUT_DIR: \"PATH/TO/SAVE/RESULTS\" # PATH TO SAVE THE OUTPUT RESULTS\nDATALOADER:\n NUM_WORKERS: 8\nVERSION: 2\n\n" }, { "path": "model/anchor_based/huh7_config.yaml", "content": "MODEL:\n META_ARCHITECTURE: \"GeneralizedRCNN\"\n WEIGHTS: \"https://s3.us-west-1.wasabisys.com/resnest/detectron/mask_cascade_rcnn_ResNeSt_200_FPN_dcn_syncBN_all_tricks_3x-e1901134.pth\"\n BACKBONE:\n NAME: \"build_resnet_fpn_backbone\"\n FREEZE_AT: 0\n MASK_ON: True\n RESNETS:\n OUT_FEATURES: [\"res2\", \"res3\", \"res4\", \"res5\"]\n DEPTH: 200\n STRIDE_IN_1X1: False\n RADIX: 2\n DEFORM_ON_PER_STAGE: [False, True, True, True] # on Res3,Res4,Res5\n DEFORM_MODULATED: True\n DEFORM_NUM_GROUPS: 2\n NORM: \"SyncBN\"\n\n FPN:\n NORM: \"SyncBN\"\n IN_FEATURES: [\"res2\", \"res3\", \"res4\", \"res5\"]\n ANCHOR_GENERATOR:\n SIZES: [[4], [9], [17], [31], [64], [127]] # One size for each in feature map\n ASPECT_RATIOS: [[0.25, 0.5, 1.0, 2.0, 4.0]] # Three aspect ratios (same for all in feature maps)\n ROI_HEADS:\n NUM_CLASSES: 1\n BATCH_SIZE_PER_IMAGE: 512\n NAME: CascadeROIHeads\n IN_FEATURES: [\"p2\", \"p3\", \"p4\", \"p5\"]\n ROI_BOX_HEAD:\n NAME: \"FastRCNNConvFCHead\"\n NUM_CONV: 4\n NUM_FC: 1\n NORM: \"SyncBN\"\n POOLER_RESOLUTION: 7\n CLS_AGNOSTIC_BBOX_REG: True\n ROI_MASK_HEAD:\n NUM_CONV: 8\n NORM: \"SyncBN\"\n RPN:\n IN_FEATURES: [\"p2\" ,\"p2\", \"p3\", \"p4\", \"p5\", \"p6\"]\n BATCH_SIZE_PER_IMAGE: 256\n POST_NMS_TOPK_TEST: 3000\n POST_NMS_TOPK_TRAIN: 3000\n PRE_NMS_TOPK_TEST: 6000\n PRE_NMS_TOPK_TRAIN: 12000\n RETINANET:\n NUM_CLASSES: 1\n TOPK_CANDIDATES_TEST: 3000\n PIXEL_MEAN: [128, 128, 128]\n PIXEL_STD: [11.578, 11.578, 11.578]\nSOLVER:\n IMS_PER_BATCH: 16\n BASE_LR: 0.02\n STEPS: (17500, 20000)\n MAX_ITER: 30000\n CHECKPOINT_PERIOD: 500\nDATASETS:\n TRAIN: (\"TRAIN\",) #REPLACE TRAIN WITH THE REGISTERED NAME \n TEST: (\"TEST\",) #REPLACE TRAIN WITH THE REGISTERED NAME\n PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000\nINPUT:\n MIN_SIZE_TRAIN: (440, 480, 520, 560, 580, 620)\n\n CROP:\n ENABLED: False\n FORMAT: \"BGR\"\nTEST:\n DETECTIONS_PER_IMAGE: 3000 # 1000\n EVAL_PERIOD: 500\n PRECISE_BN:\n ENABLED: False\n AUG:\n ENABLED: False\nOUTPUT_DIR: \"PATH/TO/SAVE/RESULTS\" # PATH TO SAVE THE OUTPUT RESULTS\nDATALOADER:\n NUM_WORKERS: 8\nVERSION: 2\n\n" }, { "path": "model/anchor_based/livecell_config.yaml", "content": "MODEL:\n META_ARCHITECTURE: \"GeneralizedRCNN\"\n WEIGHTS: \"https://s3.us-west-1.wasabisys.com/resnest/detectron/mask_cascade_rcnn_ResNeSt_200_FPN_dcn_syncBN_all_tricks_3x-e1901134.pth\"\n BACKBONE:\n NAME: \"build_resnet_fpn_backbone\"\n FREEZE_AT: 0\n MASK_ON: True\n RESNETS:\n OUT_FEATURES: [\"res2\", \"res3\", \"res4\", \"res5\"]\n DEPTH: 200\n STRIDE_IN_1X1: False\n RADIX: 2\n DEFORM_ON_PER_STAGE: [False, True, True, True] # on Res3,Res4,Res5\n DEFORM_MODULATED: True\n DEFORM_NUM_GROUPS: 2\n NORM: \"SyncBN\"\n\n FPN:\n NORM: \"SyncBN\"\n IN_FEATURES: [\"res2\", \"res3\", \"res4\", \"res5\"]\n ANCHOR_GENERATOR:\n SIZES: [[4], [9], [17], [31], [64], [127]] # One size for each in feature map\n ASPECT_RATIOS: [[0.25, 0.5, 1.0, 2.0, 4.0]] # Three aspect ratios (same for all in feature maps)\n ROI_HEADS:\n NUM_CLASSES: 1\n BATCH_SIZE_PER_IMAGE: 512\n NAME: CascadeROIHeads\n IN_FEATURES: [\"p2\", \"p3\", \"p4\", \"p5\"]\n ROI_BOX_HEAD:\n NAME: \"FastRCNNConvFCHead\"\n NUM_CONV: 4\n NUM_FC: 1\n NORM: \"SyncBN\"\n POOLER_RESOLUTION: 7\n CLS_AGNOSTIC_BBOX_REG: True\n ROI_MASK_HEAD:\n NUM_CONV: 8\n NORM: \"SyncBN\"\n RPN:\n IN_FEATURES: [\"p2\" ,\"p2\", \"p3\", \"p4\", \"p5\", \"p6\"]\n BATCH_SIZE_PER_IMAGE: 256\n POST_NMS_TOPK_TEST: 3000\n POST_NMS_TOPK_TRAIN: 3000\n PRE_NMS_TOPK_TEST: 6000\n PRE_NMS_TOPK_TRAIN: 12000\n RETINANET:\n NUM_CLASSES: 1\n TOPK_CANDIDATES_TEST: 3000\n PIXEL_MEAN: [128, 128, 128]\n PIXEL_STD: [11.578, 11.578, 11.578]\nSOLVER:\n IMS_PER_BATCH: 16\n BASE_LR: 0.02\n STEPS: (17500, 20000)\n MAX_ITER: 30000\n CHECKPOINT_PERIOD: 500\nDATASETS:\n TRAIN: (\"TRAIN\",) #REPLACE TRAIN WITH THE REGISTERED NAME \n TEST: (\"TEST\",) #REPLACE TRAIN WITH THE REGISTERED NAME\n PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000\nINPUT:\n MIN_SIZE_TRAIN: (440, 480, 520, 560, 580, 620)\n\n CROP:\n ENABLED: False\n FORMAT: \"BGR\"\nTEST:\n DETECTIONS_PER_IMAGE: 3000 # 1000\n EVAL_PERIOD: 500\n PRECISE_BN:\n ENABLED: False\n AUG:\n ENABLED: False\nOUTPUT_DIR: \"PATH/TO/SAVE/RESULTS\" # PATH TO SAVE THE OUTPUT RESULTS\nDATALOADER:\n NUM_WORKERS: 8\nVERSION: 2\n\n" }, { "path": "model/anchor_based/mcf7_config.yaml", "content": "MODEL:\n META_ARCHITECTURE: \"GeneralizedRCNN\"\n WEIGHTS: \"https://s3.us-west-1.wasabisys.com/resnest/detectron/mask_cascade_rcnn_ResNeSt_200_FPN_dcn_syncBN_all_tricks_3x-e1901134.pth\"\n BACKBONE:\n NAME: \"build_resnet_fpn_backbone\"\n FREEZE_AT: 0\n MASK_ON: True\n RESNETS:\n OUT_FEATURES: [\"res2\", \"res3\", \"res4\", \"res5\"]\n DEPTH: 200\n STRIDE_IN_1X1: False\n RADIX: 2\n DEFORM_ON_PER_STAGE: [False, True, True, True] # on Res3,Res4,Res5\n DEFORM_MODULATED: True\n DEFORM_NUM_GROUPS: 2\n NORM: \"SyncBN\"\n\n FPN:\n NORM: \"SyncBN\"\n IN_FEATURES: [\"res2\", \"res3\", \"res4\", \"res5\"]\n ANCHOR_GENERATOR:\n SIZES: [[4], [9], [17], [31], [64], [127]] # One size for each in feature map\n ASPECT_RATIOS: [[0.25, 0.5, 1.0, 2.0, 4.0]] # Three aspect ratios (same for all in feature maps)\n ROI_HEADS:\n NUM_CLASSES: 1\n BATCH_SIZE_PER_IMAGE: 512\n NAME: CascadeROIHeads\n IN_FEATURES: [\"p2\", \"p3\", \"p4\", \"p5\"]\n ROI_BOX_HEAD:\n NAME: \"FastRCNNConvFCHead\"\n NUM_CONV: 4\n NUM_FC: 1\n NORM: \"SyncBN\"\n POOLER_RESOLUTION: 7\n CLS_AGNOSTIC_BBOX_REG: True\n ROI_MASK_HEAD:\n NUM_CONV: 8\n NORM: \"SyncBN\"\n RPN:\n IN_FEATURES: [\"p2\" ,\"p2\", \"p3\", \"p4\", \"p5\", \"p6\"]\n BATCH_SIZE_PER_IMAGE: 256\n POST_NMS_TOPK_TEST: 3000\n POST_NMS_TOPK_TRAIN: 3000\n PRE_NMS_TOPK_TEST: 6000\n PRE_NMS_TOPK_TRAIN: 12000\n RETINANET:\n NUM_CLASSES: 1\n TOPK_CANDIDATES_TEST: 3000\n PIXEL_MEAN: [128, 128, 128]\n PIXEL_STD: [11.578, 11.578, 11.578]\nSOLVER:\n IMS_PER_BATCH: 16\n BASE_LR: 0.02\n STEPS: (17500, 20000)\n MAX_ITER: 30000\n CHECKPOINT_PERIOD: 500\nDATASETS:\n TRAIN: (\"TRAIN\",) #REPLACE TRAIN WITH THE REGISTERED NAME \n TEST: (\"TEST\",) #REPLACE TRAIN WITH THE REGISTERED NAME\n PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000\nINPUT:\n MIN_SIZE_TRAIN: (440, 480, 520, 560, 580, 620)\n\n CROP:\n ENABLED: False\n FORMAT: \"BGR\"\nTEST:\n DETECTIONS_PER_IMAGE: 3000 # 1000\n EVAL_PERIOD: 500\n PRECISE_BN:\n ENABLED: False\n AUG:\n ENABLED: False\nOUTPUT_DIR: \"PATH/TO/SAVE/RESULTS\" # PATH TO SAVE THE OUTPUT RESULTS\nDATALOADER:\n NUM_WORKERS: 8\nVERSION: 2\n\n" }, { "path": "model/anchor_based/shsy5y_config.yaml", "content": "MODEL:\n META_ARCHITECTURE: \"GeneralizedRCNN\"\n WEIGHTS: \"https://s3.us-west-1.wasabisys.com/resnest/detectron/mask_cascade_rcnn_ResNeSt_200_FPN_dcn_syncBN_all_tricks_3x-e1901134.pth\"\n BACKBONE:\n NAME: \"build_resnet_fpn_backbone\"\n FREEZE_AT: 0\n MASK_ON: True\n RESNETS:\n OUT_FEATURES: [\"res2\", \"res3\", \"res4\", \"res5\"]\n DEPTH: 200\n STRIDE_IN_1X1: False\n RADIX: 2\n DEFORM_ON_PER_STAGE: [False, True, True, True] # on Res3,Res4,Res5\n DEFORM_MODULATED: True\n DEFORM_NUM_GROUPS: 2\n NORM: \"SyncBN\"\n\n FPN:\n NORM: \"SyncBN\"\n IN_FEATURES: [\"res2\", \"res3\", \"res4\", \"res5\"]\n ANCHOR_GENERATOR:\n SIZES: [[4], [9], [17], [31], [64], [127]] # One size for each in feature map\n ASPECT_RATIOS: [[0.25, 0.5, 1.0, 2.0, 4.0]] # Three aspect ratios (same for all in feature maps)\n ROI_HEADS:\n NUM_CLASSES: 1\n BATCH_SIZE_PER_IMAGE: 512\n NAME: CascadeROIHeads\n IN_FEATURES: [\"p2\", \"p3\", \"p4\", \"p5\"]\n ROI_BOX_HEAD:\n NAME: \"FastRCNNConvFCHead\"\n NUM_CONV: 4\n NUM_FC: 1\n NORM: \"SyncBN\"\n POOLER_RESOLUTION: 7\n CLS_AGNOSTIC_BBOX_REG: True\n ROI_MASK_HEAD:\n NUM_CONV: 8\n NORM: \"SyncBN\"\n RPN:\n IN_FEATURES: [\"p2\" ,\"p2\", \"p3\", \"p4\", \"p5\", \"p6\"]\n BATCH_SIZE_PER_IMAGE: 256\n POST_NMS_TOPK_TEST: 3000\n POST_NMS_TOPK_TRAIN: 3000\n PRE_NMS_TOPK_TEST: 6000\n PRE_NMS_TOPK_TRAIN: 12000\n RETINANET:\n NUM_CLASSES: 1\n TOPK_CANDIDATES_TEST: 3000\n PIXEL_MEAN: [128, 128, 128]\n PIXEL_STD: [11.578, 11.578, 11.578]\nSOLVER:\n IMS_PER_BATCH: 16\n BASE_LR: 0.02\n STEPS: (17500, 20000)\n MAX_ITER: 30000\n CHECKPOINT_PERIOD: 500\nDATASETS:\n TRAIN: (\"TRAIN\",) #REPLACE TRAIN WITH THE REGISTERED NAME \n TEST: (\"TEST\",) #REPLACE TRAIN WITH THE REGISTERED NAME\n PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000\nINPUT:\n MIN_SIZE_TRAIN: (440, 480, 520, 560, 580, 620)\n\n CROP:\n ENABLED: False\n FORMAT: \"BGR\"\nTEST:\n DETECTIONS_PER_IMAGE: 3000 # 1000\n EVAL_PERIOD: 500\n PRECISE_BN:\n ENABLED: False\n AUG:\n ENABLED: False\nOUTPUT_DIR: \"PATH/TO/SAVE/RESULTS\" # PATH TO SAVE THE OUTPUT RESULTS\nDATALOADER:\n NUM_WORKERS: 8\nVERSION: 2\n\n" }, { "path": "model/anchor_based/skbr3_config.yaml", "content": "MODEL:\n META_ARCHITECTURE: \"GeneralizedRCNN\"\n WEIGHTS: \"https://s3.us-west-1.wasabisys.com/resnest/detectron/mask_cascade_rcnn_ResNeSt_200_FPN_dcn_syncBN_all_tricks_3x-e1901134.pth\"\n BACKBONE:\n NAME: \"build_resnet_fpn_backbone\"\n FREEZE_AT: 0\n MASK_ON: True\n RESNETS:\n OUT_FEATURES: [\"res2\", \"res3\", \"res4\", \"res5\"]\n DEPTH: 200\n STRIDE_IN_1X1: False\n RADIX: 2\n DEFORM_ON_PER_STAGE: [False, True, True, True] # on Res3,Res4,Res5\n DEFORM_MODULATED: True\n DEFORM_NUM_GROUPS: 2\n NORM: \"SyncBN\"\n\n FPN:\n NORM: \"SyncBN\"\n IN_FEATURES: [\"res2\", \"res3\", \"res4\", \"res5\"]\n ANCHOR_GENERATOR:\n SIZES: [[4], [9], [17], [31], [64], [127]] # One size for each in feature map\n ASPECT_RATIOS: [[0.25, 0.5, 1.0, 2.0, 4.0]] # Three aspect ratios (same for all in feature maps)\n ROI_HEADS:\n NUM_CLASSES: 1\n BATCH_SIZE_PER_IMAGE: 512\n NAME: CascadeROIHeads\n IN_FEATURES: [\"p2\", \"p3\", \"p4\", \"p5\"]\n ROI_BOX_HEAD:\n NAME: \"FastRCNNConvFCHead\"\n NUM_CONV: 4\n NUM_FC: 1\n NORM: \"SyncBN\"\n POOLER_RESOLUTION: 7\n CLS_AGNOSTIC_BBOX_REG: True\n ROI_MASK_HEAD:\n NUM_CONV: 8\n NORM: \"SyncBN\"\n RPN:\n IN_FEATURES: [\"p2\" ,\"p2\", \"p3\", \"p4\", \"p5\", \"p6\"]\n BATCH_SIZE_PER_IMAGE: 256\n POST_NMS_TOPK_TEST: 3000\n POST_NMS_TOPK_TRAIN: 3000\n PRE_NMS_TOPK_TEST: 6000\n PRE_NMS_TOPK_TRAIN: 12000\n RETINANET:\n NUM_CLASSES: 1\n TOPK_CANDIDATES_TEST: 3000\n PIXEL_MEAN: [128, 128, 128]\n PIXEL_STD: [11.578, 11.578, 11.578]\nSOLVER:\n IMS_PER_BATCH: 16\n BASE_LR: 0.02\n STEPS: (17500, 20000)\n MAX_ITER: 30000\n CHECKPOINT_PERIOD: 500\nDATASETS:\n TRAIN: (\"TRAIN\",) #REPLACE TRAIN WITH THE REGISTERED NAME \n TEST: (\"TEST\",) #REPLACE TRAIN WITH THE REGISTERED NAME\n PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000\nINPUT:\n MIN_SIZE_TRAIN: (440, 480, 520, 560, 580, 620)\n\n CROP:\n ENABLED: False\n FORMAT: \"BGR\"\nTEST:\n DETECTIONS_PER_IMAGE: 3000 # 1000\n EVAL_PERIOD: 500\n PRECISE_BN:\n ENABLED: False\n AUG:\n ENABLED: False\nOUTPUT_DIR: \"PATH/TO/SAVE/RESULTS\" # PATH TO SAVE THE OUTPUT RESULTS\nDATALOADER:\n NUM_WORKERS: 8\nVERSION: 2\n\n" }, { "path": "model/anchor_based/skov3_config.yaml", "content": "MODEL:\n META_ARCHITECTURE: \"GeneralizedRCNN\"\n WEIGHTS: \"https://s3.us-west-1.wasabisys.com/resnest/detectron/mask_cascade_rcnn_ResNeSt_200_FPN_dcn_syncBN_all_tricks_3x-e1901134.pth\"\n BACKBONE:\n NAME: \"build_resnet_fpn_backbone\"\n FREEZE_AT: 0\n MASK_ON: True\n RESNETS:\n OUT_FEATURES: [\"res2\", \"res3\", \"res4\", \"res5\"]\n DEPTH: 200\n STRIDE_IN_1X1: False\n RADIX: 2\n DEFORM_ON_PER_STAGE: [False, True, True, True] # on Res3,Res4,Res5\n DEFORM_MODULATED: True\n DEFORM_NUM_GROUPS: 2\n NORM: \"SyncBN\"\n\n FPN:\n NORM: \"SyncBN\"\n IN_FEATURES: [\"res2\", \"res3\", \"res4\", \"res5\"]\n ANCHOR_GENERATOR:\n SIZES: [[4], [9], [17], [31], [64], [127]] # One size for each in feature map\n ASPECT_RATIOS: [[0.25, 0.5, 1.0, 2.0, 4.0]] # Three aspect ratios (same for all in feature maps)\n ROI_HEADS:\n NUM_CLASSES: 1\n BATCH_SIZE_PER_IMAGE: 512\n NAME: CascadeROIHeads\n IN_FEATURES: [\"p2\", \"p3\", \"p4\", \"p5\"]\n ROI_BOX_HEAD:\n NAME: \"FastRCNNConvFCHead\"\n NUM_CONV: 4\n NUM_FC: 1\n NORM: \"SyncBN\"\n POOLER_RESOLUTION: 7\n CLS_AGNOSTIC_BBOX_REG: True\n ROI_MASK_HEAD:\n NUM_CONV: 8\n NORM: \"SyncBN\"\n RPN:\n IN_FEATURES: [\"p2\" ,\"p2\", \"p3\", \"p4\", \"p5\", \"p6\"]\n BATCH_SIZE_PER_IMAGE: 256\n POST_NMS_TOPK_TEST: 3000\n POST_NMS_TOPK_TRAIN: 3000\n PRE_NMS_TOPK_TEST: 6000\n PRE_NMS_TOPK_TRAIN: 12000\n RETINANET:\n NUM_CLASSES: 1\n TOPK_CANDIDATES_TEST: 3000\n PIXEL_MEAN: [128, 128, 128]\n PIXEL_STD: [11.578, 11.578, 11.578]\nSOLVER:\n IMS_PER_BATCH: 16\n BASE_LR: 0.02\n STEPS: (17500, 20000)\n MAX_ITER: 30000\n CHECKPOINT_PERIOD: 500\nDATASETS:\n TRAIN: (\"TRAIN\",) #REPLACE TRAIN WITH THE REGISTERED NAME \n TEST: (\"TEST\",) #REPLACE TRAIN WITH THE REGISTERED NAME\n PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000\nINPUT:\n MIN_SIZE_TRAIN: (440, 480, 520, 560, 580, 620)\n\n CROP:\n ENABLED: False\n FORMAT: \"BGR\"\nTEST:\n DETECTIONS_PER_IMAGE: 3000 # 1000\n EVAL_PERIOD: 500\n PRECISE_BN:\n ENABLED: False\n AUG:\n ENABLED: False\nOUTPUT_DIR: \"PATH/TO/SAVE/RESULTS\" # PATH TO SAVE THE OUTPUT RESULTS\nDATALOADER:\n NUM_WORKERS: 8\nVERSION: 2\n\n" }, { "path": "model/anchor_free/Base-CenterMask-VoVNet.yaml", "content": "MODEL:\n META_ARCHITECTURE: \"GeneralizedRCNN\"\n BACKBONE:\n NAME: \"build_fcos_vovnet_fpn_backbone\"\n FREEZE_AT: 0\n VOVNET:\n OUT_FEATURES: [\"stage3\", \"stage4\", \"stage5\"]\n FPN:\n IN_FEATURES: [\"stage3\", \"stage4\", \"stage5\"]\n PROPOSAL_GENERATOR:\n NAME: \"FCOS\" \n FCOS:\n POST_NMS_TOPK_TEST: 50\n # PIXEL_MEAN: [102.9801, 115.9465, 122.7717]\n MASK_ON: True\n MASKIOU_ON: True\n ROI_HEADS:\n NAME: \"CenterROIHeads\"\n IN_FEATURES: [\"p3\", \"p4\", \"p5\"]\n ROI_MASK_HEAD:\n NAME: \"SpatialAttentionMaskHead\"\n ASSIGN_CRITERION: \"ratio\"\n NUM_CONV: 4\n POOLER_RESOLUTION: 14\nDATASETS:\n TRAIN: (\"coco_2017_train\",)\n TEST: (\"coco_2017_val\",)\nSOLVER:\n CHECKPOINT_PERIOD: 10000\n IMS_PER_BATCH: 16\n BASE_LR: 0.01 # Note that RetinaNet uses a different default learning rate\n STEPS: (60000, 80000)\n MAX_ITER: 90000\nINPUT:\n MIN_SIZE_TRAIN: (640, 672, 704, 736, 768, 800)\n" }, { "path": "model/anchor_free/a172_config.yaml", "content": "MODEL:\n META_ARCHITECTURE: \"GeneralizedRCNN\"\n BACKBONE:\n NAME: \"build_fcos_vovnet_fpn_backbone\"\n FREEZE_AT: 0\n WEIGHTS: \"https://www.dropbox.com/s/1mlv31coewx8trd/vovnet99_ese_detectron2.pth?dl=1\"\n VOVNET:\n CONV_BODY : \"V-99-eSE\"\n OUT_FEATURES: [\"stage3\", \"stage4\", \"stage5\"]\n FPN:\n IN_FEATURES: [\"stage3\", \"stage4\", \"stage5\"]\n PROPOSAL_GENERATOR:\n NAME: \"FCOS\" \n MASK_ON: True\n MASKIOU_ON: True\n FCOS:\n NUM_CLASSES: 1\n PRE_NMS_TOPK_TRAIN: 2000\n PRE_NMS_TOPK_TEST: 4000\n POST_NMS_TOPK_TRAIN: 15000\n POST_NMS_TOPK_TEST: 3000\n RPN:\n BATCH_SIZE_PER_IMAGE: 3000\n POST_NMS_TOPK_TEST: 3000\n POST_NMS_TOPK_TRAIN: 3000\n ROI_HEADS:\n NUM_CLASSES: 1\n BATCH_SIZE_PER_IMAGE: 3000\n NAME: \"CenterROIHeads\"\n IN_FEATURES: [\"p3\", \"p4\", \"p5\"]\n ROI_MASK_HEAD:\n NAME: \"SpatialAttentionMaskHead\"\n ASSIGN_CRITERION: \"ratio\"\n NUM_CONV: 4\n POOLER_RESOLUTION: 14\n RETINANET:\n NUM_CLASSES: 1\n TOPK_CANDIDATES_TEST: 3000\n PIXEL_MEAN: [128, 128, 128]\n PIXEL_STD: [11.578, 11.578, 11.578]\nSOLVER:\n STEPS: (26000, 28000)\n MAX_ITER: 30000\n IMS_PER_BATCH: 16\n BASE_LR: 0.01\n CHECKPOINT_PERIOD: 1000\nDATASETS:\n TRAIN: (\"TRAIN\",) #REPLACE TRAIN WITH THE REGISTERED NAME \n TEST: (\"TEST\",) #REPLACE TRAIN WITH THE REGISTERED NAME\n PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000\nINPUT:\n MIN_SIZE_TRAIN: (440, 480, 520, 560, 580, 620)\nTEST:\n DETECTIONS_PER_IMAGE: 2000 # 1000\nDATALOADER:\n NUM_WORKERS: 12\nOUTPUT_DIR: \"\"" }, { "path": "model/anchor_free/bt474_config.yaml", "content": "MODEL:\n META_ARCHITECTURE: \"GeneralizedRCNN\"\n BACKBONE:\n NAME: \"build_fcos_vovnet_fpn_backbone\"\n FREEZE_AT: 0\n WEIGHTS: \"https://www.dropbox.com/s/1mlv31coewx8trd/vovnet99_ese_detectron2.pth?dl=1\"\n VOVNET:\n CONV_BODY : \"V-99-eSE\"\n OUT_FEATURES: [\"stage3\", \"stage4\", \"stage5\"]\n FPN:\n IN_FEATURES: [\"stage3\", \"stage4\", \"stage5\"]\n PROPOSAL_GENERATOR:\n NAME: \"FCOS\" \n MASK_ON: True\n MASKIOU_ON: True\n FCOS:\n NUM_CLASSES: 1\n PRE_NMS_TOPK_TRAIN: 2000\n PRE_NMS_TOPK_TEST: 4000\n POST_NMS_TOPK_TRAIN: 15000\n POST_NMS_TOPK_TEST: 3000\n RPN:\n BATCH_SIZE_PER_IMAGE: 3000\n POST_NMS_TOPK_TEST: 3000\n POST_NMS_TOPK_TRAIN: 3000\n ROI_HEADS:\n NUM_CLASSES: 1\n BATCH_SIZE_PER_IMAGE: 3000\n NAME: \"CenterROIHeads\"\n IN_FEATURES: [\"p3\", \"p4\", \"p5\"]\n ROI_MASK_HEAD:\n NAME: \"SpatialAttentionMaskHead\"\n ASSIGN_CRITERION: \"ratio\"\n NUM_CONV: 4\n POOLER_RESOLUTION: 14\n RETINANET:\n NUM_CLASSES: 1\n TOPK_CANDIDATES_TEST: 3000\n PIXEL_MEAN: [128, 128, 128]\n PIXEL_STD: [11.578, 11.578, 11.578]\nSOLVER:\n STEPS: (8000, 9000)\n MAX_ITER: 10000\n IMS_PER_BATCH: 16\n BASE_LR: 0.01 #0.01\n CHECKPOINT_PERIOD: 500\nDATASETS:\n TRAIN: (\"TRAIN\",) #REPLACE TRAIN WITH THE REGISTERED NAME \n TEST: (\"TEST\",) #REPLACE TRAIN WITH THE REGISTERED NAME\n PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000\nINPUT:\n MIN_SIZE_TRAIN: (440, 480, 520, 560, 580, 620)\nTEST:\n DETECTIONS_PER_IMAGE: 2000 # 1000\nDATALOADER:\n NUM_WORKERS: 12\nOUTPUT_DIR: \"\"" }, { "path": "model/anchor_free/bv2_config.yaml", "content": "MODEL:\n META_ARCHITECTURE: \"GeneralizedRCNN\"\n BACKBONE:\n NAME: \"build_fcos_vovnet_fpn_backbone\"\n FREEZE_AT: 0\n WEIGHTS: \"https://www.dropbox.com/s/1mlv31coewx8trd/vovnet99_ese_detectron2.pth?dl=1\"\n VOVNET:\n CONV_BODY : \"V-99-eSE\"\n OUT_FEATURES: [\"stage3\", \"stage4\", \"stage5\"]\n FPN:\n IN_FEATURES: [\"stage3\", \"stage4\", \"stage5\"]\n PROPOSAL_GENERATOR:\n NAME: \"FCOS\" \n MASK_ON: True\n MASKIOU_ON: True\n FCOS:\n NUM_CLASSES: 1\n PRE_NMS_TOPK_TRAIN: 2000\n PRE_NMS_TOPK_TEST: 4000\n POST_NMS_TOPK_TRAIN: 15000\n POST_NMS_TOPK_TEST: 3000\n RPN:\n BATCH_SIZE_PER_IMAGE: 3000\n POST_NMS_TOPK_TEST: 3000\n POST_NMS_TOPK_TRAIN: 3000\n ROI_HEADS:\n NUM_CLASSES: 1\n BATCH_SIZE_PER_IMAGE: 3000\n NAME: \"CenterROIHeads\"\n IN_FEATURES: [\"p3\", \"p4\", \"p5\"]\n ROI_MASK_HEAD:\n NAME: \"SpatialAttentionMaskHead\"\n ASSIGN_CRITERION: \"ratio\"\n NUM_CONV: 4\n POOLER_RESOLUTION: 14\n RETINANET:\n NUM_CLASSES: 1\n TOPK_CANDIDATES_TEST: 3000\n PIXEL_MEAN: [128, 128, 128]\n PIXEL_STD: [11.578, 11.578, 11.578]\nSOLVER:\n STEPS: (8000, 9000)\n MAX_ITER: 10000\n IMS_PER_BATCH: 16\n BASE_LR: 0.01\n CHECKPOINT_PERIOD: 500\nDATASETS:\n TRAIN: (\"TRAIN\",) #REPLACE TRAIN WITH THE REGISTERED NAME \n TEST: (\"TEST\",) #REPLACE TRAIN WITH THE REGISTERED NAME\n PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000\nINPUT:\n MIN_SIZE_TRAIN: (440, 480, 520, 560, 580, 620)\nTEST:\n DETECTIONS_PER_IMAGE: 2000 # 1000\nDATALOADER:\n NUM_WORKERS: 12\nOUTPUT_DIR: \"\"" }, { "path": "model/anchor_free/huh7_config.yaml", "content": "MODEL:\n META_ARCHITECTURE: \"GeneralizedRCNN\"\n BACKBONE:\n NAME: \"build_fcos_vovnet_fpn_backbone\"\n FREEZE_AT: 0\n WEIGHTS: \"https://www.dropbox.com/s/1mlv31coewx8trd/vovnet99_ese_detectron2.pth?dl=1\"\n VOVNET:\n CONV_BODY : \"V-99-eSE\"\n OUT_FEATURES: [\"stage3\", \"stage4\", \"stage5\"]\n FPN:\n IN_FEATURES: [\"stage3\", \"stage4\", \"stage5\"]\n PROPOSAL_GENERATOR:\n NAME: \"FCOS\" \n MASK_ON: True\n MASKIOU_ON: True\n FCOS:\n NUM_CLASSES: 1\n PRE_NMS_TOPK_TRAIN: 2000\n PRE_NMS_TOPK_TEST: 4000\n POST_NMS_TOPK_TRAIN: 15000\n POST_NMS_TOPK_TEST: 3000\n RPN:\n BATCH_SIZE_PER_IMAGE: 3000\n POST_NMS_TOPK_TEST: 3000\n POST_NMS_TOPK_TRAIN: 3000\n ROI_HEADS:\n NUM_CLASSES: 1\n BATCH_SIZE_PER_IMAGE: 3000\n NAME: \"CenterROIHeads\"\n IN_FEATURES: [\"p3\", \"p4\", \"p5\"]\n ROI_MASK_HEAD:\n NAME: \"SpatialAttentionMaskHead\"\n ASSIGN_CRITERION: \"ratio\"\n NUM_CONV: 4\n POOLER_RESOLUTION: 14\n RETINANET:\n NUM_CLASSES: 1\n TOPK_CANDIDATES_TEST: 3000\n PIXEL_MEAN: [128, 128, 128]\n PIXEL_STD: [11.578, 11.578, 11.578]\nSOLVER:\n STEPS: (8000, 9000)\n MAX_ITER: 10000\n IMS_PER_BATCH: 16\n BASE_LR: 0.01\n CHECKPOINT_PERIOD: 500\nDATASETS:\n TRAIN: (\"TRAIN\",) #REPLACE TRAIN WITH THE REGISTERED NAME \n TEST: (\"TEST\",) #REPLACE TRAIN WITH THE REGISTERED NAME\n PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000\nINPUT:\n MIN_SIZE_TRAIN: (440, 480, 520, 560, 580, 620)\nTEST:\n DETECTIONS_PER_IMAGE: 2000 # 1000\nDATALOADER:\n NUM_WORKERS: 12\nOUTPUT_DIR: \"\"" }, { "path": "model/anchor_free/livecell_config.yaml", "content": "MODEL:\n META_ARCHITECTURE: \"GeneralizedRCNN\"\n BACKBONE:\n NAME: \"build_fcos_vovnet_fpn_backbone\"\n FREEZE_AT: 0\n WEIGHTS: \"https://www.dropbox.com/s/1mlv31coewx8trd/vovnet99_ese_detectron2.pth?dl=1\"\n VOVNET:\n CONV_BODY : \"V-99-eSE\"\n OUT_FEATURES: [\"stage3\", \"stage4\", \"stage5\"]\n FPN:\n IN_FEATURES: [\"stage3\", \"stage4\", \"stage5\"]\n PROPOSAL_GENERATOR:\n NAME: \"FCOS\" \n MASK_ON: True\n MASKIOU_ON: True\n FCOS:\n NUM_CLASSES: 1\n PRE_NMS_TOPK_TRAIN: 2000\n PRE_NMS_TOPK_TEST: 4000\n POST_NMS_TOPK_TRAIN: 15000\n POST_NMS_TOPK_TEST: 3000\n RPN:\n BATCH_SIZE_PER_IMAGE: 3000\n POST_NMS_TOPK_TEST: 3000\n POST_NMS_TOPK_TRAIN: 3000\n ROI_HEADS:\n NUM_CLASSES: 1\n BATCH_SIZE_PER_IMAGE: 3000\n NAME: \"CenterROIHeads\"\n IN_FEATURES: [\"p3\", \"p4\", \"p5\"]\n ROI_MASK_HEAD:\n NAME: \"SpatialAttentionMaskHead\"\n ASSIGN_CRITERION: \"ratio\"\n NUM_CONV: 4\n POOLER_RESOLUTION: 14\n RETINANET:\n NUM_CLASSES: 1\n TOPK_CANDIDATES_TEST: 3000\n PIXEL_MEAN: [128, 128, 128]\n PIXEL_STD: [11.578, 11.578, 11.578]\nSOLVER:\n STEPS: (80000, 90000)\n MAX_ITER: 100000\n IMS_PER_BATCH: 16\n BASE_LR: 0.01\n CHECKPOINT_PERIOD: 2500\nDATASETS:\n TRAIN: (\"TRAIN\",) #REPLACE TRAIN WITH THE REGISTERED NAME \n TEST: (\"TEST\",) #REPLACE TRAIN WITH THE REGISTERED NAME\n PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000\nINPUT:\n MIN_SIZE_TRAIN: (440, 480, 520, 560, 580, 620)\nTEST:\n DETECTIONS_PER_IMAGE: 2000 # 1000\nDATALOADER:\n NUM_WORKERS: 12\nOUTPUT_DIR: \"\"\n" }, { "path": "model/anchor_free/mcf7_config.yaml", "content": "MODEL:\n META_ARCHITECTURE: \"GeneralizedRCNN\"\n BACKBONE:\n NAME: \"build_fcos_vovnet_fpn_backbone\"\n FREEZE_AT: 0\n WEIGHTS: \"https://www.dropbox.com/s/1mlv31coewx8trd/vovnet99_ese_detectron2.pth?dl=1\"\n VOVNET:\n CONV_BODY : \"V-99-eSE\"\n OUT_FEATURES: [\"stage3\", \"stage4\", \"stage5\"]\n FPN:\n IN_FEATURES: [\"stage3\", \"stage4\", \"stage5\"]\n PROPOSAL_GENERATOR:\n NAME: \"FCOS\" \n MASK_ON: True\n MASKIOU_ON: True\n FCOS:\n NUM_CLASSES: 1\n PRE_NMS_TOPK_TRAIN: 2000\n PRE_NMS_TOPK_TEST: 4000\n POST_NMS_TOPK_TRAIN: 15000\n POST_NMS_TOPK_TEST: 3000\n RPN:\n BATCH_SIZE_PER_IMAGE: 3000\n POST_NMS_TOPK_TEST: 3000\n POST_NMS_TOPK_TRAIN: 3000\n ROI_HEADS:\n NUM_CLASSES: 1\n BATCH_SIZE_PER_IMAGE: 3000\n NAME: \"CenterROIHeads\"\n IN_FEATURES: [\"p3\", \"p4\", \"p5\"]\n ROI_MASK_HEAD:\n NAME: \"SpatialAttentionMaskHead\"\n ASSIGN_CRITERION: \"ratio\"\n NUM_CONV: 4\n POOLER_RESOLUTION: 14\n RETINANET:\n NUM_CLASSES: 1\n TOPK_CANDIDATES_TEST: 3000\n PIXEL_MEAN: [128, 128, 128]\n PIXEL_STD: [11.578, 11.578, 11.578]\nSOLVER:\n STEPS: (8000, 9000)\n MAX_ITER: 10000\n IMS_PER_BATCH: 16\n BASE_LR: 0.01\n CHECKPOINT_PERIOD: 500\nDATASETS:\n TRAIN: (\"TRAIN\",) #REPLACE TRAIN WITH THE REGISTERED NAME \n TEST: (\"TEST\",) #REPLACE TRAIN WITH THE REGISTERED NAME\n PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000\nINPUT:\n MIN_SIZE_TRAIN: (440, 480, 520, 560, 580, 620)\nTEST:\n DETECTIONS_PER_IMAGE: 2000 # 1000\nDATALOADER:\n NUM_WORKERS: 12\nOUTPUT_DIR: \"\"" }, { "path": "model/anchor_free/shsy5y_config.yaml", "content": "MODEL:\n META_ARCHITECTURE: \"GeneralizedRCNN\"\n BACKBONE:\n NAME: \"build_fcos_vovnet_fpn_backbone\"\n FREEZE_AT: 0\n WEIGHTS: \"https://www.dropbox.com/s/1mlv31coewx8trd/vovnet99_ese_detectron2.pth?dl=1\"\n VOVNET:\n CONV_BODY : \"V-99-eSE\"\n OUT_FEATURES: [\"stage3\", \"stage4\", \"stage5\"]\n FPN:\n IN_FEATURES: [\"stage3\", \"stage4\", \"stage5\"]\n PROPOSAL_GENERATOR:\n NAME: \"FCOS\" \n MASK_ON: True\n MASKIOU_ON: True\n FCOS:\n NUM_CLASSES: 1\n PRE_NMS_TOPK_TRAIN: 2000\n PRE_NMS_TOPK_TEST: 4000\n POST_NMS_TOPK_TRAIN: 15000\n POST_NMS_TOPK_TEST: 3000\n RPN:\n BATCH_SIZE_PER_IMAGE: 3000\n POST_NMS_TOPK_TEST: 3000\n POST_NMS_TOPK_TRAIN: 3000\n ROI_HEADS:\n NUM_CLASSES: 1\n BATCH_SIZE_PER_IMAGE: 3000\n NAME: \"CenterROIHeads\"\n IN_FEATURES: [\"p3\", \"p4\", \"p5\"]\n ROI_MASK_HEAD:\n NAME: \"SpatialAttentionMaskHead\"\n ASSIGN_CRITERION: \"ratio\"\n NUM_CONV: 4\n POOLER_RESOLUTION: 14\n RETINANET:\n NUM_CLASSES: 1\n TOPK_CANDIDATES_TEST: 3000\n PIXEL_MEAN: [128, 128, 128]\n PIXEL_STD: [11.578, 11.578, 11.578]\nSOLVER:\n STEPS: (18000, 19000)\n MAX_ITER: 20000\n IMS_PER_BATCH: 16\n BASE_LR: 0.01 \n CHECKPOINT_PERIOD: 250\n WARMUP_FACTOR: 0.00005\n WARMUP_ITERS: 5000\nDATASETS:\n TRAIN: (\"TRAIN\",) #REPLACE TRAIN WITH THE REGISTERED NAME \n TEST: (\"TEST\",) #REPLACE TRAIN WITH THE REGISTERED NAME\n PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000\nINPUT:\n MIN_SIZE_TRAIN: (440, 480, 520, 560, 580, 620)\nTEST:\n DETECTIONS_PER_IMAGE: 2000 # 1000\nDATALOADER:\n NUM_WORKERS: 12\nOUTPUT_DIR: \"\"" }, { "path": "model/anchor_free/skbr3_config.yaml", "content": "MODEL:\n META_ARCHITECTURE: \"GeneralizedRCNN\"\n BACKBONE:\n NAME: \"build_fcos_vovnet_fpn_backbone\"\n FREEZE_AT: 0\n WEIGHTS: \"https://www.dropbox.com/s/1mlv31coewx8trd/vovnet99_ese_detectron2.pth?dl=1\"\n VOVNET:\n CONV_BODY : \"V-99-eSE\"\n OUT_FEATURES: [\"stage3\", \"stage4\", \"stage5\"]\n FPN:\n IN_FEATURES: [\"stage3\", \"stage4\", \"stage5\"]\n PROPOSAL_GENERATOR:\n NAME: \"FCOS\" \n MASK_ON: True\n MASKIOU_ON: True\n FCOS:\n NUM_CLASSES: 1\n PRE_NMS_TOPK_TRAIN: 2000\n PRE_NMS_TOPK_TEST: 4000\n POST_NMS_TOPK_TRAIN: 15000\n POST_NMS_TOPK_TEST: 3000\n RPN:\n BATCH_SIZE_PER_IMAGE: 3000\n POST_NMS_TOPK_TEST: 3000\n POST_NMS_TOPK_TRAIN: 3000\n ROI_HEADS:\n NUM_CLASSES: 1\n BATCH_SIZE_PER_IMAGE: 3000\n NAME: \"CenterROIHeads\"\n IN_FEATURES: [\"p3\", \"p4\", \"p5\"]\n ROI_MASK_HEAD:\n NAME: \"SpatialAttentionMaskHead\"\n ASSIGN_CRITERION: \"ratio\"\n NUM_CONV: 4\n POOLER_RESOLUTION: 14\n RETINANET:\n NUM_CLASSES: 1\n TOPK_CANDIDATES_TEST: 3000\n PIXEL_MEAN: [128, 128, 128]\n PIXEL_STD: [11.578, 11.578, 11.578]\nSOLVER:\n STEPS: (19000, 19500)\n MAX_ITER: 20000\n IMS_PER_BATCH: 16\n BASE_LR: 0.01\n CHECKPOINT_PERIOD: 500\nDATASETS:\n TRAIN: (\"TRAIN\",) #REPLACE TRAIN WITH THE REGISTERED NAME \n TEST: (\"TEST\",) #REPLACE TRAIN WITH THE REGISTERED NAME\n PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000\nINPUT:\n MIN_SIZE_TRAIN: (440, 480, 520, 560, 580, 620)\nTEST:\n DETECTIONS_PER_IMAGE: 2000 # 1000\nDATALOADER:\n NUM_WORKERS: 12\nOUTPUT_DIR: \"\"" }, { "path": "model/anchor_free/skov3_config.yaml", "content": "MODEL:\n META_ARCHITECTURE: \"GeneralizedRCNN\"\n BACKBONE:\n NAME: \"build_fcos_vovnet_fpn_backbone\"\n FREEZE_AT: 0\n WEIGHTS: \"https://www.dropbox.com/s/1mlv31coewx8trd/vovnet99_ese_detectron2.pth?dl=1\"\n VOVNET:\n CONV_BODY : \"V-99-eSE\"\n OUT_FEATURES: [\"stage3\", \"stage4\", \"stage5\"]\n FPN:\n IN_FEATURES: [\"stage3\", \"stage4\", \"stage5\"]\n PROPOSAL_GENERATOR:\n NAME: \"FCOS\" \n MASK_ON: True\n MASKIOU_ON: True\n FCOS:\n NUM_CLASSES: 1\n PRE_NMS_TOPK_TRAIN: 2000\n PRE_NMS_TOPK_TEST: 4000\n POST_NMS_TOPK_TRAIN: 15000\n POST_NMS_TOPK_TEST: 3000\n RPN:\n BATCH_SIZE_PER_IMAGE: 3000\n POST_NMS_TOPK_TEST: 3000\n POST_NMS_TOPK_TRAIN: 3000\n ROI_HEADS:\n NUM_CLASSES: 1\n BATCH_SIZE_PER_IMAGE: 3000\n NAME: \"CenterROIHeads\"\n IN_FEATURES: [\"p3\", \"p4\", \"p5\"]\n ROI_MASK_HEAD:\n NAME: \"SpatialAttentionMaskHead\"\n ASSIGN_CRITERION: \"ratio\"\n NUM_CONV: 4\n POOLER_RESOLUTION: 14\n RETINANET:\n NUM_CLASSES: 1\n TOPK_CANDIDATES_TEST: 3000\n PIXEL_MEAN: [128, 128, 128]\n PIXEL_STD: [11.578, 11.578, 11.578]\nSOLVER:\n STEPS: (8000, 9000)\n MAX_ITER: 10000\n IMS_PER_BATCH: 16\n BASE_LR: 0.01\n CHECKPOINT_PERIOD: 500\nDATASETS:\n TRAIN: (\"TRAIN\",) #REPLACE TRAIN WITH THE REGISTERED NAME \n TEST: (\"TEST\",) #REPLACE TRAIN WITH THE REGISTERED NAME\n PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000\nINPUT:\n MIN_SIZE_TRAIN: (440, 480, 520, 560, 580, 620)\nTEST:\n DETECTIONS_PER_IMAGE: 2000 # 1000\nDATALOADER:\n NUM_WORKERS: 12\nOUTPUT_DIR: \"\"" } ]