[
{
"path": ".gitignore",
"content": "# Byte-compiled / optimized / DLL files\n__pycache__/\n*.py[cod]\n*$py.class\n\n# C extensions\n*.so\n\n# Distribution / packaging\n.Python\nbuild/\ndevelop-eggs/\ndist/\ndownloads/\neggs/\n.eggs/\nlib/\nlib64/\nparts/\nsdist/\nvar/\nwheels/\nshare/python-wheels/\n*.egg-info/\n.installed.cfg\n*.egg\nMANIFEST\n\n# PyInstaller\n# Usually these files are written by a python script from a template\n# before PyInstaller builds the exe, so as to inject date/other infos into it.\n*.manifest\n*.spec\n\n# Installer logs\npip-log.txt\npip-delete-this-directory.txt\n\n# Unit test / coverage reports\nhtmlcov/\n.tox/\n.nox/\n.coverage\n.coverage.*\n.cache\nnosetests.xml\ncoverage.xml\n*.cover\n*.py,cover\n.hypothesis/\n.pytest_cache/\ncover/\n\n# Translations\n*.mo\n*.pot\n\n# Django stuff:\n*.log\nlocal_settings.py\ndb.sqlite3\ndb.sqlite3-journal\n\n# Flask stuff:\ninstance/\n.webassets-cache\n\n# Scrapy stuff:\n.scrapy\n\n# Sphinx documentation\ndocs/_build/\n\n# PyBuilder\n.pybuilder/\ntarget/\n\n# Jupyter Notebook\n.ipynb_checkpoints\n\n# IPython\nprofile_default/\nipython_config.py\n\n# pyenv\n# For a library or package, you might want to ignore these files since the code is\n# intended to run in multiple environments; otherwise, check them in:\n# .python-version\n\n# pipenv\n# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.\n# However, in case of collaboration, if having platform-specific dependencies or dependencies\n# having no cross-platform support, pipenv may install dependencies that don't work, or not\n# install all needed dependencies.\n#Pipfile.lock\n\n# poetry\n# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.\n# This is especially recommended for binary packages to ensure reproducibility, and is more\n# commonly ignored for libraries.\n# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control\n#poetry.lock\n\n# PEP 582; used by e.g. github.com/David-OConnor/pyflow\n__pypackages__/\n\n# Celery stuff\ncelerybeat-schedule\ncelerybeat.pid\n\n# SageMath parsed files\n*.sage.py\n\n# Environments\n.env\n.venv\nenv/\nvenv/\nENV/\nenv.bak/\nvenv.bak/\n\n# Spyder project settings\n.spyderproject\n.spyproject\n\n# Rope project settings\n.ropeproject\n\n# mkdocs documentation\n/site\n\n# mypy\n.mypy_cache/\n.dmypy.json\ndmypy.json\n\n# Pyre type checker\n.pyre/\n\n# pytype static type analyzer\n.pytype/\n\n# Cython debug symbols\ncython_debug/\n\n# PyCharm\n# JetBrains specific template is maintainted in a separate JetBrains.gitignore that can\n# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore\n# and can be added to the global gitignore or merged into this file. For a more nuclear\n# option (not recommended) you can uncomment the following to ignore the entire idea folder.\n#.idea/\n\n.vscode/*\n./utils/.ipynb_checkpoints/*\nInputArray.desktop\n"
},
{
"path": "README.md",
"content": "RELLIS-3D: A Multi-modal Dataset for Off-Road Robotics \n\n
\n\nPeng Jiang1 , Philip Osteen2 , Maggie Wigness2 and Srikanth Saripalli1 Texas A&M University; 2. CCDC Army Research Laboratory [Website] [Paper] [Github] \n
\n\n## Updates\n* 11/26/2020 v1.0 release\n* 02/25/2021 improve camera and lidar calibration parameter\n* 03/04/2021 update ROS bag with new tf (v1.1 release)\n* 06/14/2021 fix missing labels of point cloud and fix wrong poses\n* 01/24/2022 add Velodyne point clouds in kitti format and labels transfered from Ouster\n## Overview\nSemantic scene understanding is crucial for robust and safe autonomous navigation, particularly so in off-road environments. Recent deep learning advances for 3D semantic segmentation rely heavily on large sets of training data; however, existing autonomy datasets represent urban environments or lack multimodal off-road data. We fill this gap with RELLIS-3D, a multimodal dataset collected in an off-road environment containing annotations for **13,556 LiDAR scans** and **6,235 images**. The data was collected on the Rellis Campus of Texas A\\&M University and presents challenges to existing algorithms related to class imbalance and environmental topography. Additionally, we evaluate the current state of the art deep learning semantic segmentation models on this dataset. Experimental results show that RELLIS-3D presents challenges for algorithms designed for segmentation in urban environments. Except for the annotated data, the dataset also provides full-stack sensor data in ROS bag format, including **RGB camera images**, **LiDAR point clouds**, **a pair of stereo images**, **high-precision GPS measurement**, and **IMU data**. This novel dataset provides the resources needed by researchers to develop more advanced algorithms and investigate new research directions to enhance autonomous navigation in off-road environments.\n\n\n\n### Recording Platform\n* [Clearpath Robobtics Warthog](https://clearpathrobotics.com/warthog-unmanned-ground-vehicle-robot/)\n\n### Sensor Setup\n* 64 channels Lidar: [Ouster OS1](https://ouster.com/products/os1-lidar-sensor)\n* 32 Channels Lidar: [Velodyne Ultra Puck](https://velodynelidar.com/vlp-32c.html)\n* 3D Stereo Camera: [Nerian Karmin2](https://nerian.com/products/karmin2-3d-stereo-camera/) + [Nerian SceneScan](https://nerian.com/products/scenescan-stereo-vision/) [(Sensor Configuration)](https://nerian.com/support/calculator/?1,10,0,6,2,1600,1200,1,1,1,800,592,1,6,2,0,61.4,0,0,1,1,0,5,0,0,0,0.66,0,1,25,1,0,1,256,0.25,256,4.0,5.0,0,1.5,1,#results)\n* RGB Camera: [Basler acA1920-50gc](https://www.baslerweb.com/en/products/cameras/area-scan-cameras/ace/aca1920-50gc/) + [Edmund Optics 16mm/F1.8 86-571](https://www.edmundoptics.com/p/16mm-focal-length-hp-series-fixed-focal-length-lens/28990/)\n* Inertial Navigation System (GPS/IMU): [Vectornav VN-300 Dual Antenna GNSS/INS](https://www.vectornav.com/products/vn-300)\n\n\n\n\n## Folder structure\n\n\nRellis-3D\n├── pt_test.lst\n├── pt_val.lst\n├── pt_train.lst\n├── pt_test.lst\n├── pt_train.lst\n├── pt_val.lst\n├── 00000\n ├── os1_cloud_node_kitti_bin/ -- directory containing \".bin\" files with Ouster 64-Channels point clouds. \n ├── os1_cloud_node_semantickitti_label_id/ -- containing, \".label\" files for Ouster Lidar point cloud with manually labelled semantics label\n ├── vel_cloud_node_kitti_bin/ -- directory containing \".bin\" files with Velodyne 32-Channels point clouds. \n ├── vel_cloud_node_semantickitti_label_id/ -- containing, \".label\" files for Velodyne Lidar point cloud transfered from Ouster point cloud.\n ├── pylon_camera_node/ -- directory containing \".png\" files from the color camera. \n ├── pylon_camera_node_label_color -- color image lable\n ├── pylon_camera_node_label_id -- id image lable\n ├── calib.txt -- calibration of velodyne vs. camera. needed for projection of point cloud into camera. \n └── poses.txt -- file containing the poses of every scan.\n \n\n## Download Link on BaiDu Pan:\n链接: https://pan.baidu.com/s/1akqSm7mpIMyUJhn_qwg3-w?pwd=4gk3 提取码: 4gk3 复制这段内容后打开百度网盘手机App，操作更方便哦\n\n## Annotated Data:\n### Ontology:\nWith the goal of providing multi-modal data to enhance autonomous off-road navigation, we defined an ontology of object and terrain classes, which largely derives from [the RUGD dataset](http://rugd.vision/) but also includes unique terrain and object classes not present in RUGD. Specifically, sequences from this dataset includes classes such as mud, man-made barriers, and rubble piles. Additionally, this dataset provides a finer-grained class structure for water sources, i.e., puddle and deep water, as these two classes present different traversability scenarios for most robotic platforms. Overall, 20 classes (including void class) are present in the data.\n\n**Ontology Definition** ([Download 18KB](https://drive.google.com/file/d/1K8Zf0ju_xI5lnx3NTDLJpVTs59wmGPI6/view?usp=sharing))\n\n### Images Statics:\n\n\n\n Note: Due to the limitation of Google Drive, the downloads might be constrained. Please wait for 24h and try again. If you still can't access the file, please email maskjp@tamu.edu with the title \"RELLIS-3D Access Request\". .\n\n\n### Image Download: \n\n**Image with Annotation Examples** ([Download 3MB](https://drive.google.com/file/d/1wIig-LCie571DnK72p2zNAYYWeclEz1D/view?usp=sharing))\n\n**Full Images** ([Download 11GB](https://drive.google.com/file/d/1F3Leu0H_m6aPVpZITragfreO_SGtL2yV/view?usp=sharing))\n\n**Full Image Annotations Color Format** ([Download 119MB](https://drive.google.com/file/d/1HJl8Fi5nAjOr41DPUFmkeKWtDXhCZDke/view?usp=sharing))\n\n**Full Image Annotations ID Format** ([Download 94MB](https://drive.google.com/file/d/16URBUQn_VOGvUqfms-0I8HHKMtjPHsu5/view?usp=sharing))\n\n**Image Split File** ([44KB](https://drive.google.com/file/d/1zHmnVaItcYJAWat3Yti1W_5Nfux194WQ/view?usp=sharing))\n\n### LiDAR Scans Statics:\n\n\n\n### LiDAR Download: \n\n**Ouster LiDAR with Annotation Examples** ([Download 24MB](https://drive.google.com/file/d/1QikPnpmxneyCuwefr6m50fBOSB2ny4LC/view?usp=sharing)) \n\n**Ouster LiDAR with Color Annotation PLY Format** ([Download 26GB](https://drive.google.com/file/d/1BZWrPOeLhbVItdN0xhzolfsABr6ymsRr/view?usp=sharing))\n\nThe header of the PLY file is described as followed:\n```\nelement vertex\nproperty float x\nproperty float y\nproperty float z\nproperty float intensity\nproperty uint t\nproperty ushort reflectivity\nproperty uchar ring\nproperty ushort noise\nproperty uint range\nproperty uchar label\nproperty uchar red\nproperty uchar green\nproperty uchar blue\n```\nTo visualize the color of the ply file, please use [CloudCompare](https://www.danielgm.net/cc/) or [Open3D](http://www.open3d.org/). Meshlab has problem to visualize the color.\n\n\n**Ouster LiDAR SemanticKITTI Format** ([Download 14GB](https://drive.google.com/file/d/1lDSVRf_kZrD0zHHMsKJ0V1GN9QATR4wH/view?usp=sharing))\n\nTo visualize the datasets using the SemanticKITTI tools, please use this fork: [https://github.com/unmannedlab/point_labeler](https://github.com/unmannedlab/point_labeler)\n\n**Ouster LiDAR Annotation SemanticKITTI Format** ([Download 174MB](https://drive.google.com/file/d/12bsblHXtob60KrjV7lGXUQTdC5PhV8Er/view?usp=sharing))\n\n**Ouster LiDAR Scan Poses files** ([Download 174MB](https://drive.google.com/file/d/1V3PT_NJhA41N7TBLp5AbW31d0ztQDQOX/view?usp=sharing))\n\n**Ouster LiDAR Split File** ([75KB](https://drive.google.com/file/d/1raQJPySyqDaHpc53KPnJVl3Bln6HlcVS/view?usp=sharing)) \n\n\n**Velodyne LiDAR SemanticKITTI Format** ([Download 5.58GB](https://drive.google.com/file/d/1PiQgPQtJJZIpXumuHSig5Y6kxhAzz1cz/view?usp=sharing))\n\n\n**Velodyne LiDAR Annotation SemanticKITTI Format** ([Download 143.6MB](https://drive.google.com/file/d/1n-9FkpiH4QUP7n0PnQBp-s7nzbSzmxp8/view?usp=sharing))\n\n### Calibration Download: \n**Camera Instrinsic** ([Download 2KB](https://drive.google.com/file/d/1NAigZTJYocRSOTfgFBddZYnDsI_CSpwK/view?usp=sharing))\n\n**Basler Camera to Ouster LiDAR** ([Download 3KB](https://drive.google.com/file/d/19EOqWS9fDUFp4nsBrMCa69xs9LgIlS2e/view?usp=sharing))\n\n**Velodyne LiDAR to Ouster LiDAR** ([Download 3KB](https://drive.google.com/file/d/1T6yPwcdzJoU-ifFRelLtDLPuPQswIQwf/view?usp=sharing))\n\n**Stereo Calibration** ([Download 3KB](https://drive.google.com/file/d/1cP5-l_nYt3kZ4hZhEAHEdpt2fzToar0R/view?usp=sharing))\n\n\n**Calibration Raw Data** ([Download 774MB](https://drive.google.com/drive/folders/1VAb-98lh6HWEe_EKLhUC1Xle0jkpp2Fl?usp=sharing\n))\n\n## Benchmarks\n\n### Image Semantic Segmenation \nmodels | sky | grass |tr ee | bush | concrete | mud | person | puddle | rubble | barrier | log | fence | vehicle | object | pole | water | asphalt | building | mean\n-------| ----| ------|------|------|----------|-----| -------| -------|--------|---------|-----|-------| --------| -------|------|-------|---------|----------| ----\n[HRNet+OCR](https://github.com/HRNet/HRNet-Semantic-Segmentation/tree/HRNet-OCR) | 96.94 | 90.20 | 80.53 | 76.76 | 84.22 | 43.29 | 89.48 | 73.94 | 62.03 | 54.86 | 0.00 | 39.52 | 41.54 | 46.44 | 9.51 | 0.72 | 33.25 | 4.60 | 48.83\n[GSCNN](https://github.com/nv-tlabs/GSCNN) | 97.02 | 84.95 | 78.52 | 70.33 | 83.82 | 45.52 | 90.31 | 71.49 | 66.03 | 55.12 | 2.92 | 41.86 | 46.51 | 54.64 | 6.90 | 0.94 | 44.18 | 11.47 | 50.13\n\n[](https://www.youtube.com/watch?v=vr3g6lCTKRM)\n\n### LiDAR Semantic Segmenation\nmodels | sky | grass |tr ee | bush | concrete | mud | person | puddle | rubble | barrier | log | fence | vehicle | object | pole | water | asphalt | building | mean\n-------| ----| ------|------|------|----------|-----| -------| -------|--------|---------|-----|-------| --------| -------|------|-------|---------|----------| ----\n[SalsaNext](https://github.com/Halmstad-University/SalsaNext) | - | 64.74 | 79.04 | 72.90 | 75.27 | 9.58 | 83.17 | 23.20 | 5.01 | 75.89 | 18.76 | 16.13| 23.12 | - | 56.26 | 0.00 | - | - | 40.20 \n[KPConv](https://github.com/HuguesTHOMAS/KPConv) | - | 56.41 | 49.25 | 58.45 | 33.91 | 0.00 | 81.20 | 0.00 | 0.00 | 0.00 | 0.00 | 0.40 | 0.00 | - | 0.00 | 0.00 | - | - | 18.64\n\n[](https://www.youtube.com/watch?v=wkm8UiVNGao)\n\n### Benchmark Reproduction\n\nTo reproduce the results, please refer to [here](./benchmarks/README.md)\n\n## ROS Bag Raw Data\n\nData included in raw ROS bagfiles:\n\nTopic Name | Message Tpye | Message Descriptison\n------------ | ------------- | ---------------------------------\n/img_node/intensity_image | sensor_msgs/Image | Intensity image generated by ouster Lidar\n/img_node/noise_image | sensor_msgs/Image | Noise image generated by ouster Lidar\n/img_node/range_image | sensor_msgs/Image | Range image generated by ouster Lidar\n/imu/data | sensor_msgs/Imu | Filtered imu data from embeded imu of Warthog\n/imu/data_raw | sensor_msgs/Imu | Raw imu data from embeded imu of Warthog\n/imu/mag | sensor_msgs/MagneticField | Raw magnetic field data from embeded imu of Warthog\n/left_drive/status/battery_current | std_msgs/Float64 | \n/left_drive/status/battery_voltage | std_msgs/Float64 |\n/mcu/status | warthog_msgs/Status |\n/nerian/left/camera_info | sensor_msgs/CameraInfo | \n/nerian/left/image_raw | sensor_msgs/Image | Left image from Nerian Karmin2 \n/nerian/right/camera_info | sensor_msgs/CameraInfo | \n/nerian/right/image_raw | sensor_msgs/Image | Right image from Nerian Karmin2 \n/odometry/filtered | nav_msgs/Odometry | A filtered local-ization estimate based on wheel odometry (en-coders) and integrated IMU from Warthog\n/os1_cloud_node/imu | sensor_msgs/Imu | Raw imu data from embeded imu of Ouster Lidar\n/os1_cloud_node/points | sensor_msgs/PointCloud2 | Point cloud data from Ouster Lidar\n/os1_node/imu_packets | ouster_ros/PacketMsg | Raw imu data from Ouster Lidar\n/os1_node/lidar_packets | ouster_ros/PacketMsg | Raw lidar data from Ouster Lidar\n/pylon_camera_node/camera_info | sensor_msgs/CameraInfo | \n/pylon_camera_node/image_raw | sensor_msgs/Image |\n/right_drive/status/battery_current | std_msgs/Float64 | \n/right_drive/status/battery_voltage | std_msgs/Float64 |\n/tf | tf2_msgs/TFMessage |\n/tf_static | tf2_msgs/TFMessage\n/vectornav/GPS | sensor_msgs/NavSatFix | INS data from VectorNav-VN300\n/vectornav/IMU | sensor_msgs/Imu | Imu data from VectorNav-VN300\n/vectornav/Mag | sensor_msgs/MagneticField | Raw magnetic field data from VectorNav-VN300\n/vectornav/Odom | nav_msgs/Odometry | Odometry from VectorNav-VN300\n/vectornav/Pres | sensor_msgs/FluidPressure |\n/vectornav/Temp | sensor_msgs/Temperature |\n/velodyne_points | sensor_msgs/PointCloud2 | PointCloud produced by the Velodyne Lidar\n/warthog_velocity_controller/cmd_vel | geometry_msgs/Twist |\n/warthog_velocity_controller/odom | nav_msgs/Odometry |\n### ROS Bag Download\n\nThe following are the links for the ROS Bag files. \n* Synced data (60 seconds example [2 GB](https://drive.google.com/file/d/13EHwiJtU0aAWBQn-ZJhTJwC1Yx2zDVUv/view?usp=sharing)): includes synced */os1_cloud_node/points*, */pylon_camera_node/camera_info* and */pylon_camera_node/image_raw*\n* Full-stack Merged data:(60 seconds example [4.2 GB](https://drive.google.com/file/d/1qSeOoY6xbQGjcrZycgPM8Ty37eKDjpJL/view?usp=sharing)): includes all data in above table and extrinsic calibration info data embedded in the tf tree.\n* Full-stack Split Raw data:(60 seconds example [4.3 GB](https://drive.google.com/file/d/1-TDpelP4wKTWUDTIn0dNuZIT3JkBoZ_R/view?usp=sharing)): is orignal data recorded by ```rosbag record``` command. \n\n**Sequence 00000**: Synced data: ([12GB](https://drive.google.com/file/d/1bIb-6fWbaiI9Q8Pq9paANQwXWn7GJDtl/view?usp=sharing)) Full-stack Merged data: ([23GB](https://drive.google.com/file/d/1grcYRvtAijiA0Kzu-AV_9K4k2C1Kc3Tn/view?usp=sharing)) Full-stack Split Raw data: ([29GB](https://drive.google.com/drive/folders/1IZ-Tn_kzkp82mNbOL_4sNAniunD7tsYU?usp=sharing))\n\n[](https://www.youtube.com/watch?v=Qc7IepWGKr8)\n\n**Sequence 00001**: Synced data: ([8GB](https://drive.google.com/file/d/1xNjAFE3cv6X8n046irm8Bo5QMerNbwP1/view?usp=sharing)) Full-stack Merged data: ([16GB](https://drive.google.com/file/d/1geoU45pPavnabQ0arm4ILeHSsG3cU6ti/view?usp=sharing)) Full-stack Split Raw data: ([22GB](https://drive.google.com/drive/folders/1hf-vF5zyTKcCLqIiddIGdemzKT742T1t?usp=sharing))\n\n[](https://www.youtube.com/watch?v=nO5JADjDWQ0)\n\n**Sequence 00002**: Synced data: ([14GB](https://drive.google.com/file/d/1gy0ehP9Buj-VkpfvU9Qwyz1euqXXQ_mj/view?usp=sharing)) Full-stack Merged data: ([28GB](https://drive.google.com/file/d/1h0CVg62jTXiJ91LnR6md-WrUBDxT543n/view?usp=sharing)) Full-stack Split Raw data: ([37GB](https://drive.google.com/drive/folders/1R8jP5Qo7Z6uKPoG9XUvFCStwJu6rtliu?usp=sharing))\n\n[](https://www.youtube.com/watch?v=aXaOmzjHmNE)\n\n**Sequence 00003**:Synced data: ([8GB](https://drive.google.com/file/d/1vCeZusijzyn1ZrZbg4JaHKYSc2th7GEt/view?usp=sharing)) Full-stack Merged data: ([15GB](https://drive.google.com/file/d/1glJzgnTYLIB_ar3CgHpc_MBp5AafQpy9/view?usp=sharing)) Full-stack Split Raw data: ([19GB](https://drive.google.com/drive/folders/1iP0k6dbmPdAH9kkxs6ugi6-JbrkGhm5o?usp=sharing))\n\n\n[](https://www.youtube.com/watch?v=Kjo3tGDSbtU)\n\n**Sequence 00004**:Synced data: ([7GB](https://drive.google.com/file/d/1gxODhAd8CBM5AGvsoyuqN7yGpWazzmVy/view?usp=sharing)) Full-stack Merged data: ([14GB](https://drive.google.com/file/d/1AuEjX0do3jGZhGKPszSEUNoj85YswNya/view?usp=sharing)) Full-stack Split Raw data: ([17GB](https://drive.google.com/drive/folders/1WV9pecF2beESyM7N29W-nhi-JaoKvEqc?usp=sharing))\n\n\n[](https://www.youtube.com/watch?v=lLLYTI4TCD4)\n\n### ROS Environment Installment\nThe ROS workspace includes a plaftform description package which can provide rough tf tree for running the rosbag.\n\nTo run cartographer on RELLIS-3D please refer to [here](https://github.com/unmannedlab/cartographer)\n\n\n\n\n## Full Data Download:\n[Access Link](https://drive.google.com/drive/folders/1aZ1tJ3YYcWuL3oWKnrTIC5gq46zx1bMc?usp=sharing)\n\n## Citation\n```\n@misc{jiang2020rellis3d,\n title={RELLIS-3D Dataset: Data, Benchmarks and Analysis}, \n author={Peng Jiang and Philip Osteen and Maggie Wigness and Srikanth Saripalli},\n year={2020},\n eprint={2011.12954},\n archivePrefix={arXiv},\n primaryClass={cs.CV}\n}\n```\n\n## Collaborator\nGated Shape CNN \n \n Gated-SCNN \n Gated Shape CNNs for Semantic Segmentation \n \n\n \n \n \n \n 1 NVIDIA \n \n \n \n 2 University of Waterloo \n \n \n \n 3 University of Toronto \n \n \n \n 4 Vector Institute \n \n \n
\n \n \n \n \n \n\t\tICCV, 2019 \n \n \n \n
\n \n \n \n \n \n \n \n \n \n \n \n \n
\n \n \n \n \n \n \n \n \n \n \n \n\n\t\t\t Current state-of-the-art methods for image segmentation form a dense image representation where the color, shape and texture information are all processed together inside a deep CNN. This however may not be ideal as they contain very different type of information relevant for recognition. We propose a new architecture that adds a shape stream to the classical CNN architecture. The two streams process the image in parallel, and their information gets fused in the very top layers. Key to this architecture is a new type of gates that connect the intermediate layers of the two streams. Specifically, we use the higher-level activations in the classical stream to gate the lower-level activations in the shape stream, effectively removing noise and helping the shape stream to only focus on processing the relevant boundary-related information. This enables us to use a very shallow architecture for the shape stream that operates on the image-level resolution. Our experiments show that this leads to a highly effective architecture that produces sharper predictions around object boundaries and significantly boosts performance on thinner and smaller objects. Our method achieves state-of-the-art performance on the Cityscapes benchmark, in terms of both mask (mIoU) and boundary (F-score) quality, improving by 2% and 4% over strong baselines.\n
\n \n \n \n \n \n\n \n News \n \n\t\t[August 2019] Code released on GitHub \n [July 2019] Paper accepted at ICCV 2019! \n [July 2019] Paper released on arXiv \n \n \n
\n \n Paper \n Towaki Takikawa* , David Acuna* , Varun Jampani , Sanja Fidler(* denotes equal contribution) \n
\n GSCNN in a nutshell \n \n \n \n \n
\n\n Results \n \n \n \n \n \n \n \n \n \n \n Qualitative Segmentation Results\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n Qualitative Semantic Boundary Results\n \n \n \n \n\n \n \n \n \n \n \n \n \n \n \n \n Quantitative Results\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n Evaluation at different distances, measured by crop factor.\n \n \n \n \n\n
\n \n\t January 2003\n \"\"\"\n\n\tseg = seg.astype(np.bool)\n\tseg[seg>0] = 1\n\n\tassert np.atleast_3d(seg).shape[2] == 1\n\n\twidth = seg.shape[1] if width is None else width\n\theight = seg.shape[0] if height is None else height\n\n\th,w = seg.shape[:2]\n\n\tar1 = float(width) / float(height)\n\tar2 = float(w) / float(h)\n\n\tassert not (width>w | height>h | abs(ar1-ar2)>0.01),\\\n\t\t\t'Can''t convert %dx%d seg to %dx%d bmap.'%(w,h,width,height)\n\n\te = np.zeros_like(seg)\n\ts = np.zeros_like(seg)\n\tse = np.zeros_like(seg)\n\n\te[:,:-1] = seg[:,1:]\n\ts[:-1,:] = seg[1:,:]\n\tse[:-1,:-1] = seg[1:,1:]\n\n\tb = seg^e | seg^s | seg^se\n\tb[-1,:] = seg[-1,:]^e[-1,:]\n\tb[:,-1] = seg[:,-1]^s[:,-1]\n\tb[-1,-1] = 0\n\n\tif w == width and h == height:\n\t\tbmap = b\n\telse:\n\t\tbmap = np.zeros((height,width))\n\t\tfor x in range(w):\n\t\t\tfor y in range(h):\n\t\t\t\tif b[y,x]:\n\t\t\t\t\tj = 1+floor((y-1)+height / h)\n\t\t\t\t\ti = 1+floor((x-1)+width / h)\n\t\t\t\t\tbmap[j,i] = 1;\n\n\treturn bmap\n"
},
{
"path": "benchmarks/GSCNN-master/utils/image_page.py",
"content": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).\n\"\"\"\n\nimport glob\nimport os\n\nclass ImagePage(object):\n '''\n This creates an HTML page of embedded images, useful for showing evaluation results.\n\n Usage:\n ip = ImagePage(html_fn)\n\n # Add a table with N images ...\n ip.add_table((img, descr), (img, descr), ...)\n\n # Generate html page\n ip.write_page()\n '''\n def __init__(self, experiment_name, html_filename):\n self.experiment_name = experiment_name\n self.html_filename = html_filename\n self.outfile = open(self.html_filename, 'w')\n self.items = []\n\n def _print_header(self):\n header = '''\n\n \n Experiment = {} \n \n '''.format(self.experiment_name)\n self.outfile.write(header)\n\n def _print_footer(self):\n self.outfile.write(''' \n''')\n\n def _print_table_header(self, table_name):\n table_hdr = ''' {} \n \n '''.format(table_name)\n self.outfile.write(table_hdr)\n\n def _print_table_footer(self):\n table_ftr = ''' \n
'''\n self.outfile.write(table_ftr)\n\n def _print_table_guts(self, img_fn, descr):\n table = ''' \n \n \n
{descr}
\n \n '''.format(img_fn=img_fn, descr=descr)\n self.outfile.write(table)\n\n def add_table(self, img_label_pairs):\n self.items.append(img_label_pairs)\n\n def _write_table(self, table):\n img, _descr = table[0]\n self._print_table_header(os.path.basename(img))\n for img, descr in table:\n self._print_table_guts(img, descr)\n self._print_table_footer()\n\n def write_page(self):\n self._print_header()\n\n for table in self.items:\n self._write_table(table)\n\n self._print_footer()\n\n\ndef main():\n images = glob.glob('dump_imgs_train/*.png')\n images = [i for i in images if 'mask' not in i]\n\n ip = ImagePage('test page', 'dd.html')\n for img in images:\n basename = os.path.splitext(img)[0]\n mask_img = basename + '_mask.png'\n ip.add_table(((img, 'image'), (mask_img, 'mask')))\n ip.write_page()\n"
},
{
"path": "benchmarks/GSCNN-master/utils/misc.py",
"content": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).\n\"\"\"\n\nimport sys\nimport re\nimport os\nimport shutil\nimport torch\nfrom datetime import datetime\nimport logging\nfrom subprocess import call\nimport shlex\nfrom tensorboardX import SummaryWriter\nimport numpy as np\nfrom utils.image_page import ImagePage\nimport torchvision.transforms as standard_transforms\nimport torchvision.utils as vutils\nfrom PIL import Image\n\n# Create unique output dir name based on non-default command line args\ndef make_exp_name(args, parser):\n exp_name = '{}-{}'.format(args.dataset[:3], args.arch[:])\n dict_args = vars(args)\n\n # sort so that we get a consistent directory name\n argnames = sorted(dict_args)\n\n # build experiment name with non-default args\n for argname in argnames:\n if dict_args[argname] != parser.get_default(argname):\n if argname == 'exp' or argname == 'arch' or argname == 'prev_best_filepath':\n continue\n if argname == 'snapshot':\n arg_str = '-PT'\n elif argname == 'nosave':\n arg_str = ''\n argname=''\n elif argname == 'freeze_trunk':\n argname = ''\n arg_str = '-fr'\n elif argname == 'syncbn':\n argname = ''\n arg_str = '-sbn'\n elif argname == 'relaxedloss':\n argname = ''\n arg_str = 're-loss'\n elif isinstance(dict_args[argname], bool):\n arg_str = 'T' if dict_args[argname] else 'F'\n else:\n arg_str = str(dict_args[argname])[:6]\n exp_name += '-{}_{}'.format(str(argname), arg_str)\n # clean special chars out\n exp_name = re.sub(r'[^A-Za-z0-9_\\-]+', '', exp_name)\n exp_name = 'testing'\n return exp_name\n\n\ndef save_log(prefix, output_dir, date_str):\n fmt = '%(asctime)s.%(msecs)03d %(message)s'\n date_fmt = '%m-%d %H:%M:%S'\n filename = os.path.join(output_dir, prefix + '_' + date_str + '.log')\n logging.basicConfig(level=logging.INFO, format=fmt, datefmt=date_fmt,\n filename=filename, filemode='w')\n console = logging.StreamHandler()\n console.setLevel(logging.INFO)\n formatter = logging.Formatter(fmt=fmt, datefmt=date_fmt)\n console.setFormatter(formatter)\n logging.getLogger('').addHandler(console)\n #logging.basicConfig(level=logging.INFO, format='%(asctime)s.%(msecs)03d %(levelname)s:\\t%(message)s', datefmt='%Y-%m-%d %H:%M:%S')\n\n\ndef save_code(exp_path, date_str):\n code_root = '.' # FIXME!\n zip_outfile = os.path.join(exp_path, 'code_{}.tgz'.format(date_str))\n print('Saving code to {}'.format(zip_outfile))\n cmd = 'tar -czvf {zip_outfile} --exclude=\\'*.pyc\\' --exclude=\\'*.png\\' ' +\\\n '--exclude=\\'*tfevents*\\' {root}/train.py ' + \\\n ' {root}/utils {root}/datasets {root}/models'\n cmd = cmd.format(zip_outfile=zip_outfile, root=code_root)\n call(shlex.split(cmd), stdout=open(os.devnull, 'wb'))\n\n\ndef prep_experiment(args, parser):\n '''\n Make output directories, setup logging, Tensorboard, snapshot code.\n '''\n ckpt_path = args.ckpt\n tb_path = args.tb_path\n exp_name = make_exp_name(args, parser)\n args.exp_path = os.path.join(ckpt_path, args.exp, exp_name)\n args.tb_exp_path = os.path.join(tb_path, args.exp, exp_name)\n args.ngpu = torch.cuda.device_count()\n args.date_str = str(datetime.now().strftime('%Y_%m_%d_%H_%M_%S'))\n args.best_record = {'epoch': -1, 'iter': 0, 'val_loss': 1e10, 'acc': 0,\n 'acc_cls': 0, 'mean_iu': 0, 'fwavacc': 0}\n args.last_record = {}\n os.makedirs(args.exp_path, exist_ok=True)\n os.makedirs(args.tb_exp_path, exist_ok=True)\n save_log('log', args.exp_path, args.date_str)\n #save_code(args.exp_path, args.date_str)\n open(os.path.join(args.exp_path, args.date_str + '.txt'), 'w').write(\n str(args) + '\\n\\n')\n\n writer = SummaryWriter(logdir=args.tb_exp_path, comment=args.tb_tag)\n return writer\n\nclass AverageMeter(object):\n\n def __init__(self):\n self.reset()\n\n def reset(self):\n self.val = 0\n self.avg = 0\n self.sum = 0\n self.count = 0\n\n def update(self, val, n=1):\n self.val = val\n self.sum += val * n\n self.count += n\n self.avg = self.sum / self.count\n\n\n\ndef evaluate_eval(args, net, optimizer, val_loss, mf_score, hist, dump_images, heatmap_images, writer, epoch=0, dataset=None, ):\n '''\n Modified IOU mechanism for on-the-fly IOU calculations ( prevents memory overflow for\n large dataset) Only applies to eval/eval.py\n '''\n # axis 0: gt, axis 1: prediction\n acc = np.diag(hist).sum() / hist.sum()\n acc_cls = np.diag(hist) / hist.sum(axis=1)\n acc_cls = np.nanmean(acc_cls)\n iu = np.diag(hist) / (hist.sum(axis=1) + hist.sum(axis=0) - np.diag(hist))\n\n print_evaluate_results(hist, iu, writer, epoch, dataset)\n freq = hist.sum(axis=1) / hist.sum()\n mean_iu = np.nanmean(iu)\n logging.info('mean {}'.format(mean_iu))\n fwavacc = (freq[freq > 0] * iu[freq > 0]).sum()\n #return acc, acc_cls, mean_iu, fwavacc\n\n # update latest snapshot\n if 'mean_iu' in args.last_record:\n last_snapshot = 'last_epoch_{}_mean-iu_{:.5f}.pth'.format(\n args.last_record['epoch'], args.last_record['mean_iu'])\n last_snapshot = os.path.join(args.exp_path, last_snapshot)\n try:\n os.remove(last_snapshot)\n except OSError:\n pass\n last_snapshot = 'last_epoch_{}_mean-iu_{:.5f}.pth'.format(epoch, mean_iu)\n last_snapshot = os.path.join(args.exp_path, last_snapshot)\n args.last_record['mean_iu'] = mean_iu\n args.last_record['epoch'] = epoch\n \n torch.cuda.synchronize()\n \n torch.save({\n 'state_dict': net.state_dict(),\n 'optimizer': optimizer.state_dict(),\n 'epoch': epoch,\n 'mean_iu': mean_iu,\n 'command': ' '.join(sys.argv[1:])\n }, last_snapshot)\n\n # update best snapshot\n if mean_iu > args.best_record['mean_iu'] :\n # remove old best snapshot\n if args.best_record['epoch'] != -1:\n best_snapshot = 'best_epoch_{}_mean-iu_{:.5f}.pth'.format(\n args.best_record['epoch'], args.best_record['mean_iu'])\n best_snapshot = os.path.join(args.exp_path, best_snapshot)\n assert os.path.exists(best_snapshot), \\\n 'cant find old snapshot {}'.format(best_snapshot)\n os.remove(best_snapshot)\n\n \n # save new best\n args.best_record['val_loss'] = val_loss.avg\n args.best_record['mask_f1_score'] = mf_score.avg\n args.best_record['epoch'] = epoch\n args.best_record['acc'] = acc\n args.best_record['acc_cls'] = acc_cls\n args.best_record['mean_iu'] = mean_iu\n args.best_record['fwavacc'] = fwavacc\n\n best_snapshot = 'best_epoch_{}_mean-iu_{:.5f}.pth'.format(\n args.best_record['epoch'], args.best_record['mean_iu'])\n best_snapshot = os.path.join(args.exp_path, best_snapshot)\n shutil.copyfile(last_snapshot, best_snapshot)\n \n \n to_save_dir = os.path.join(args.exp_path, 'best_images')\n os.makedirs(to_save_dir, exist_ok=True)\n ip = ImagePage(epoch, '{}/index.html'.format(to_save_dir))\n\n val_visual = []\n \n idx = 0\n \n visualize = standard_transforms.Compose([\n standard_transforms.Scale(384),\n standard_transforms.ToTensor()\n ])\n for bs_idx, bs_data in enumerate(dump_images):\n for local_idx, data in enumerate(zip(bs_data[0], bs_data[1],bs_data[2])):\n gt_pil = args.dataset_cls.colorize_mask(data[0].cpu().numpy())\n pred = data[1].cpu().numpy()\n predictions_pil = args.dataset_cls.colorize_mask(pred)\n img_name = data[2]\n \n prediction_fn = '{}_prediction.png'.format(img_name)\n predictions_pil.save(os.path.join(to_save_dir, prediction_fn))\n gt_fn = '{}_gt.png'.format(img_name)\n gt_pil.save(os.path.join(to_save_dir, gt_fn))\n ip.add_table([(gt_fn, 'gt'), (prediction_fn, 'prediction')])\n val_visual.extend([visualize(gt_pil.convert('RGB')),\n visualize(predictions_pil.convert('RGB'))])\n idx = idx+1\n if idx >= 9:\n ip.write_page()\n break\n for bs_idx, bs_data in enumerate(heatmap_images):\n for local_idx, data in enumerate(zip(bs_data[0], bs_data[1],bs_data[2])):\n \n gt_pil = args.dataset_cls.colorize_mask(data[0].cpu().numpy())\n \n predictions_pil = data[1].cpu().numpy()\n predictions_pil = (predictions_pil / predictions_pil.max()) * 255\n predictions_pil = Image.fromarray(predictions_pil.astype(np.uint8))\n img_name = data[2]\n \n prediction_fn = '{}_prediction.png'.format(img_name)\n predictions_pil.save(os.path.join(to_save_dir, prediction_fn))\n gt_fn = '{}_gt.png'.format(img_name)\n gt_pil.save(os.path.join(to_save_dir, gt_fn))\n ip.add_table([(gt_fn, 'gt'), (prediction_fn, 'prediction')])\n val_visual.extend([visualize(gt_pil.convert('RGB')),\n visualize(predictions_pil.convert('RGB'))])\n idx = idx+1\n if idx >= 9:\n ip.write_page()\n break\n\n val_visual = torch.stack(val_visual, 0)\n val_visual = vutils.make_grid(val_visual, nrow=10, padding=5)\n writer.add_image(last_snapshot, val_visual)\n\n logging.info('-' * 107)\n fmt_str = '[epoch %d], [val loss %.5f], [mask f1 %.5f], [acc %.5f], [acc_cls %.5f], ' +\\\n '[mean_iu %.5f], [fwavacc %.5f]'\n logging.info(fmt_str % (epoch, val_loss.avg, mf_score.avg, acc, acc_cls, mean_iu, fwavacc))\n fmt_str = 'best record: [val loss %.5f], [mask f1 %.5f], [acc %.5f], [acc_cls %.5f], ' +\\\n '[mean_iu %.5f], [fwavacc %.5f], [epoch %d], '\n logging.info(fmt_str % (args.best_record['val_loss'], args.best_record['mask_f1_score'],\n args.best_record['acc'],\n args.best_record['acc_cls'], args.best_record['mean_iu'],\n args.best_record['fwavacc'], args.best_record['epoch']))\n logging.info('-' * 107)\n\n # tensorboard logging of validation phase metrics\n\n writer.add_scalar('training/acc', acc, epoch)\n writer.add_scalar('training/acc_cls', acc_cls, epoch)\n writer.add_scalar('training/mean_iu', mean_iu, epoch)\n writer.add_scalar('training/val_loss', val_loss.avg, epoch)\n writer.add_scalar('training/mask_f1_score', mf_score.avg, epoch)\n\n\ndef fast_hist(label_pred, label_true, num_classes):\n mask = (label_true >= 0) & (label_true < num_classes)\n hist = np.bincount(\n num_classes * label_true[mask].astype(int) +\n label_pred[mask], minlength=num_classes ** 2).reshape(num_classes, num_classes)\n return hist\n\n\n\ndef print_evaluate_results(hist, iu, writer=None, epoch=0, dataset=None):\n try:\n id2cat = dataset.id2cat\n except:\n id2cat = {i: i for i in range(dataset.num_classes)}\n iu_false_positive = hist.sum(axis=1) - np.diag(hist)\n iu_false_negative = hist.sum(axis=0) - np.diag(hist)\n iu_true_positive = np.diag(hist)\n logging.info('IoU:')\n logging.info('label_id label iU Precision Recall TP FP FN')\n for idx, i in enumerate(iu):\n idx_string = \"{:2d}\".format(idx)\n class_name = \"{:>13}\".format(id2cat[idx]) if idx in id2cat else ''\n iu_string = '{:5.2f}'.format(i * 100)\n total_pixels = hist.sum()\n tp = '{:5.2f}'.format(100 * iu_true_positive[idx] / total_pixels)\n fp = '{:5.2f}'.format(\n iu_false_positive[idx] / iu_true_positive[idx])\n fn = '{:5.2f}'.format(iu_false_negative[idx] / iu_true_positive[idx])\n precision = '{:5.2f}'.format(\n iu_true_positive[idx] / (iu_true_positive[idx] + iu_false_positive[idx]))\n recall = '{:5.2f}'.format(\n iu_true_positive[idx] / (iu_true_positive[idx] + iu_false_negative[idx]))\n logging.info('{} {} {} {} {} {} {} {}'.format(\n idx_string, class_name, iu_string, precision, recall, tp, fp, fn))\n\n writer.add_scalar('val_class_iu/{}'.format(id2cat[idx]), i * 100, epoch)\n writer.add_scalar('val_class_precision/{}'.format(id2cat[idx]),\n iu_true_positive[idx] / (iu_true_positive[idx] +\n iu_false_positive[idx]), epoch)\n writer.add_scalar('val_class_recall/{}'.format(id2cat[idx]),\n iu_true_positive[idx] / (iu_true_positive[idx] +\n iu_false_negative[idx]), epoch)\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/.gitignore",
"content": ".vscode\n__pycache__/\n*.py[co]\ndata/\nlog/\noutput/\nscripts/\ndetail-api/\ndata/list"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/LICENSE",
"content": "MIT License\n\nCopyright (c) [2019] [Microsoft]\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n\n=======================================================================================\n3-clause BSD licenses\n=======================================================================================\n1. syncbn - For details, see lib/models/syncbn/LICENSE\n Copyright (c) 2017 mapillary\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/README.md",
"content": "# High-resolution networks (HRNets) for Semantic Segmentation\n## Branches\n- This is the implementation for HRNet + OCR.\n- The PyTroch 1.1 version ia available [here](https://github.com/HRNet/HRNet-Semantic-Segmentation/tree/pytorch-v1.1).\n- The PyTroch 0.4.1 version is available [here](https://github.com/HRNet/HRNet-Semantic-Segmentation/tree/master).\n\n## News\n- [2020/08/16] [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) has supported our HRNet + OCR.\n- [2020/07/20] The researchers from AInnovation have achieved **Rank#1** on [ADE20K Leaderboard](http://sceneparsing.csail.mit.edu/) via training our HRNet + OCR with a semi-supervised learning scheme. More details are in their [Technical Report](https://arxiv.org/pdf/2007.10591.pdf).\n- [2020/07/09] Our paper is accepted by ECCV 2020: [Object-Contextual Representations for Semantic Segmentation](https://arxiv.org/pdf/1909.11065.pdf). Notably, the reseachers from Nvidia set a new state-of-the-art performance on Cityscapes leaderboard: [85.4%](https://www.cityscapes-dataset.com/method-details/?submissionID=7836) via combining our HRNet + OCR with a new [hierarchical mult-scale attention scheme](https://arxiv.org/abs/2005.10821). \n- [2020/03/13] Our paper is accepted by TPAMI: [Deep High-Resolution Representation Learning for Visual Recognition](https://arxiv.org/pdf/1908.07919.pdf).\n- HRNet + OCR + SegFix: Rank \\#1 (84.5) in [Cityscapes leaderboard](https://www.cityscapes-dataset.com/benchmarks/). OCR: object contextual represenations [pdf](https://arxiv.org/pdf/1909.11065.pdf). ***HRNet + OCR is reproduced [here](https://github.com/HRNet/HRNet-Semantic-Segmentation/tree/HRNet-OCR)***.\n- Thanks Google and UIUC researchers. A modified HRNet combined with semantic and instance multi-scale context achieves SOTA panoptic segmentation result on the Mapillary Vista challenge. See [the paper](https://arxiv.org/pdf/1910.04751.pdf).\n- Small HRNet models for Cityscapes segmentation. Superior to MobileNetV2Plus ....\n- Rank \\#1 (83.7) in [Cityscapes leaderboard](https://www.cityscapes-dataset.com/benchmarks/). HRNet combined with an extension of [object context](https://arxiv.org/pdf/1809.00916.pdf)\n\n- Pytorch-v1.1 and the official Sync-BN supported. We have reproduced the cityscapes results on the new codebase. Please check the [pytorch-v1.1 branch](https://github.com/HRNet/HRNet-Semantic-Segmentation/tree/pytorch-v1.1).\n\n## Introduction\nThis is the official code of [high-resolution representations for Semantic Segmentation](https://arxiv.org/abs/1904.04514). \nWe augment the HRNet with a very simple segmentation head shown in the figure below. We aggregate the output representations at four different resolutions, and then use a 1x1 convolutions to fuse these representations. The output representations is fed into the classifier. We evaluate our methods on three datasets, Cityscapes, PASCAL-Context and LIP.\n\n\n\n \n \n\nBesides, we further combine HRNet with [Object Contextual Representation](https://arxiv.org/pdf/1909.11065.pdf) and achieve higher performance on the three datasets. The code of HRNet+OCR is contained in this branch. We illustrate the overall framework of OCR in the Figure as shown below:\n\n\n \n \n\n## Segmentation models\nThe models are initialized by the weights pretrained on the ImageNet. You can download the pretrained models from https://github.com/HRNet/HRNet-Image-Classification. *Slightly different, we use align_corners = True for upsampling in HRNet*. \n\n1. Performance on the Cityscapes dataset. The models are trained and tested with the input size of 512x1024 and 1024x2048 respectively.\nIf multi-scale testing is used, we adopt scales: 0.5,0.75,1.0,1.25,1.5,1.75.\n\n| model | Train Set | Test Set | OHEM | Multi-scale| Flip | mIoU | Link |\n| :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: |\n| HRNetV2-W48 | Train | Val | No | No | No | 80.9 | [GoogleDrive](https://drive.google.com/file/d/15DCds5j95hI-nsjg4eBM1G3sIUWR9tmf/view?usp=sharing)/[BaiduYun(Access Code:pmix)](https://pan.baidu.com/s/1KyiOUOR0SYxKtJfIlD5o-w)|\n| HRNetV2-W48 + OCR | Train | Val | No | No | No | 81.6 | [GoogleDrive](https://drive.google.com/file/d/1QDxjWQhkBX_B3qVJykmtYUC3KkXVZIzT/view?usp=sharing)/[BaiduYun(Access Code:fa6i)](https://pan.baidu.com/s/1BGNt4Xmx3yfXUS8yjde0hQ)|\n| HRNetV2-W48 + OCR | Train + Val | Test | No | Yes | Yes | 82.3 | [GoogleDrive](https://drive.google.com/file/d/1HiB3pdFhhTtQnrM-zuKrNTmexz_7WmQa/view?usp=sharing)/[BaiduYun(Access Code:ycrk)](https://pan.baidu.com/s/16mD81UnGzjUBD-haDQfzIQ)|\n\n2. Performance on the LIP dataset. The models are trained and tested with the input size of 473x473.\n\n| model | OHEM | Multi-scale| Flip | mIoU | Link |\n| :--: | :--: | :--: | :--: | :--: | :--: |\n| HRNetV2-W48 | No | No | Yes | 55.83 | [GoogleDrive](https://drive.google.com/file/d/19Iva2nFGJkvvY9MUs_u3pH7wd50O-L6n/view?usp=sharing)/[BaiduYun(Access Code:fahi)](https://pan.baidu.com/s/15DamFiGEoxwDDF1TwuZdnA)|\n| HRNetV2-W48 + OCR | No | No | Yes | 56.48 | [GoogleDrive](https://drive.google.com/file/d/1coUt0IhZ7Ift7Ch7NdeUJAueohsrhEwF/view?usp=sharing)/[BaiduYun(Access Code:xex2)](https://pan.baidu.com/s/1dFYSR2bahRnvpIOdh88kOQ)|\n\n**Note** Currently we could only reproduce HRNet+OCR results on LIP dataset with PyTorch 0.4.1.\n\n3. Performance on the PASCAL-Context dataset. The models are trained and tested with the input size of 520x520.\nIf multi-scale testing is used, we adopt scales: 0.5,0.75,1.0,1.25,1.5,1.75,2.0 (the same as EncNet, DANet etc.).\n\n| model |num classes | OHEM | Multi-scale| Flip | mIoU | Link |\n| :--: | :--: | :--: | :--: | :--: | :--: | :--: |\n| HRNetV2-W48 | 59 classes | No | Yes | Yes | 54.1 | [GoogleDrive](https://drive.google.com/file/d/1yUcF4pxO4a2vUAdCUICF-DM2t9KT37hC/view?usp=sharing)/[BaiduYun(Access Code:wz6v)](https://pan.baidu.com/s/1m0MqpHSk0SX380EYEMawSA)|\n| HRNetV2-W48 + OCR | 59 classes | No | Yes | Yes | 56.2 | [GoogleDrive](https://drive.google.com/file/d/1ubHPoCErl7cYDLjjTHblMeBs1hHWBCOV/view?usp=sharing)/[BaiduYun(Access Code:yyxh)](https://pan.baidu.com/s/1XYP54gr3XB76tHmCcKdU9g)|\n| HRNetV2-W48 | 60 classes | No | Yes | Yes | 48.3 | [OneDrive](https://1drv.ms/u/s!Aus8VCZ_C_33gQEHDQrZCiv4R5mf)/[BaiduYun(Access Code:9uf8)](https://pan.baidu.com/s/1pgYt8P8ht2HOOzcA0F7Kag)|\n| HRNetV2-W48 + OCR | 60 classes | No | Yes | Yes | 50.1 | [GoogleDrive](https://drive.google.com/file/d/1ZAZ94GME3wmijF7ax5bqa0P3KxNLPUXR/view?usp=sharing)/[BaiduYun(Access Code:gtkb)](https://pan.baidu.com/s/13AYjwzh1LJSlipJwNpJ3Uw)|\n\n4. Performance on the COCO-Stuff dataset. The models are trained and tested with the input size of 520x520.\nIf multi-scale testing is used, we adopt scales: 0.5,0.75,1.0,1.25,1.5,1.75,2.0 (the same as EncNet, DANet etc.).\n\n| model | OHEM | Multi-scale| Flip | mIoU | Link |\n| :--: | :--: | :--: | :--: | :--: | :--: |\n| HRNetV2-W48 | Yes | No | No | 36.2 | [GoogleDrive](https://drive.google.com/open?id=1tXSWTCNyG4ETLfROJM1L6Lswg8wj5WvL)/[BaiduYun(Access Code:92gw)](https://pan.baidu.com/s/1VAV6KThH1Irzv9HZgLWE2Q)|\n| HRNetV2-W48 + OCR | Yes | No | No | 39.7 | [GoogleDrive](https://drive.google.com/open?id=1yMJ7-1-7LbbWotrqj1S4vM6M6Nj0feXv)/[BaiduYun(Access Code:sjc4)](https://pan.baidu.com/s/1HFSYyVwKBG3E6y76gcPjDA)|\n| HRNetV2-W48 | Yes | Yes | Yes | 37.9 | [GoogleDrive](https://drive.google.com/open?id=1tXSWTCNyG4ETLfROJM1L6Lswg8wj5WvL)/[BaiduYun(Access Code:92gw)](https://pan.baidu.com/s/1VAV6KThH1Irzv9HZgLWE2Q) |\n| HRNetV2-W48 + OCR | Yes | Yes | Yes | 40.6 | [GoogleDrive](https://drive.google.com/open?id=1yMJ7-1-7LbbWotrqj1S4vM6M6Nj0feXv)/[BaiduYun(Access Code:sjc4)](https://pan.baidu.com/s/1HFSYyVwKBG3E6y76gcPjDA) |\n\n**Note** We reproduce HRNet+OCR results on COCO-Stuff dataset with PyTorch 0.4.1.\n\n5. Performance on the ADE20K dataset. The models are trained and tested with the input size of 520x520.\nIf multi-scale testing is used, we adopt scales: 0.5,0.75,1.0,1.25,1.5,1.75,2.0 (the same as EncNet, DANet etc.).\n\n| model | OHEM | Multi-scale| Flip | mIoU | Link |\n| :--: | :--: | :--: | :--: | :--: | :--: |\n| HRNetV2-W48 | Yes | No | No | 43.1 | [GoogleDrive](https://drive.google.com/open?id=1OlTm8k3fIQpZXmOKXipd5BdxVYtbSWt-)/[BaiduYun(Access Code:f6xf)](https://pan.baidu.com/s/11neVkzxx27qS2-mPFW9dfg)|\n| HRNetV2-W48 + OCR | Yes | No | No | 44.5 | [GoogleDrive](https://drive.google.com/open?id=1JEzwhkcPUc-HXnq5ErbWy0vWnNpI9sZ8)/[BaiduYun(Access Code:peg4)](https://pan.baidu.com/s/1HLhjiLIdgaOHs0SzEtkgkQ)|\n| HRNetV2-W48 | Yes | Yes | Yes | 44.2 | [GoogleDrive](https://drive.google.com/open?id=1OlTm8k3fIQpZXmOKXipd5BdxVYtbSWt-)/[BaiduYun(Access Code:f6xf)](https://pan.baidu.com/s/11neVkzxx27qS2-mPFW9dfg) |\n| HRNetV2-W48 + OCR | Yes | Yes | Yes | 45.5 | [GoogleDrive](https://drive.google.com/open?id=1JEzwhkcPUc-HXnq5ErbWy0vWnNpI9sZ8)/[BaiduYun(Access Code:peg4)](https://pan.baidu.com/s/1HLhjiLIdgaOHs0SzEtkgkQ) |\n\n**Note** We reproduce HRNet+OCR results on ADE20K dataset with PyTorch 0.4.1.\n\n## Quick start\n### Install\n1. For LIP dataset, install PyTorch=0.4.1 following the [official instructions](https://pytorch.org/). For Cityscapes and PASCAL-Context, we use PyTorch=1.1.0.\n2. `git clone https://github.com/HRNet/HRNet-Semantic-Segmentation $SEG_ROOT`\n3. Install dependencies: pip install -r requirements.txt\n\nIf you want to train and evaluate our models on PASCAL-Context, you need to install [details](https://github.com/zhanghang1989/detail-api).\n````bash\npip install git+https://github.com/zhanghang1989/detail-api.git#subdirectory=PythonAPI\n````\n\n### Data preparation\nYou need to download the [Cityscapes](https://www.cityscapes-dataset.com/), [LIP](http://sysu-hcp.net/lip/) and [PASCAL-Context](https://cs.stanford.edu/~roozbeh/pascal-context/) datasets.\n\nYour directory tree should be look like this:\n````bash\n$SEG_ROOT/data\n├── cityscapes\n│ ├── gtFine\n│ │ ├── test\n│ │ ├── train\n│ │ └── val\n│ └── leftImg8bit\n│ ├── test\n│ ├── train\n│ └── val\n├── lip\n│ ├── TrainVal_images\n│ │ ├── train_images\n│ │ └── val_images\n│ └── TrainVal_parsing_annotations\n│ ├── train_segmentations\n│ ├── train_segmentations_reversed\n│ └── val_segmentations\n├── pascal_ctx\n│ ├── common\n│ ├── PythonAPI\n│ ├── res\n│ └── VOCdevkit\n│ └── VOC2010\n├── cocostuff\n│ ├── train\n│ │ ├── image\n│ │ └── label\n│ └── val\n│ ├── image\n│ └── label\n├── ade20k\n│ ├── train\n│ │ ├── image\n│ │ └── label\n│ └── val\n│ ├── image\n│ └── label\n├── list\n│ ├── cityscapes\n│ │ ├── test.lst\n│ │ ├── trainval.lst\n│ │ └── val.lst\n│ ├── lip\n│ │ ├── testvalList.txt\n│ │ ├── trainList.txt\n│ │ └── valList.txt\n````\n\n### Train and Test\n\n#### PyTorch Version Differences\n\nNote that the codebase supports both PyTorch 0.4.1 and 1.1.0, and they use different command for training. In the following context, we use `$PY_CMD` to denote different startup command.\n\n```bash\n# For PyTorch 0.4.1\nPY_CMD=\"python\"\n# For PyTorch 1.1.0\nPY_CMD=\"python -m torch.distributed.launch --nproc_per_node=4\"\n```\n\ne.g., when training on Cityscapes, we use PyTorch 1.1.0. So the command\n````bash\n$PY_CMD tools/train.py --cfg experiments/cityscapes/seg_hrnet_ocr_w48_train_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484.yaml\n````\nindicates\n````bash\npython -m torch.distributed.launch --nproc_per_node=4 tools/train.py --cfg experiments/cityscapes/seg_hrnet_ocr_w48_train_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484.yaml\n````\n#### Training\n\nJust specify the configuration file for `tools/train.py`.\n\nFor example, train the HRNet-W48 on Cityscapes with a batch size of 12 on 4 GPUs:\n````bash\n$PY_CMD tools/train.py --cfg experiments/cityscapes/seg_hrnet_w48_train_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484.yaml\n````\nFor example, train the HRNet-W48 + OCR on Cityscapes with a batch size of 12 on 4 GPUs:\n````bash\n$PY_CMD tools/train.py --cfg experiments/cityscapes/seg_hrnet_ocr_w48_train_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484.yaml\n````\n\nNote that we only reproduce HRNet+OCR on LIP dataset using PyTorch 0.4.1. So we recommend to use PyTorch 0.4.1 if you want to train on LIP dataset.\n\n#### Testing\n\nFor example, evaluating HRNet+OCR on the Cityscapes validation set with multi-scale and flip testing:\n````bash\npython tools/test.py --cfg experiments/cityscapes/seg_hrnet_ocr_w48_train_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484.yaml \\\n TEST.MODEL_FILE hrnet_ocr_cs_8162_torch11.pth \\\n TEST.SCALE_LIST 0.5,0.75,1.0,1.25,1.5,1.75 \\\n TEST.FLIP_TEST True\n````\nEvaluating HRNet+OCR on the Cityscapes test set with multi-scale and flip testing:\n````bash\npython tools/test.py --cfg experiments/cityscapes/seg_hrnet_ocr_w48_train_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484.yaml \\\n DATASET.TEST_SET list/cityscapes/test.lst \\\n TEST.MODEL_FILE hrnet_ocr_trainval_cs_8227_torch11.pth \\\n TEST.SCALE_LIST 0.5,0.75,1.0,1.25,1.5,1.75 \\\n TEST.FLIP_TEST True\n````\nEvaluating HRNet+OCR on the PASCAL-Context validation set with multi-scale and flip testing:\n````bash\npython tools/test.py --cfg experiments/pascal_ctx/seg_hrnet_ocr_w48_cls59_520x520_sgd_lr1e-3_wd1e-4_bs_16_epoch200.yaml \\\n DATASET.TEST_SET testval \\\n TEST.MODEL_FILE hrnet_ocr_pascal_ctx_5618_torch11.pth \\\n TEST.SCALE_LIST 0.5,0.75,1.0,1.25,1.5,1.75,2.0 \\\n TEST.FLIP_TEST True\n````\nEvaluating HRNet+OCR on the LIP validation set with flip testing:\n````bash\npython tools/test.py --cfg experiments/lip/seg_hrnet_w48_473x473_sgd_lr7e-3_wd5e-4_bs_40_epoch150.yaml \\\n DATASET.TEST_SET list/lip/testvalList.txt \\\n TEST.MODEL_FILE hrnet_ocr_lip_5648_torch04.pth \\\n TEST.FLIP_TEST True \\\n TEST.NUM_SAMPLES 0\n````\nEvaluating HRNet+OCR on the COCO-Stuff validation set with multi-scale and flip testing:\n````bash\npython tools/test.py --cfg experiments/cocostuff/seg_hrnet_ocr_w48_520x520_ohem_sgd_lr1e-3_wd1e-4_bs_16_epoch110.yaml \\\n DATASET.TEST_SET list/cocostuff/testval.lst \\\n TEST.MODEL_FILE hrnet_ocr_cocostuff_3965_torch04.pth \\\n TEST.SCALE_LIST 0.5,0.75,1.0,1.25,1.5,1.75,2.0 \\\n TEST.MULTI_SCALE True TEST.FLIP_TEST True\n````\nEvaluating HRNet+OCR on the ADE20K validation set with multi-scale and flip testing:\n````bash\npython tools/test.py --cfg experiments/ade20k/seg_hrnet_ocr_w48_520x520_ohem_sgd_lr2e-2_wd1e-4_bs_16_epoch120.yaml \\\n DATASET.TEST_SET list/cocostuff/testval.lst \\\n TEST.MODEL_FILE hrnet_ocr_ade20k_4451_torch04.pth \\\n TEST.SCALE_LIST 0.5,0.75,1.0,1.25,1.5,1.75,2.0 \\\n TEST.MULTI_SCALE True TEST.FLIP_TEST True\n````\n\n## Other applications of HRNet\n* [Human pose estimation](https://github.com/leoxiaobin/deep-high-resolution-net.pytorch)\n* [Image Classification](https://github.com/HRNet/HRNet-Image-Classification)\n* [Object detection](https://github.com/HRNet/HRNet-Object-Detection)\n* [Facial landmark detection](https://github.com/HRNet/HRNet-Facial-Landmark-Detection)\n\n## Citation\nIf you find this work or code is helpful in your research, please cite:\n````\n@inproceedings{SunXLW19,\n title={Deep High-Resolution Representation Learning for Human Pose Estimation},\n author={Ke Sun and Bin Xiao and Dong Liu and Jingdong Wang},\n booktitle={CVPR},\n year={2019}\n}\n\n@article{WangSCJDZLMTWLX19,\n title={Deep High-Resolution Representation Learning for Visual Recognition},\n author={Jingdong Wang and Ke Sun and Tianheng Cheng and \n Borui Jiang and Chaorui Deng and Yang Zhao and Dong Liu and Yadong Mu and \n Mingkui Tan and Xinggang Wang and Wenyu Liu and Bin Xiao},\n journal={TPAMI},\n year={2019}\n}\n\n@article{YuanCW19,\n title={Object-Contextual Representations for Semantic Segmentation},\n author={Yuhui Yuan and Xilin Chen and Jingdong Wang},\n booktitle={ECCV},\n year={2020}\n}\n````\n\n## Reference\n[1] Deep High-Resolution Representation Learning for Visual Recognition. Jingdong Wang, Ke Sun, Tianheng Cheng, \n Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang, Wenyu Liu, Bin Xiao. Accepted by TPAMI. [download](https://arxiv.org/pdf/1908.07919.pdf)\n \n[2] Object-Contextual Representations for Semantic Segmentation. Yuhui Yuan, Xilin Chen, Jingdong Wang. [download](https://arxiv.org/pdf/1909.11065.pdf)\n\n## Acknowledgement\nWe adopt sync-bn implemented by [InplaceABN](https://github.com/mapillary/inplace_abn) for PyTorch 0.4.1 experiments and the official \nsync-bn provided by PyTorch for PyTorch 1.10 experiments.\n\nWe adopt data precosessing on the PASCAL-Context dataset, implemented by [PASCAL API](https://github.com/zhanghang1989/detail-api).\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/experiments/ade20k/seg_hrnet_ocr_w48_520x520_ohem_sgd_lr2e-2_wd1e-4_bs_16_epoch120.yaml",
"content": "CUDNN:\n BENCHMARK: true\n DETERMINISTIC: false\n ENABLED: true\nGPUS: (0,1,2,3)\nOUTPUT_DIR: 'output'\nLOG_DIR: 'log'\nWORKERS: 4\nPRINT_FREQ: 10\n\nDATASET:\n DATASET: ade20k\n ROOT: 'data/'\n TEST_SET: 'list/ade20k/val.lst'\n TRAIN_SET: 'list/ade20k/train.lst'\n NUM_CLASSES: 150\nMODEL:\n NAME: seg_hrnet_ocr\n NUM_OUTPUTS: 2\n PRETRAINED: 'pretrained_models/hrnetv2_w48_imagenet_pretrained.pth'\n EXTRA:\n FINAL_CONV_KERNEL: 1\n STAGE1:\n NUM_MODULES: 1\n NUM_RANCHES: 1\n BLOCK: BOTTLENECK\n NUM_BLOCKS:\n - 4\n NUM_CHANNELS:\n - 64\n FUSE_METHOD: SUM\n STAGE2:\n NUM_MODULES: 1\n NUM_BRANCHES: 2\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n FUSE_METHOD: SUM\n STAGE3:\n NUM_MODULES: 4\n NUM_BRANCHES: 3\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n FUSE_METHOD: SUM\n STAGE4:\n NUM_MODULES: 3\n NUM_BRANCHES: 4\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n - 384\n FUSE_METHOD: SUM\nLOSS:\n USE_OHEM: true\n OHEMTHRES: 0.9\n OHEMKEEP: 131072\n BALANCE_WEIGHTS: [0.4, 1]\nTRAIN:\n IMAGE_SIZE:\n - 520\n - 520\n BASE_SIZE: 520\n BATCH_SIZE_PER_GPU: 4\n SHUFFLE: true\n BEGIN_EPOCH: 0\n END_EPOCH: 120\n RESUME: true\n OPTIMIZER: sgd\n LR: 0.02\n WD: 0.0001\n MOMENTUM: 0.9\n NESTEROV: false\n FLIP: true\n MULTI_SCALE: true\n DOWNSAMPLERATE: 1\n IGNORE_LABEL: 255\n SCALE_FACTOR: 16\nTEST:\n IMAGE_SIZE:\n - 520\n - 520\n BASE_SIZE: 520\n BATCH_SIZE_PER_GPU: 1\n NUM_SAMPLES: 200\n FLIP_TEST: false\n MULTI_SCALE: false\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/experiments/ade20k/seg_hrnet_w48_520x520_ohem_sgd_lr2e-2_wd1e-4_bs_16_epoch120.yaml",
"content": "CUDNN:\n BENCHMARK: true\n DETERMINISTIC: false\n ENABLED: true\nGPUS: (0,1,2,3)\nOUTPUT_DIR: 'output'\nLOG_DIR: 'log'\nWORKERS: 4\nPRINT_FREQ: 10\n\nDATASET:\n DATASET: ade20k\n ROOT: 'data/'\n TEST_SET: 'list/ade20k/val.lst'\n TRAIN_SET: 'list/ade20k/train.lst'\n NUM_CLASSES: 150\nMODEL:\n NAME: seg_hrnet\n NUM_OUTPUTS: 1\n PRETRAINED: 'pretrained_models/hrnetv2_w48_imagenet_pretrained.pth'\n EXTRA:\n FINAL_CONV_KERNEL: 1\n STAGE1:\n NUM_MODULES: 1\n NUM_RANCHES: 1\n BLOCK: BOTTLENECK\n NUM_BLOCKS:\n - 4\n NUM_CHANNELS:\n - 64\n FUSE_METHOD: SUM\n STAGE2:\n NUM_MODULES: 1\n NUM_BRANCHES: 2\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n FUSE_METHOD: SUM\n STAGE3:\n NUM_MODULES: 4\n NUM_BRANCHES: 3\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n FUSE_METHOD: SUM\n STAGE4:\n NUM_MODULES: 3\n NUM_BRANCHES: 4\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n - 384\n FUSE_METHOD: SUM\nLOSS:\n USE_OHEM: false\n OHEMTHRES: 0.9\n OHEMKEEP: 131072\nTRAIN:\n IMAGE_SIZE:\n - 520\n - 520\n BASE_SIZE: 520\n BATCH_SIZE_PER_GPU: 4\n SHUFFLE: true\n BEGIN_EPOCH: 0\n END_EPOCH: 120\n RESUME: true\n OPTIMIZER: sgd\n LR: 0.02\n WD: 0.0001\n MOMENTUM: 0.9\n NESTEROV: false\n FLIP: true\n MULTI_SCALE: true\n DOWNSAMPLERATE: 1\n IGNORE_LABEL: 255\n SCALE_FACTOR: 11\nTEST:\n IMAGE_SIZE:\n - 520\n - 520\n BASE_SIZE: 520\n BATCH_SIZE_PER_GPU: 1\n NUM_SAMPLES: 200\n FLIP_TEST: false\n MULTI_SCALE: false\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/experiments/ade20k/seg_hrnet_w48_520x520_sgd_lr2e-2_wd1e-4_bs_16_epoch120.yaml",
"content": "CUDNN:\n BENCHMARK: true\n DETERMINISTIC: false\n ENABLED: true\nGPUS: (0,1,2,3)\nOUTPUT_DIR: 'output'\nLOG_DIR: 'log'\nWORKERS: 4\nPRINT_FREQ: 10\n\nDATASET:\n DATASET: ade20k\n ROOT: 'data/'\n TEST_SET: 'list/ade20k/val.lst'\n TRAIN_SET: 'list/ade20k/train.lst'\n NUM_CLASSES: 150\nMODEL:\n NAME: seg_hrnet\n NUM_OUTPUTS: 1\n PRETRAINED: 'pretrained_models/hrnetv2_w48_imagenet_pretrained.pth'\n EXTRA:\n FINAL_CONV_KERNEL: 1\n STAGE1:\n NUM_MODULES: 1\n NUM_RANCHES: 1\n BLOCK: BOTTLENECK\n NUM_BLOCKS:\n - 4\n NUM_CHANNELS:\n - 64\n FUSE_METHOD: SUM\n STAGE2:\n NUM_MODULES: 1\n NUM_BRANCHES: 2\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n FUSE_METHOD: SUM\n STAGE3:\n NUM_MODULES: 4\n NUM_BRANCHES: 3\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n FUSE_METHOD: SUM\n STAGE4:\n NUM_MODULES: 3\n NUM_BRANCHES: 4\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n - 384\n FUSE_METHOD: SUM\nLOSS:\n USE_OHEM: false\n OHEMTHRES: 0.9\n OHEMKEEP: 131072\nTRAIN:\n IMAGE_SIZE:\n - 520\n - 520\n BASE_SIZE: 520\n BATCH_SIZE_PER_GPU: 4\n SHUFFLE: true\n BEGIN_EPOCH: 0\n END_EPOCH: 120\n RESUME: true\n OPTIMIZER: sgd\n LR: 0.02\n WD: 0.0001\n MOMENTUM: 0.9\n NESTEROV: false\n FLIP: true\n MULTI_SCALE: true\n DOWNSAMPLERATE: 1\n IGNORE_LABEL: 255\n SCALE_FACTOR: 11\nTEST:\n IMAGE_SIZE:\n - 520\n - 520\n BASE_SIZE: 520\n BATCH_SIZE_PER_GPU: 1\n NUM_SAMPLES: 200\n FLIP_TEST: false\n MULTI_SCALE: false\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/experiments/cityscapes/seg_hrnet_ocr_w48_train_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484.yaml",
"content": "CUDNN:\n BENCHMARK: true\n DETERMINISTIC: false\n ENABLED: true\nGPUS: (0,1,2,3)\nOUTPUT_DIR: 'output'\nLOG_DIR: 'log'\nWORKERS: 4\nPRINT_FREQ: 10\n\nDATASET:\n DATASET: cityscapes\n ROOT: data/\n TEST_SET: 'list/cityscapes/val.lst'\n TRAIN_SET: 'list/cityscapes/train.lst'\n NUM_CLASSES: 19\nMODEL:\n NAME: seg_hrnet_ocr\n NUM_OUTPUTS: 2\n PRETRAINED: \"pretrained_models/hrnetv2_w48_imagenet_pretrained.pth\"\n EXTRA:\n FINAL_CONV_KERNEL: 1\n STAGE1:\n NUM_MODULES: 1\n NUM_RANCHES: 1\n BLOCK: BOTTLENECK\n NUM_BLOCKS:\n - 4\n NUM_CHANNELS:\n - 64\n FUSE_METHOD: SUM\n STAGE2:\n NUM_MODULES: 1\n NUM_BRANCHES: 2\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n FUSE_METHOD: SUM\n STAGE3:\n NUM_MODULES: 4\n NUM_BRANCHES: 3\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n FUSE_METHOD: SUM\n STAGE4:\n NUM_MODULES: 3\n NUM_BRANCHES: 4\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n - 384\n FUSE_METHOD: SUM\nLOSS:\n USE_OHEM: false\n OHEMTHRES: 0.9\n OHEMKEEP: 131072\n BALANCE_WEIGHTS: [0.4, 1]\nTRAIN:\n IMAGE_SIZE:\n - 1024\n - 512\n BASE_SIZE: 2048\n BATCH_SIZE_PER_GPU: 3\n SHUFFLE: true\n BEGIN_EPOCH: 0\n END_EPOCH: 484\n RESUME: true\n OPTIMIZER: sgd\n LR: 0.01\n WD: 0.0005\n MOMENTUM: 0.9\n NESTEROV: false\n FLIP: true\n MULTI_SCALE: true\n DOWNSAMPLERATE: 1\n IGNORE_LABEL: 255\n SCALE_FACTOR: 16\nTEST:\n IMAGE_SIZE:\n - 2048\n - 1024\n BASE_SIZE: 2048\n BATCH_SIZE_PER_GPU: 4\n FLIP_TEST: false\n MULTI_SCALE: false\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/experiments/cityscapes/seg_hrnet_ocr_w48_trainval_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484.yaml",
"content": "CUDNN:\n BENCHMARK: true\n DETERMINISTIC: false\n ENABLED: true\nGPUS: (0,1,2,3)\nOUTPUT_DIR: 'output'\nLOG_DIR: 'log'\nWORKERS: 4\nPRINT_FREQ: 10\n\nDATASET:\n DATASET: cityscapes\n ROOT: data/\n TEST_SET: 'list/cityscapes/val.lst'\n TRAIN_SET: 'list/cityscapes/trainval.lst'\n NUM_CLASSES: 19\nMODEL:\n NAME: seg_hrnet_ocr\n NUM_OUTPUTS: 2\n PRETRAINED: \"pretrained_models/hrnetv2_w48_imagenet_pretrained.pth\"\n EXTRA:\n FINAL_CONV_KERNEL: 1\n STAGE1:\n NUM_MODULES: 1\n NUM_RANCHES: 1\n BLOCK: BOTTLENECK\n NUM_BLOCKS:\n - 4\n NUM_CHANNELS:\n - 64\n FUSE_METHOD: SUM\n STAGE2:\n NUM_MODULES: 1\n NUM_BRANCHES: 2\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n FUSE_METHOD: SUM\n STAGE3:\n NUM_MODULES: 4\n NUM_BRANCHES: 3\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n FUSE_METHOD: SUM\n STAGE4:\n NUM_MODULES: 3\n NUM_BRANCHES: 4\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n - 384\n FUSE_METHOD: SUM\nLOSS:\n USE_OHEM: false\n OHEMTHRES: 0.9\n OHEMKEEP: 131072\n BALANCE_WEIGHTS: [0.4, 1]\nTRAIN:\n IMAGE_SIZE:\n - 1024\n - 512\n BASE_SIZE: 2048\n BATCH_SIZE_PER_GPU: 3\n SHUFFLE: true\n BEGIN_EPOCH: 0\n END_EPOCH: 484\n RESUME: true\n OPTIMIZER: sgd\n LR: 0.01\n WD: 0.0005\n MOMENTUM: 0.9\n NESTEROV: false\n FLIP: true\n MULTI_SCALE: true\n DOWNSAMPLERATE: 1\n IGNORE_LABEL: 255\n SCALE_FACTOR: 16\nTEST:\n IMAGE_SIZE:\n - 2048\n - 1024\n BASE_SIZE: 2048\n BATCH_SIZE_PER_GPU: 4\n FLIP_TEST: false\n MULTI_SCALE: false\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/experiments/cityscapes/seg_hrnet_w48_train_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484.yaml",
"content": "CUDNN:\n BENCHMARK: true\n DETERMINISTIC: false\n ENABLED: true\nGPUS: (0,1,2,3)\nOUTPUT_DIR: 'output'\nLOG_DIR: 'log'\nWORKERS: 4\nPRINT_FREQ: 100\n\nDATASET:\n DATASET: cityscapes\n ROOT: 'data/'\n TEST_SET: 'list/cityscapes/val.lst'\n TRAIN_SET: 'list/cityscapes/train.lst'\n NUM_CLASSES: 19\nMODEL:\n NAME: seg_hrnet\n ALIGN_CORNERS: False\n PRETRAINED: 'pretrained_models/hrnetv2_w48_imagenet_pretrained.pth'\n EXTRA:\n FINAL_CONV_KERNEL: 1\n STAGE1:\n NUM_MODULES: 1\n NUM_RANCHES: 1\n BLOCK: BOTTLENECK\n NUM_BLOCKS:\n - 4\n NUM_CHANNELS:\n - 64\n FUSE_METHOD: SUM\n STAGE2:\n NUM_MODULES: 1\n NUM_BRANCHES: 2\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n FUSE_METHOD: SUM\n STAGE3:\n NUM_MODULES: 4\n NUM_BRANCHES: 3\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n FUSE_METHOD: SUM\n STAGE4:\n NUM_MODULES: 3\n NUM_BRANCHES: 4\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n - 384\n FUSE_METHOD: SUM\nLOSS:\n USE_OHEM: false\n OHEMTHRES: 0.9\n OHEMKEEP: 131072\nTRAIN:\n IMAGE_SIZE:\n - 1024\n - 512\n BASE_SIZE: 2048\n BATCH_SIZE_PER_GPU: 3\n SHUFFLE: true\n BEGIN_EPOCH: 0\n END_EPOCH: 484\n RESUME: true\n OPTIMIZER: sgd\n LR: 0.01\n WD: 0.0005\n MOMENTUM: 0.9\n NESTEROV: false\n FLIP: true\n MULTI_SCALE: true\n DOWNSAMPLERATE: 1\n IGNORE_LABEL: 255\n SCALE_FACTOR: 16\nTEST:\n IMAGE_SIZE:\n - 2048\n - 1024\n BASE_SIZE: 2048\n BATCH_SIZE_PER_GPU: 4\n FLIP_TEST: false\n MULTI_SCALE: false\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/experiments/cityscapes/seg_hrnet_w48_train_ohem_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484.yaml",
"content": "CUDNN:\n BENCHMARK: true\n DETERMINISTIC: false\n ENABLED: true\nGPUS: (0,1,2,3)\nOUTPUT_DIR: 'output'\nLOG_DIR: 'log'\nWORKERS: 4\nPRINT_FREQ: 100\n\nDATASET:\n DATASET: cityscapes\n ROOT: 'data/'\n TEST_SET: 'list/cityscapes/val.lst'\n TRAIN_SET: 'list/cityscapes/train.lst'\n NUM_CLASSES: 19\nMODEL:\n NAME: seg_hrnet\n ALIGN_CORNERS: False\n PRETRAINED: 'pretrained_models/hrnetv2_w48_imagenet_pretrained.pth'\n EXTRA:\n FINAL_CONV_KERNEL: 1\n STAGE1:\n NUM_MODULES: 1\n NUM_RANCHES: 1\n BLOCK: BOTTLENECK\n NUM_BLOCKS:\n - 4\n NUM_CHANNELS:\n - 64\n FUSE_METHOD: SUM\n STAGE2:\n NUM_MODULES: 1\n NUM_BRANCHES: 2\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n FUSE_METHOD: SUM\n STAGE3:\n NUM_MODULES: 4\n NUM_BRANCHES: 3\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n FUSE_METHOD: SUM\n STAGE4:\n NUM_MODULES: 3\n NUM_BRANCHES: 4\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n - 384\n FUSE_METHOD: SUM\nLOSS:\n USE_OHEM: true\n OHEMTHRES: 0.9\n OHEMKEEP: 131072\nTRAIN:\n IMAGE_SIZE:\n - 1024\n - 512\n BASE_SIZE: 2048\n BATCH_SIZE_PER_GPU: 3\n SHUFFLE: true\n BEGIN_EPOCH: 0\n END_EPOCH: 484\n RESUME: true\n OPTIMIZER: sgd\n LR: 0.01\n WD: 0.0005\n MOMENTUM: 0.9\n NESTEROV: false\n FLIP: true\n MULTI_SCALE: true\n DOWNSAMPLERATE: 1\n IGNORE_LABEL: 255\n SCALE_FACTOR: 16\nTEST:\n IMAGE_SIZE:\n - 2048\n - 1024\n BASE_SIZE: 2048\n BATCH_SIZE_PER_GPU: 4\n FLIP_TEST: false\n MULTI_SCALE: false\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/experiments/cityscapes/seg_hrnet_w48_trainval_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484x2.yaml",
"content": "CUDNN:\n BENCHMARK: true\n DETERMINISTIC: false\n ENABLED: true\nGPUS: (0,1,2,3)\nOUTPUT_DIR: 'output'\nLOG_DIR: 'log'\nWORKERS: 4\nPRINT_FREQ: 100\n\nDATASET:\n DATASET: cityscapes\n ROOT: 'data/'\n TEST_SET: 'list/cityscapes/val.lst'\n TRAIN_SET: 'list/cityscapes/train.lst'\n EXTRA_TRAIN_SET: 'list/cityscapes/trainval.lst'\n NUM_CLASSES: 19\nMODEL:\n NAME: seg_hrnet\n ALIGN_CORNERS: False\n PRETRAINED: 'pretrained_models/hrnetv2_w48_imagenet_pretrained.pth'\n EXTRA:\n FINAL_CONV_KERNEL: 1\n STAGE1:\n NUM_MODULES: 1\n NUM_RANCHES: 1\n BLOCK: BOTTLENECK\n NUM_BLOCKS:\n - 4\n NUM_CHANNELS:\n - 64\n FUSE_METHOD: SUM\n STAGE2:\n NUM_MODULES: 1\n NUM_BRANCHES: 2\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n FUSE_METHOD: SUM\n STAGE3:\n NUM_MODULES: 4\n NUM_BRANCHES: 3\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n FUSE_METHOD: SUM\n STAGE4:\n NUM_MODULES: 3\n NUM_BRANCHES: 4\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n - 384\n FUSE_METHOD: SUM\nLOSS:\n USE_OHEM: false\n OHEMTHRES: 0.9\n OHEMKEEP: 131072\nTRAIN:\n IMAGE_SIZE:\n - 1024\n - 512\n BASE_SIZE: 2048\n BATCH_SIZE_PER_GPU: 3\n SHUFFLE: true\n BEGIN_EPOCH: 0\n END_EPOCH: 484\n EXTRA_EPOCH: 484\n RESUME: true\n OPTIMIZER: sgd\n LR: 0.01\n EXTRA_LR: 0.001\n WD: 0.0005\n MOMENTUM: 0.9\n NESTEROV: false\n FLIP: true\n MULTI_SCALE: true\n DOWNSAMPLERATE: 1\n IGNORE_LABEL: 255\n SCALE_FACTOR: 16\nTEST:\n IMAGE_SIZE:\n - 2048\n - 1024\n BASE_SIZE: 2048\n BATCH_SIZE_PER_GPU: 4\n FLIP_TEST: false\n MULTI_SCALE: false\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/experiments/cityscapes/seg_hrnet_w48_trainval_ohem_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484x2.yaml",
"content": "CUDNN:\n BENCHMARK: true\n DETERMINISTIC: false\n ENABLED: true\nGPUS: (0,1,2,3)\nOUTPUT_DIR: 'output'\nLOG_DIR: 'log'\nWORKERS: 4\nPRINT_FREQ: 100\n\nDATASET:\n DATASET: cityscapes\n ROOT: 'data/'\n TEST_SET: 'list/cityscapes/val.lst'\n TRAIN_SET: 'list/cityscapes/train.lst'\n EXTRA_TRAIN_SET: 'list/cityscapes/trainval.lst'\n NUM_CLASSES: 19\nMODEL:\n NAME: seg_hrnet\n ALIGN_CORNERS: False\n PRETRAINED: 'pretrained_models/hrnetv2_w48_imagenet_pretrained.pth'\n EXTRA:\n FINAL_CONV_KERNEL: 1\n STAGE1:\n NUM_MODULES: 1\n NUM_RANCHES: 1\n BLOCK: BOTTLENECK\n NUM_BLOCKS:\n - 4\n NUM_CHANNELS:\n - 64\n FUSE_METHOD: SUM\n STAGE2:\n NUM_MODULES: 1\n NUM_BRANCHES: 2\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n FUSE_METHOD: SUM\n STAGE3:\n NUM_MODULES: 4\n NUM_BRANCHES: 3\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n FUSE_METHOD: SUM\n STAGE4:\n NUM_MODULES: 3\n NUM_BRANCHES: 4\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n - 384\n FUSE_METHOD: SUM\nLOSS:\n USE_OHEM: true\n OHEMTHRES: 0.9\n OHEMKEEP: 131072\nTRAIN:\n IMAGE_SIZE:\n - 1024\n - 512\n BASE_SIZE: 2048\n BATCH_SIZE_PER_GPU: 3\n SHUFFLE: true\n BEGIN_EPOCH: 0\n END_EPOCH: 484\n EXTRA_EPOCH: 484\n RESUME: true\n OPTIMIZER: sgd\n LR: 0.01\n EXTRA_LR: 0.001\n WD: 0.0005\n MOMENTUM: 0.9\n NESTEROV: false\n FLIP: true\n MULTI_SCALE: true\n DOWNSAMPLERATE: 1\n IGNORE_LABEL: 255\n SCALE_FACTOR: 16\nTEST:\n IMAGE_SIZE:\n - 2048\n - 1024\n BASE_SIZE: 2048\n BATCH_SIZE_PER_GPU: 4\n FLIP_TEST: false\n MULTI_SCALE: false\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/experiments/cocostuff/seg_hrnet_ocr_w48_520x520_ohem_sgd_lr1e-3_wd1e-4_bs_16_epoch110.yaml",
"content": "CUDNN:\n BENCHMARK: true\n DETERMINISTIC: false\n ENABLED: true\nGPUS: (0,1,2,3)\nOUTPUT_DIR: 'output'\nLOG_DIR: 'log'\nWORKERS: 4\nPRINT_FREQ: 10\n\nDATASET:\n DATASET: cocostuff\n ROOT: 'data/'\n TEST_SET: 'list/cocostuff/val.lst'\n TRAIN_SET: 'list/cocostuff/train.lst'\n NUM_CLASSES: 171\nMODEL:\n NAME: seg_hrnet_ocr\n NUM_OUTPUTS: 2\n PRETRAINED: 'pretrained_models/hrnetv2_w48_imagenet_pretrained.pth'\n EXTRA:\n FINAL_CONV_KERNEL: 1\n STAGE1:\n NUM_MODULES: 1\n NUM_RANCHES: 1\n BLOCK: BOTTLENECK\n NUM_BLOCKS:\n - 4\n NUM_CHANNELS:\n - 64\n FUSE_METHOD: SUM\n STAGE2:\n NUM_MODULES: 1\n NUM_BRANCHES: 2\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n FUSE_METHOD: SUM\n STAGE3:\n NUM_MODULES: 4\n NUM_BRANCHES: 3\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n FUSE_METHOD: SUM\n STAGE4:\n NUM_MODULES: 3\n NUM_BRANCHES: 4\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n - 384\n FUSE_METHOD: SUM\nLOSS:\n USE_OHEM: true\n OHEMTHRES: 0.9\n OHEMKEEP: 131072\n BALANCE_WEIGHTS: [0.4, 1]\nTRAIN:\n IMAGE_SIZE:\n - 520\n - 520\n BASE_SIZE: 520\n BATCH_SIZE_PER_GPU: 4\n SHUFFLE: true\n BEGIN_EPOCH: 0\n END_EPOCH: 110\n RESUME: true\n OPTIMIZER: sgd\n LR: 0.001\n WD: 0.0001\n NONBACKBONE_KEYWORDS: ['cls', 'aux', 'ocr']\n NONBACKBONE_MULT: 10\n MOMENTUM: 0.9\n NESTEROV: false\n FLIP: true\n MULTI_SCALE: true\n DOWNSAMPLERATE: 1\n IGNORE_LABEL: 255\n SCALE_FACTOR: 16\nTEST:\n IMAGE_SIZE:\n - 520\n - 520\n BASE_SIZE: 520\n BATCH_SIZE_PER_GPU: 1\n NUM_SAMPLES: 200\n FLIP_TEST: false\n MULTI_SCALE: false\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/experiments/cocostuff/seg_hrnet_w48_520x520_ohem_sgd_lr1e-3_wd1e-4_bs_16_epoch110.yaml",
"content": "CUDNN:\n BENCHMARK: true\n DETERMINISTIC: false\n ENABLED: true\nGPUS: (0,1,2,3)\nOUTPUT_DIR: 'output'\nLOG_DIR: 'log'\nWORKERS: 4\nPRINT_FREQ: 10\n\nDATASET:\n DATASET: cocostuff\n ROOT: 'data/'\n TEST_SET: 'list/cocostuff/val.lst'\n TRAIN_SET: 'list/cocostuff/train.lst'\n NUM_CLASSES: 171\nMODEL:\n NAME: seg_hrnet\n NUM_OUTPUTS: 1\n PRETRAINED: 'pretrained_models/hrnetv2_w48_imagenet_pretrained.pth'\n EXTRA:\n FINAL_CONV_KERNEL: 1\n STAGE1:\n NUM_MODULES: 1\n NUM_RANCHES: 1\n BLOCK: BOTTLENECK\n NUM_BLOCKS:\n - 4\n NUM_CHANNELS:\n - 64\n FUSE_METHOD: SUM\n STAGE2:\n NUM_MODULES: 1\n NUM_BRANCHES: 2\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n FUSE_METHOD: SUM\n STAGE3:\n NUM_MODULES: 4\n NUM_BRANCHES: 3\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n FUSE_METHOD: SUM\n STAGE4:\n NUM_MODULES: 3\n NUM_BRANCHES: 4\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n - 384\n FUSE_METHOD: SUM\nLOSS:\n USE_OHEM: true\n OHEMTHRES: 0.9\n OHEMKEEP: 131072\nTRAIN:\n IMAGE_SIZE:\n - 520\n - 520\n BASE_SIZE: 520\n BATCH_SIZE_PER_GPU: 4\n SHUFFLE: true\n BEGIN_EPOCH: 0\n END_EPOCH: 110\n RESUME: true\n OPTIMIZER: sgd\n LR: 0.001\n WD: 0.0001\n NONBACKBONE_KEYWORDS: ['last_layer']\n NONBACKBONE_MULT: 10\n MOMENTUM: 0.9\n NESTEROV: false\n FLIP: true\n MULTI_SCALE: true\n DOWNSAMPLERATE: 1\n IGNORE_LABEL: 255\n SCALE_FACTOR: 16\nTEST:\n IMAGE_SIZE:\n - 520\n - 520\n BASE_SIZE: 520\n BATCH_SIZE_PER_GPU: 1\n NUM_SAMPLES: 200\n FLIP_TEST: false\n MULTI_SCALE: false\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/experiments/cocostuff/seg_hrnet_w48_520x520_sgd_lr1e-3_wd1e-4_bs_16_epoch110.yaml",
"content": "CUDNN:\n BENCHMARK: true\n DETERMINISTIC: false\n ENABLED: true\nGPUS: (0,1,2,3)\nOUTPUT_DIR: 'output'\nLOG_DIR: 'log'\nWORKERS: 4\nPRINT_FREQ: 10\n\nDATASET:\n DATASET: cocostuff\n ROOT: 'data/'\n TEST_SET: 'list/cocostuff/val.lst'\n TRAIN_SET: 'list/cocostuff/train.lst'\n NUM_CLASSES: 171\nMODEL:\n NAME: seg_hrnet\n NUM_OUTPUTS: 1\n PRETRAINED: 'pretrained_models/hrnetv2_w48_imagenet_pretrained.pth'\n EXTRA:\n FINAL_CONV_KERNEL: 1\n STAGE1:\n NUM_MODULES: 1\n NUM_RANCHES: 1\n BLOCK: BOTTLENECK\n NUM_BLOCKS:\n - 4\n NUM_CHANNELS:\n - 64\n FUSE_METHOD: SUM\n STAGE2:\n NUM_MODULES: 1\n NUM_BRANCHES: 2\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n FUSE_METHOD: SUM\n STAGE3:\n NUM_MODULES: 4\n NUM_BRANCHES: 3\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n FUSE_METHOD: SUM\n STAGE4:\n NUM_MODULES: 3\n NUM_BRANCHES: 4\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n - 384\n FUSE_METHOD: SUM\nLOSS:\n USE_OHEM: false\n OHEMTHRES: 0.9\n OHEMKEEP: 131072\nTRAIN:\n IMAGE_SIZE:\n - 520\n - 520\n BASE_SIZE: 520\n BATCH_SIZE_PER_GPU: 4\n SHUFFLE: true\n BEGIN_EPOCH: 0\n END_EPOCH: 110\n RESUME: true\n OPTIMIZER: sgd\n LR: 0.001\n WD: 0.0001\n NONBACKBONE_KEYWORDS: ['last_layer']\n NONBACKBONE_MULT: 10\n MOMENTUM: 0.9\n NESTEROV: false\n FLIP: true\n MULTI_SCALE: true\n DOWNSAMPLERATE: 1\n IGNORE_LABEL: 255\n SCALE_FACTOR: 16\nTEST:\n IMAGE_SIZE:\n - 520\n - 520\n BASE_SIZE: 520\n BATCH_SIZE_PER_GPU: 1\n NUM_SAMPLES: 200\n FLIP_TEST: false\n MULTI_SCALE: false\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/experiments/lip/seg_hrnet_ocr_w48_473x473_sgd_lr7e-3_wd5e-4_bs_40_epoch150.yaml",
"content": "CUDNN:\n BENCHMARK: true\n DETERMINISTIC: false\n ENABLED: true\nGPUS: (0,1,2,3)\nOUTPUT_DIR: 'output'\nLOG_DIR: 'log'\nWORKERS: 4\nPRINT_FREQ: 10\n\nDATASET:\n DATASET: lip\n ROOT: 'data/'\n TEST_SET: 'list/lip/valList.txt'\n TRAIN_SET: 'list/lip/trainList.txt'\n NUM_CLASSES: 20\nMODEL:\n NAME: seg_hrnet_ocr\n NUM_OUTPUTS: 2\n PRETRAINED: 'pretrained_models/hrnetv2_w48_imagenet_pretrained_2.pth'\n EXTRA:\n FINAL_CONV_KERNEL: 1\n STAGE1:\n NUM_MODULES: 1\n NUM_RANCHES: 1\n BLOCK: BOTTLENECK\n NUM_BLOCKS:\n - 4\n NUM_CHANNELS:\n - 64\n FUSE_METHOD: SUM\n STAGE2:\n NUM_MODULES: 1\n NUM_BRANCHES: 2\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n FUSE_METHOD: SUM\n STAGE3:\n NUM_MODULES: 4\n NUM_BRANCHES: 3\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n FUSE_METHOD: SUM\n STAGE4:\n NUM_MODULES: 3\n NUM_BRANCHES: 4\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n - 384\n FUSE_METHOD: SUM\nLOSS:\n USE_OHEM: false\n OHEMTHRES: 0.9\n OHEMKEEP: 131072\n BALANCE_WEIGHTS: [0.4, 1]\nTRAIN:\n IMAGE_SIZE:\n - 473\n - 473\n BASE_SIZE: 473\n BATCH_SIZE_PER_GPU: 10\n SHUFFLE: true\n BEGIN_EPOCH: 0\n END_EPOCH: 150\n RESUME: true\n OPTIMIZER: sgd\n LR: 0.007\n WD: 0.0005\n MOMENTUM: 0.9\n NESTEROV: false\n FLIP: true\n MULTI_SCALE: true\n DOWNSAMPLERATE: 1\n IGNORE_LABEL: 255\n SCALE_FACTOR: 11\nTEST:\n IMAGE_SIZE:\n - 473\n - 473\n BASE_SIZE: 473\n BATCH_SIZE_PER_GPU: 10\n NUM_SAMPLES: 2000\n FLIP_TEST: false\n MULTI_SCALE: false\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/experiments/lip/seg_hrnet_w48_473x473_sgd_lr7e-3_wd5e-4_bs_40_epoch150.yaml",
"content": "CUDNN:\n BENCHMARK: true\n DETERMINISTIC: false\n ENABLED: true\nGPUS: (0,1,2,3)\nOUTPUT_DIR: 'output'\nLOG_DIR: 'log'\nWORKERS: 4\nPRINT_FREQ: 100\n\nDATASET:\n DATASET: lip\n ROOT: 'data/'\n TEST_SET: 'list/lip/valList.txt'\n TRAIN_SET: 'list/lip/trainList.txt'\n NUM_CLASSES: 20\nMODEL:\n NAME: seg_hrnet\n ALIGN_CORNERS: False\n PRETRAINED: 'pretrained_models/hrnetv2_w48_imagenet_pretrained.pth'\n EXTRA:\n FINAL_CONV_KERNEL: 1\n STAGE1:\n NUM_MODULES: 1\n NUM_RANCHES: 1\n BLOCK: BOTTLENECK\n NUM_BLOCKS:\n - 4\n NUM_CHANNELS:\n - 64\n FUSE_METHOD: SUM\n STAGE2:\n NUM_MODULES: 1\n NUM_BRANCHES: 2\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n FUSE_METHOD: SUM\n STAGE3:\n NUM_MODULES: 4\n NUM_BRANCHES: 3\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n FUSE_METHOD: SUM\n STAGE4:\n NUM_MODULES: 3\n NUM_BRANCHES: 4\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n - 384\n FUSE_METHOD: SUM\nLOSS:\n USE_OHEM: false\n OHEMTHRES: 0.9\n OHEMKEEP: 131072\nTRAIN:\n IMAGE_SIZE:\n - 473\n - 473\n BASE_SIZE: 473\n BATCH_SIZE_PER_GPU: 10\n SHUFFLE: true\n BEGIN_EPOCH: 0\n END_EPOCH: 150\n RESUME: true\n OPTIMIZER: sgd\n LR: 0.007\n WD: 0.0005\n MOMENTUM: 0.9\n NESTEROV: false\n FLIP: true\n MULTI_SCALE: true\n DOWNSAMPLERATE: 1\n IGNORE_LABEL: 255\n SCALE_FACTOR: 11\nTEST:\n IMAGE_SIZE:\n - 473\n - 473\n BASE_SIZE: 473\n BATCH_SIZE_PER_GPU: 16\n NUM_SAMPLES: 2000\n FLIP_TEST: false\n MULTI_SCALE: false\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/experiments/pascal_ctx/seg_hrnet_ocr_w48_cls59_520x520_sgd_lr1e-3_wd1e-4_bs_16_epoch200.yaml",
"content": "CUDNN:\n BENCHMARK: true\n DETERMINISTIC: false\n ENABLED: true\nGPUS: (0,1,2,3)\nOUTPUT_DIR: 'output'\nLOG_DIR: 'log'\nWORKERS: 4\nPRINT_FREQ: 10\n\nDATASET:\n DATASET: pascal_ctx\n ROOT: 'data/'\n TEST_SET: 'val'\n TRAIN_SET: 'train'\n NUM_CLASSES: 59\nMODEL:\n NAME: seg_hrnet_ocr\n NUM_OUTPUTS: 2\n PRETRAINED: 'pretrained_models/hrnetv2_w48_imagenet_pretrained.pth'\n EXTRA:\n FINAL_CONV_KERNEL: 1\n STAGE1:\n NUM_MODULES: 1\n NUM_RANCHES: 1\n BLOCK: BOTTLENECK\n NUM_BLOCKS:\n - 4\n NUM_CHANNELS:\n - 64\n FUSE_METHOD: SUM\n STAGE2:\n NUM_MODULES: 1\n NUM_BRANCHES: 2\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n FUSE_METHOD: SUM\n STAGE3:\n NUM_MODULES: 4\n NUM_BRANCHES: 3\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n FUSE_METHOD: SUM\n STAGE4:\n NUM_MODULES: 3\n NUM_BRANCHES: 4\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n - 384\n FUSE_METHOD: SUM\nLOSS:\n USE_OHEM: false\n OHEMTHRES: 0.9\n OHEMKEEP: 131072\n BALANCE_WEIGHTS: [0.4, 1]\nTRAIN:\n IMAGE_SIZE:\n - 520\n - 520\n BASE_SIZE: 520\n BATCH_SIZE_PER_GPU: 4\n NONBACKBONE_KEYWORDS: ['cls', 'aux', 'ocr']\n NONBACKBONE_MULT: 10 \n SHUFFLE: true\n BEGIN_EPOCH: 0\n END_EPOCH: 200\n RESUME: true\n OPTIMIZER: sgd\n LR: 0.001\n WD: 0.0001\n MOMENTUM: 0.9\n NESTEROV: false\n FLIP: true\n MULTI_SCALE: true\n DOWNSAMPLERATE: 1\n IGNORE_LABEL: -1\n SCALE_FACTOR: 16\nTEST:\n IMAGE_SIZE:\n - 520\n - 520\n BASE_SIZE: 520\n BATCH_SIZE_PER_GPU: 16\n FLIP_TEST: false\n MULTI_SCALE: false\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/experiments/pascal_ctx/seg_hrnet_ocr_w48_cls60_520x520_sgd_lr1e-3_wd1e-4_bs_16_epoch200.yaml",
"content": "CUDNN:\n BENCHMARK: true\n DETERMINISTIC: false\n ENABLED: true\nGPUS: (0,1,2,3)\nOUTPUT_DIR: 'output'\nLOG_DIR: 'log'\nWORKERS: 4\nPRINT_FREQ: 10\n\nDATASET:\n DATASET: pascal_ctx\n ROOT: 'data/'\n TEST_SET: 'val'\n TRAIN_SET: 'train'\n NUM_CLASSES: 60\nMODEL:\n NAME: seg_hrnet_ocr\n NUM_OUTPUTS: 2\n PRETRAINED: 'pretrained_models/hrnetv2_w48_imagenet_pretrained.pth'\n EXTRA:\n FINAL_CONV_KERNEL: 1\n STAGE1:\n NUM_MODULES: 1\n NUM_RANCHES: 1\n BLOCK: BOTTLENECK\n NUM_BLOCKS:\n - 4\n NUM_CHANNELS:\n - 64\n FUSE_METHOD: SUM\n STAGE2:\n NUM_MODULES: 1\n NUM_BRANCHES: 2\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n FUSE_METHOD: SUM\n STAGE3:\n NUM_MODULES: 4\n NUM_BRANCHES: 3\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n FUSE_METHOD: SUM\n STAGE4:\n NUM_MODULES: 3\n NUM_BRANCHES: 4\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n - 384\n FUSE_METHOD: SUM\nLOSS:\n USE_OHEM: false\n OHEMTHRES: 0.9\n OHEMKEEP: 131072\n BALANCE_WEIGHTS: [0.4, 1]\nTRAIN:\n IMAGE_SIZE:\n - 520\n - 520\n BASE_SIZE: 520\n BATCH_SIZE_PER_GPU: 4\n NONBACKBONE_KEYWORDS: ['cls', 'aux', 'ocr']\n NONBACKBONE_MULT: 10 \n SHUFFLE: true\n BEGIN_EPOCH: 0\n END_EPOCH: 200\n RESUME: true\n OPTIMIZER: sgd\n LR: 0.001\n WD: 0.0001\n MOMENTUM: 0.9\n NESTEROV: false\n FLIP: true\n MULTI_SCALE: true\n DOWNSAMPLERATE: 1\n IGNORE_LABEL: -1\n SCALE_FACTOR: 16\nTEST:\n IMAGE_SIZE:\n - 520\n - 520\n BASE_SIZE: 520\n BATCH_SIZE_PER_GPU: 16\n FLIP_TEST: false\n MULTI_SCALE: false\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/experiments/pascal_ctx/seg_hrnet_w48_cls59_520x520_sgd_lr1e-3_wd1e-4_bs_16_epoch200.yaml",
"content": "CUDNN:\n BENCHMARK: true\n DETERMINISTIC: false\n ENABLED: true\nGPUS: (0,1,2,3)\nOUTPUT_DIR: 'output'\nLOG_DIR: 'log'\nWORKERS: 4\nPRINT_FREQ: 10\n\nDATASET:\n DATASET: pascal_ctx\n ROOT: 'data/'\n TEST_SET: 'val'\n TRAIN_SET: 'train'\n NUM_CLASSES: 59\nMODEL:\n NAME: seg_hrnet\n ALIGN_CORNERS: False\n NUM_OUTPUTS: 1\n PRETRAINED: 'pretrained_models/hrnetv2_w48_imagenet_pretrained.pth'\n EXTRA:\n FINAL_CONV_KERNEL: 1\n STAGE1:\n NUM_MODULES: 1\n NUM_RANCHES: 1\n BLOCK: BOTTLENECK\n NUM_BLOCKS:\n - 4\n NUM_CHANNELS:\n - 64\n FUSE_METHOD: SUM\n STAGE2:\n NUM_MODULES: 1\n NUM_BRANCHES: 2\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n FUSE_METHOD: SUM\n STAGE3:\n NUM_MODULES: 4\n NUM_BRANCHES: 3\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n FUSE_METHOD: SUM\n STAGE4:\n NUM_MODULES: 3\n NUM_BRANCHES: 4\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n - 384\n FUSE_METHOD: SUM\nLOSS:\n USE_OHEM: false\n OHEMTHRES: 0.9\n OHEMKEEP: 131072\nTRAIN:\n IMAGE_SIZE:\n - 520\n - 520\n BASE_SIZE: 520\n BATCH_SIZE_PER_GPU: 4\n NONBACKBONE_KEYWORDS: ['last_layer']\n NONBACKBONE_MULT: 10 \n SHUFFLE: true\n BEGIN_EPOCH: 0\n END_EPOCH: 200\n RESUME: true\n OPTIMIZER: sgd\n LR: 0.001\n WD: 0.0001\n MOMENTUM: 0.9\n NESTEROV: false\n FLIP: true\n MULTI_SCALE: true\n DOWNSAMPLERATE: 1\n IGNORE_LABEL: -1\n SCALE_FACTOR: 16\nTEST:\n IMAGE_SIZE:\n - 520\n - 520\n BASE_SIZE: 520\n BATCH_SIZE_PER_GPU: 16\n FLIP_TEST: false\n MULTI_SCALE: false\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/experiments/rellis/seg_hrnet_ocr_w48_train_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484.yaml",
"content": "CUDNN:\n BENCHMARK: true\n DETERMINISTIC: false\n ENABLED: true\nGPUS: (0,1)\nOUTPUT_DIR: 'output'\nLOG_DIR: 'log'\nWORKERS: 4\nPRINT_FREQ: 10\n\nDATASET:\n DATASET: rellis\n ROOT: /path/to/rellis-3D/\n TEST_SET: 'val.lst'\n TRAIN_SET: 'train.lst'\n NUM_CLASSES: 19\nMODEL:\n NAME: seg_hrnet_ocr\n NUM_OUTPUTS: 2\n PRETRAINED: \"./pretrained_models/hrnetv2_w48_imagenet_pretrained.pth\"\n EXTRA:\n FINAL_CONV_KERNEL: 1\n STAGE1:\n NUM_MODULES: 1\n NUM_RANCHES: 1\n BLOCK: BOTTLENECK\n NUM_BLOCKS:\n - 4\n NUM_CHANNELS:\n - 64\n FUSE_METHOD: SUM\n STAGE2:\n NUM_MODULES: 1\n NUM_BRANCHES: 2\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n FUSE_METHOD: SUM\n STAGE3:\n NUM_MODULES: 4\n NUM_BRANCHES: 3\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n FUSE_METHOD: SUM\n STAGE4:\n NUM_MODULES: 3\n NUM_BRANCHES: 4\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n - 384\n FUSE_METHOD: SUM\nLOSS:\n USE_OHEM: false\n OHEMTHRES: 0.9\n OHEMKEEP: 131072\n BALANCE_WEIGHTS: [0.4, 1]\nTRAIN:\n IMAGE_SIZE:\n - 1024\n - 640\n BASE_SIZE: 1920\n BATCH_SIZE_PER_GPU: 4\n SHUFFLE: true\n BEGIN_EPOCH: 0\n END_EPOCH: 484\n RESUME: true\n OPTIMIZER: sgd\n LR: 0.01\n WD: 0.0005\n MOMENTUM: 0.9\n NESTEROV: false\n FLIP: true\n MULTI_SCALE: true\n DOWNSAMPLERATE: 1\n IGNORE_LABEL: 255\n SCALE_FACTOR: 16\nTEST:\n IMAGE_SIZE:\n - 1920\n - 1200\n BASE_SIZE: 1920\n BATCH_SIZE_PER_GPU: 4\n FLIP_TEST: false\n MULTI_SCALE: false\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/experiments/rellis/seg_hrnet_ocr_w48_train_512x1024_sgd_lr1e-3_wd5e-4_bs_12_epoch484.yaml",
"content": "CUDNN:\n BENCHMARK: true\n DETERMINISTIC: false\n ENABLED: true\nGPUS: (0,1)\nOUTPUT_DIR: 'output'\nLOG_DIR: 'log'\nWORKERS: 4\nPRINT_FREQ: 10\n\nDATASET:\n DATASET: rellis\n ROOT: /home/usl/Datasets/rellis/\n TEST_SET: 'val.lst'\n TRAIN_SET: 'train.lst'\n NUM_CLASSES: 19\nMODEL:\n NAME: seg_hrnet_ocr\n NUM_OUTPUTS: 2\n PRETRAINED: \"/path/to/HRNet-Semantic-Segmentation-HRNet-OCR/pretrained_models/hrnetv2_w48_imagenet_pretrained.pth\"\n EXTRA:\n FINAL_CONV_KERNEL: 1\n STAGE1:\n NUM_MODULES: 1\n NUM_RANCHES: 1\n BLOCK: BOTTLENECK\n NUM_BLOCKS:\n - 4\n NUM_CHANNELS:\n - 64\n FUSE_METHOD: SUM\n STAGE2:\n NUM_MODULES: 1\n NUM_BRANCHES: 2\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n FUSE_METHOD: SUM\n STAGE3:\n NUM_MODULES: 4\n NUM_BRANCHES: 3\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n FUSE_METHOD: SUM\n STAGE4:\n NUM_MODULES: 3\n NUM_BRANCHES: 4\n BLOCK: BASIC\n NUM_BLOCKS:\n - 4\n - 4\n - 4\n - 4\n NUM_CHANNELS:\n - 48\n - 96\n - 192\n - 384\n FUSE_METHOD: SUM\nLOSS:\n USE_OHEM: false\n OHEMTHRES: 0.9\n OHEMKEEP: 131072\n BALANCE_WEIGHTS: [0.4, 1]\nTRAIN:\n IMAGE_SIZE:\n - 1024\n - 640\n BASE_SIZE: 1920\n BATCH_SIZE_PER_GPU: 4\n SHUFFLE: true\n BEGIN_EPOCH: 0\n END_EPOCH: 484\n RESUME: true\n OPTIMIZER: sgd\n LR: 0.001\n WD: 0.0005\n MOMENTUM: 0.9\n NESTEROV: false\n FLIP: true\n MULTI_SCALE: true\n DOWNSAMPLERATE: 1\n IGNORE_LABEL: 255\n SCALE_FACTOR: 16\nTEST:\n IMAGE_SIZE:\n - 1920\n - 1200\n BASE_SIZE: 1920\n BATCH_SIZE_PER_GPU: 4\n FLIP_TEST: false\n MULTI_SCALE: false\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/hubconf.py",
"content": "\"\"\"File for accessing HRNet via PyTorch Hub https://pytorch.org/hub/\n\nUsage:\n import torch\n model = torch.hub.load('AlexeyAB/PyTorch_YOLOv4:u5_preview', 'yolov4_pacsp_s', pretrained=True, channels=3, classes=80)\n\"\"\"\n\ndependencies = ['torch']\nimport torch\nfrom lib.models.seg_hrnet import get_seg_model\n\n\nstate_dict_url = 'https://github.com/huawei-noah/ghostnet/raw/master/pytorch/models/state_dict_93.98.pth'\n\n\ndef hrnet_w48_cityscapes(pretrained=False, **kwargs):\n\t \"\"\" # This docstring shows up in hub.help()\n HRNetW48 model pretrained on Cityscapes\n pretrained (bool): kwargs, load pretrained weights into the model\n \"\"\"\n\t model = ghostnet(num_classes=1000, width=1.0, dropout=0.2)\n\t if pretrained:\n\t \t state_dict = torch.hub.load_state_dict_from_url(state_dict_url, progress=True)\n\t \t model.load_state_dict(state_dict)\n\t return model"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/pretrained_models/.gitignore",
"content": "*.pth"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/requirements.txt",
"content": "EasyDict==1.7\nopencv-python\nshapely==1.6.4\nCython\nscipy\npandas\npyyaml\njson_tricks\nscikit-image\nyacs>=0.1.5\ntensorboardX>=1.6\ntqdm\nninja\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/run_hrnet.sh",
"content": "#!/bin/bash\nexport PYTHONPATH=/home/usl/Code/Peng/data_collection/benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/:$PYTHONPATH\necho $PYTHONPATH\nPY_CMD=\"python -m torch.distributed.launch --nproc_per_node=2\"\n$PY_CMD tools/train.py --cfg experiments/rellis/seg_hrnet_ocr_w48_train_512x1024_sgd_lr1e-3_wd5e-4_bs_12_epoch484.yaml\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/run_hrnet_test.sh",
"content": "#!/bin/bash\nexport PYTHONPATH=/home/usl/Code/PengJiang/RELLIS-3D/benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/:$PYTHONPATH\npython tools/test.py --cfg experiments/rellis/seg_hrnet_ocr_w48_train_512x1024_sgd_lr1e-3_wd5e-4_bs_12_epoch484.yaml \\\n --data-cfg /path/to/benchmarks/SalsaNext/train/tasks/semantic/config/labels/rellis.yaml \\\n DATASET.TEST_SET test.lst \\\n OUTPUT_DIR /path/to/save \\\n TEST.MODEL_FILE /path/to/hrnet_best/seg_hrnet_ocr_w48_train_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484/best.pth\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/tools/_init_paths.py",
"content": "# ------------------------------------------------------------------------------\n# Copyright (c) Microsoft\n# Licensed under the MIT License.\n# Written by Ke Sun (sunk@mail.ustc.edu.cn)\n# ------------------------------------------------------------------------------\n\nfrom __future__ import absolute_import\nfrom __future__ import division\nfrom __future__ import print_function\n\nimport os.path as osp\nimport sys\n\n\ndef add_path(path):\n if path not in sys.path:\n sys.path.insert(0, path)\n\nthis_dir = osp.dirname(__file__)\n\nlib_path = osp.join(this_dir, '..', 'lib')\nadd_path(lib_path)\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/tools/test.py",
"content": "# ------------------------------------------------------------------------------\n# Copyright (c) Microsoft\n# Licensed under the MIT License.\n# Written by Ke Sun (sunk@mail.ustc.edu.cn)\n# ------------------------------------------------------------------------------\n\nimport argparse\nimport os\nimport pprint\nimport shutil\nimport sys\n\nimport logging\nimport time\nimport timeit\nimport yaml\nfrom pathlib import Path\n\nimport numpy as np\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\n\nimport _init_paths\nimport models\nimport datasets\nfrom config import config\nfrom config import update_config\nfrom core.function import testval, test\nfrom utils.modelsummary import get_model_summary\nfrom utils.utils import create_logger, FullModel\n\ndef parse_args():\n parser = argparse.ArgumentParser(description='Train segmentation network')\n \n parser.add_argument('--cfg',\n help='experiment configure file name',\n required=True,\n type=str)\n parser.add_argument('--save', dest='save',\n help=\"Save predictions to disk\",\n action='store_true')\n parser.add_argument('--data-cfg', help='data config (kitti format)',\n default='config/rellis.yaml',\n type=str)\n parser.add_argument('opts',\n help=\"Modify config options using the command-line\",\n default=None,\n nargs=argparse.REMAINDER)\n args = parser.parse_args()\n update_config(config, args)\n\n return args\n\ndef main():\n args = parse_args()\n\n logger, final_output_dir, _ = create_logger(\n config, args.cfg, 'test')\n\n logger.info(pprint.pformat(args))\n logger.info(pprint.pformat(config))\n\n # cudnn related setting\n cudnn.benchmark = config.CUDNN.BENCHMARK\n cudnn.deterministic = config.CUDNN.DETERMINISTIC\n cudnn.enabled = config.CUDNN.ENABLED\n\n # build model\n if torch.__version__.startswith('1'):\n module = eval('models.'+config.MODEL.NAME)\n module.BatchNorm2d_class = module.BatchNorm2d = torch.nn.BatchNorm2d\n model = eval('models.'+config.MODEL.NAME +\n '.get_seg_model')(config)\n\n dump_input = torch.rand(\n (1, 3, config.TRAIN.IMAGE_SIZE[1], config.TRAIN.IMAGE_SIZE[0])\n )\n logger.info(get_model_summary(model.cuda(), dump_input.cuda()))\n\n if config.TEST.MODEL_FILE:\n model_state_file = config.TEST.MODEL_FILE\n else:\n model_state_file = os.path.join(final_output_dir, 'final_state.pth') \n logger.info('=> loading model from {}'.format(model_state_file))\n \n pretrained_dict = torch.load(model_state_file)\n if 'state_dict' in pretrained_dict:\n pretrained_dict = pretrained_dict['state_dict']\n model_dict = model.state_dict()\n pretrained_dict = {k[6:]: v for k, v in pretrained_dict.items()\n if k[6:] in model_dict.keys()}\n for k, _ in pretrained_dict.items():\n logger.info(\n '=> loading {} from pretrained model'.format(k))\n model_dict.update(pretrained_dict)\n model.load_state_dict(model_dict)\n\n gpus = list(config.GPUS)\n print('GPUS:',gpus)\n model = nn.DataParallel(model, device_ids=gpus).cuda()\n\n # prepare data\n test_size = (config.TEST.IMAGE_SIZE[1], config.TEST.IMAGE_SIZE[0])\n test_dataset = eval('datasets.'+config.DATASET.DATASET)(\n root=config.DATASET.ROOT,\n list_path=config.DATASET.TEST_SET,\n num_samples=None,\n num_classes=config.DATASET.NUM_CLASSES,\n multi_scale=False,\n flip=False,\n ignore_label=config.TRAIN.IGNORE_LABEL,\n base_size=config.TEST.BASE_SIZE,\n crop_size=test_size,\n downsample_rate=1)\n\n testloader = torch.utils.data.DataLoader(\n test_dataset,\n batch_size=1,\n shuffle=False,\n num_workers=config.WORKERS,\n pin_memory=True)\n \n try:\n print(\"Opening config file %s\" % args.data_cfg)\n CFG = yaml.safe_load(open(args.data_cfg, 'r'))\n except Exception as e:\n print(e)\n print(\"Error opening yaml file.\")\n quit()\n\n id_color_map = CFG[\"color_map\"]\n start = timeit.default_timer()\n if 'val' in config.DATASET.TEST_SET:\n mean_IoU, IoU_array, pixel_acc, mean_acc = testval(config, \n test_dataset, \n testloader, \n model,\n sv_dir='test_output',\n sv_pred=args.save,\n id_color_map = id_color_map)\n \n msg = 'MeanIU: {: 4.4f}, Pixel_Acc: {: 4.4f}, \\\n Mean_Acc: {: 4.4f}, Class IoU: '.format(mean_IoU, \n pixel_acc, mean_acc)\n logging.info(msg)\n logging.info(IoU_array)\n elif 'test' in config.DATASET.TEST_SET:\n test(config, \n test_dataset, \n testloader, \n model,\n sv_dir=final_output_dir,\n sv_pred=args.save,\n id_color_map = id_color_map)\n\n end = timeit.default_timer()\n logger.info('Mins: %d' % np.int((end-start)/60))\n logger.info('Done')\n\n\nif __name__ == '__main__':\n main()\n"
},
{
"path": "benchmarks/HRNet-Semantic-Segmentation-HRNet-OCR/tools/train.py",
"content": "# ------------------------------------------------------------------------------\n# Copyright (c) Microsoft\n# Licensed under the MIT License.\n# Written by Ke Sun (sunk@mail.ustc.edu.cn)\n# ------------------------------------------------------------------------------\n\nimport argparse\nimport os\nimport pprint\nimport shutil\nimport sys\n\nimport logging\nimport time\nimport timeit\nfrom pathlib import Path\n\nimport numpy as np\n\nimport torch\nimport torch.nn as nn\nimport torch.backends.cudnn as cudnn\nimport torch.optim\nfrom tensorboardX import SummaryWriter\n\nimport _init_paths\nimport models\nimport datasets\nfrom config import config\nfrom config import update_config\nfrom core.criterion import CrossEntropy, OhemCrossEntropy\nfrom core.function import train, validate\nfrom utils.modelsummary import get_model_summary\nfrom utils.utils import create_logger, FullModel\n\ndef parse_args():\n parser = argparse.ArgumentParser(description='Train segmentation network')\n \n parser.add_argument('--cfg',\n help='experiment configure file name',\n required=True,\n type=str)\n parser.add_argument('--seed', type=int, default=304)\n parser.add_argument(\"--local_rank\", type=int, default=-1) \n parser.add_argument('opts',\n help=\"Modify config options using the command-line\",\n default=None,\n nargs=argparse.REMAINDER)\n\n args = parser.parse_args()\n update_config(config, args)\n\n return args\n\ndef get_sampler(dataset):\n from utils.distributed import is_distributed\n if is_distributed():\n from torch.utils.data.distributed import DistributedSampler\n return DistributedSampler(dataset)\n else:\n return None\n\ndef main():\n args = parse_args()\n\n if args.seed > 0:\n import random\n print('Seeding with', args.seed)\n random.seed(args.seed)\n torch.manual_seed(args.seed) \n\n logger, final_output_dir, tb_log_dir = create_logger(\n config, args.cfg, 'train')\n\n logger.info(pprint.pformat(args))\n logger.info(config)\n\n writer_dict = {\n 'writer': SummaryWriter(tb_log_dir),\n 'train_global_steps': 0,\n 'valid_global_steps': 0,\n }\n\n # cudnn related setting\n cudnn.benchmark = config.CUDNN.BENCHMARK\n cudnn.deterministic = config.CUDNN.DETERMINISTIC\n cudnn.enabled = config.CUDNN.ENABLED\n gpus = list(config.GPUS)\n distributed = args.local_rank >= 0\n if distributed:\n device = torch.device('cuda:{}'.format(args.local_rank)) \n print(device)\n torch.cuda.set_device(device)\n torch.distributed.init_process_group(\n backend=\"nccl\", init_method=\"env://\",\n ) \n\n # build model\n model = eval('models.'+config.MODEL.NAME +\n '.get_seg_model')(config)\n\n # dump_input = torch.rand(\n # (1, 3, config.TRAIN.IMAGE_SIZE[1], config.TRAIN.IMAGE_SIZE[0])\n # )\n # logger.info(get_model_summary(model.cuda(), dump_input.cuda()))\n\n # copy model file\n if distributed and args.local_rank == 0:\n this_dir = os.path.dirname(__file__)\n models_dst_dir = os.path.join(final_output_dir, 'models')\n if os.path.exists(models_dst_dir):\n shutil.rmtree(models_dst_dir)\n shutil.copytree(os.path.join(this_dir, '../lib/models'), models_dst_dir)\n\n if distributed:\n batch_size = config.TRAIN.BATCH_SIZE_PER_GPU\n else:\n batch_size = config.TRAIN.BATCH_SIZE_PER_GPU * len(gpus)\n\n # prepare data\n crop_size = (config.TRAIN.IMAGE_SIZE[1], config.TRAIN.IMAGE_SIZE[0])\n train_dataset = eval('datasets.'+config.DATASET.DATASET)(\n root=config.DATASET.ROOT,\n list_path=config.DATASET.TRAIN_SET,\n num_samples=None,\n num_classes=config.DATASET.NUM_CLASSES,\n multi_scale=config.TRAIN.MULTI_SCALE,\n flip=config.TRAIN.FLIP,\n ignore_label=config.TRAIN.IGNORE_LABEL,\n base_size=config.TRAIN.BASE_SIZE,\n crop_size=crop_size,\n downsample_rate=config.TRAIN.DOWNSAMPLERATE,\n scale_factor=config.TRAIN.SCALE_FACTOR)\n\n train_sampler = get_sampler(train_dataset)\n trainloader = torch.utils.data.DataLoader(\n train_dataset,\n batch_size=batch_size,\n shuffle=config.TRAIN.SHUFFLE and train_sampler is None,\n num_workers=config.WORKERS,\n pin_memory=True,\n drop_last=True,\n sampler=train_sampler)\n\n extra_epoch_iters = 0\n if config.DATASET.EXTRA_TRAIN_SET:\n extra_train_dataset = eval('datasets.'+config.DATASET.DATASET)(\n root=config.DATASET.ROOT,\n list_path=config.DATASET.EXTRA_TRAIN_SET,\n num_samples=None,\n num_classes=config.DATASET.NUM_CLASSES,\n multi_scale=config.TRAIN.MULTI_SCALE,\n flip=config.TRAIN.FLIP,\n ignore_label=config.TRAIN.IGNORE_LABEL,\n base_size=config.TRAIN.BASE_SIZE,\n crop_size=crop_size,\n downsample_rate=config.TRAIN.DOWNSAMPLERATE,\n scale_factor=config.TRAIN.SCALE_FACTOR)\n extra_train_sampler = get_sampler(extra_train_dataset)\n extra_trainloader = torch.utils.data.DataLoader(\n extra_train_dataset,\n batch_size=batch_size,\n shuffle=config.TRAIN.SHUFFLE and extra_train_sampler is None,\n num_workers=config.WORKERS,\n pin_memory=True,\n drop_last=True,\n sampler=extra_train_sampler)\n extra_epoch_iters = np.int(extra_train_dataset.__len__() / \n config.TRAIN.BATCH_SIZE_PER_GPU / len(gpus))\n\n\n test_size = (config.TEST.IMAGE_SIZE[1], config.TEST.IMAGE_SIZE[0])\n test_dataset = eval('datasets.'+config.DATASET.DATASET)(\n root=config.DATASET.ROOT,\n list_path=config.DATASET.TEST_SET,\n num_samples=config.TEST.NUM_SAMPLES,\n num_classes=config.DATASET.NUM_CLASSES,\n multi_scale=False,\n flip=False,\n ignore_label=config.TRAIN.IGNORE_LABEL,\n base_size=config.TEST.BASE_SIZE,\n crop_size=test_size,\n downsample_rate=1)\n\n test_sampler = get_sampler(test_dataset)\n testloader = torch.utils.data.DataLoader(\n test_dataset,\n batch_size=batch_size,\n shuffle=False,\n num_workers=config.WORKERS,\n pin_memory=True,\n sampler=test_sampler)\n\n # criterion\n if config.LOSS.USE_OHEM:\n criterion = OhemCrossEntropy(ignore_label=config.TRAIN.IGNORE_LABEL,\n thres=config.LOSS.OHEMTHRES,\n min_kept=config.LOSS.OHEMKEEP,\n weight=train_dataset.class_weights)\n else:\n criterion = CrossEntropy(ignore_label=config.TRAIN.IGNORE_LABEL,\n weight=train_dataset.class_weights)\n\n model = FullModel(model, criterion)\n if distributed:\n model = model.to(device)\n model = torch.nn.parallel.DistributedDataParallel(\n model,\n find_unused_parameters=True,\n device_ids=[args.local_rank],\n output_device=args.local_rank\n )\n else:\n model = nn.DataParallel(model, device_ids=gpus).cuda()\n \n\n # optimizer\n if config.TRAIN.OPTIMIZER == 'sgd':\n\n params_dict = dict(model.named_parameters())\n if config.TRAIN.NONBACKBONE_KEYWORDS:\n bb_lr = []\n nbb_lr = []\n nbb_keys = set()\n for k, param in params_dict.items():\n if any(part in k for part in config.TRAIN.NONBACKBONE_KEYWORDS):\n nbb_lr.append(param)\n nbb_keys.add(k)\n else:\n bb_lr.append(param)\n print(nbb_keys)\n params = [{'params': bb_lr, 'lr': config.TRAIN.LR}, {'params': nbb_lr, 'lr': config.TRAIN.LR * config.TRAIN.NONBACKBONE_MULT}]\n else:\n params = [{'params': list(params_dict.values()), 'lr': config.TRAIN.LR}]\n\n optimizer = torch.optim.SGD(params,\n lr=config.TRAIN.LR,\n momentum=config.TRAIN.MOMENTUM,\n weight_decay=config.TRAIN.WD,\n nesterov=config.TRAIN.NESTEROV,\n )\n else:\n raise ValueError('Only Support SGD optimizer')\n\n epoch_iters = np.int(train_dataset.__len__() / \n config.TRAIN.BATCH_SIZE_PER_GPU / len(gpus))\n \n best_mIoU = 0\n last_epoch = 0\n if config.TRAIN.RESUME:\n model_state_file = os.path.join(final_output_dir,\n 'checkpoint.pth.tar')\n if os.path.isfile(model_state_file):\n checkpoint = torch.load(model_state_file, map_location={'cuda:0': 'cpu'})\n best_mIoU = checkpoint['best_mIoU']\n last_epoch = checkpoint['epoch']\n dct = checkpoint['state_dict']\n \n model.module.model.load_state_dict({k.replace('model.', ''): v for k, v in checkpoint['state_dict'].items() if k.startswith('model.')})\n optimizer.load_state_dict(checkpoint['optimizer'])\n logger.info(\"=> loaded checkpoint (epoch {})\"\n .format(checkpoint['epoch']))\n if distributed:\n torch.distributed.barrier()\n\n start = timeit.default_timer()\n end_epoch = config.TRAIN.END_EPOCH + config.TRAIN.EXTRA_EPOCH\n num_iters = config.TRAIN.END_EPOCH * epoch_iters\n extra_iters = config.TRAIN.EXTRA_EPOCH * extra_epoch_iters\n \n for epoch in range(last_epoch, end_epoch):\n\n current_trainloader = extra_trainloader if epoch >= config.TRAIN.END_EPOCH else trainloader\n if current_trainloader.sampler is not None and hasattr(current_trainloader.sampler, 'set_epoch'):\n current_trainloader.sampler.set_epoch(epoch)\n\n # valid_loss, mean_IoU, IoU_array = validate(config, \n # testloader, model, writer_dict)\n\n if epoch >= config.TRAIN.END_EPOCH:\n train(config, epoch-config.TRAIN.END_EPOCH, \n config.TRAIN.EXTRA_EPOCH, extra_epoch_iters, \n config.TRAIN.EXTRA_LR, extra_iters, \n extra_trainloader, optimizer, model, writer_dict)\n else:\n train(config, epoch, config.TRAIN.END_EPOCH, \n epoch_iters, config.TRAIN.LR, num_iters,\n trainloader, optimizer, model, writer_dict)\n\n valid_loss, mean_IoU, IoU_array = validate(config, \n testloader, model, writer_dict)\n\n if args.local_rank <= 0:\n logger.info('=> saving checkpoint to {}'.format(\n final_output_dir + 'checkpoint.pth.tar'))\n torch.save({\n 'epoch': epoch+1,\n 'best_mIoU': best_mIoU,\n 'state_dict': model.module.state_dict(),\n 'optimizer': optimizer.state_dict(),\n }, os.path.join(final_output_dir,'checkpoint.pth.tar'))\n if mean_IoU > best_mIoU:\n best_mIoU = mean_IoU\n torch.save(model.module.state_dict(),\n os.path.join(final_output_dir, 'best.pth'))\n msg = 'Loss: {:.3f}, MeanIU: {: 4.4f}, Best_mIoU: {: 4.4f}'.format(\n valid_loss, mean_IoU, best_mIoU)\n logging.info(msg)\n logging.info(IoU_array)\n\n if args.local_rank <= 0:\n\n torch.save(model.module.state_dict(),\n os.path.join(final_output_dir, 'final_state.pth'))\n\n writer_dict['writer'].close()\n end = timeit.default_timer()\n logger.info('Hours: %d' % np.int((end-start)/3600))\n logger.info('Done')\n\n\nif __name__ == '__main__':\n main()\n"
},
{
"path": "benchmarks/KPConv-PyTorch-master/.gitignore",
"content": "results/\ntest/\n/cpp_wrappers/cpp_neighbors\n/cpp_wrappers/cpp_subsampling\n__pycache__"
},
{
"path": "benchmarks/KPConv-PyTorch-master/INSTALL.md",
"content": "\n# Installation instructions\n\n## Ubuntu 18.04\n \n* Make sure CUDA and cuDNN are installed. One configuration has been tested: \n - PyTorch 1.4.0, CUDA 10.1 and cuDNN 7.6\n \n* Ensure all python packages are installed :\n\n sudo apt update\n sudo apt install python3-dev python3-pip python3-tk\n\n* Follow PyTorch installation procedure .\n\n* Install the other dependencies with pip:\n - numpy\n - scikit-learn\n - PyYAML\n - matplotlib (for visualization)\n - mayavi (for visualization)\n - PyQt5 (for visualization)\n \n* Compile the C++ extension modules for python located in `cpp_wrappers`. Open a terminal in this folder, and run:\n\n sh compile_wrappers.sh\n\nYou should now be able to train Kernel-Point Convolution models\n\n## Windows 10\n \n* Make sure CUDA and cuDNN are installed. One configuration has been tested: \n - PyTorch 1.4.0, CUDA 10.1 and cuDNN 7.5\n \n* Follow PyTorch installation procedure .\n \n* We used the PyCharm IDE to pip install all python dependencies (including PyTorch) in a venv:\n - torch\n - torchvision\n - numpy\n - scikit-learn\n - PyYAML\n - matplotlib (for visualization)\n - mayavi (for visualization)\n - PyQt5 (for visualization)\n \n* Compile the C++ extension modules for python located in `cpp_wrappers`. You just have to execute two .bat files:\n\n cpp_wrappers/cpp_neighbors/build.bat\n \n and\n \n cpp_wrappers/cpp_subsampling/build.bat\n \nYou should now be able to train Kernel-Point Convolution models\n\n"
},
{
"path": "benchmarks/KPConv-PyTorch-master/README.md",
"content": "\n\n\nCreated by Hugues THOMAS\n\n## Introduction\n\nThis repository contains the implementation of **Kernel Point Convolution** (KPConv) in [PyTorch](https://pytorch.org/).\n\nKPConv is also available in [Tensorflow](https://github.com/HuguesTHOMAS/KPConv) (original but older implementation).\n\nAnother implementation of KPConv is available in [PyTorch-Points-3D](https://github.com/nicolas-chaulet/torch-points3d)\n \nKPConv is a point convolution operator presented in our ICCV2019 paper ([arXiv](https://arxiv.org/abs/1904.08889)). If you find our work useful in your \nresearch, please consider citing:\n\n```\n@article{thomas2019KPConv,\n Author = {Thomas, Hugues and Qi, Charles R. and Deschaud, Jean-Emmanuel and Marcotegui, Beatriz and Goulette, Fran{\\c{c}}ois and Guibas, Leonidas J.},\n Title = {KPConv: Flexible and Deformable Convolution for Point Clouds},\n Journal = {Proceedings of the IEEE International Conference on Computer Vision},\n Year = {2019}\n}\n```\n\n## Installation\n\nThis implementation has been tested on Ubuntu 18.04 and Windows 10. Details are provided in [INSTALL.md](./INSTALL.md).\n\n\n## Experiments\n\nWe provide scripts for three experiments: ModelNet40, S3DIS and SemanticKitti. The instructions to run these \nexperiments are in the [doc](./doc) folder.\n\n* [Object Classification](./doc/object_classification_guide.md): Instructions to train KP-CNN on an object classification\n task (Modelnet40).\n \n* [Scene Segmentation](./doc/scene_segmentation_guide.md): Instructions to train KP-FCNN on a scene segmentation \n task (S3DIS).\n \n* [SLAM Segmentation](./doc/slam_segmentation_guide.md): Instructions to train KP-FCNN on a slam segmentation \n task (SemanticKitti).\n \n* [Pretrained models](./doc/pretrained_models_guide.md): We provide pretrained weights and instructions to load them.\n \n* [Visualization scripts](./doc/visualization_guide.md): For now only one visualization script has been implemented: \nthe kernel deformations display.\n\n## Acknowledgment\n\nOur code uses the nanoflann library.\n\n## License\nOur code is released under MIT License (see LICENSE file for details).\n\n## Updates\n* 27/04/2020: Initial release.\n"
},
{
"path": "benchmarks/KPConv-PyTorch-master/cpp_wrappers/compile_wrappers.sh",
"content": "#!/bin/bash\n\n# Compile cpp subsampling\ncd cpp_subsampling\npython3 setup.py build_ext --inplace\ncd ..\n\n# Compile cpp neighbors\ncd cpp_neighbors\npython3 setup.py build_ext --inplace\ncd .."
},
{
"path": "benchmarks/KPConv-PyTorch-master/cpp_wrappers/cpp_neighbors/build.bat",
"content": "@echo off\npy setup.py build_ext --inplace\n\n\npause"
},
{
"path": "benchmarks/KPConv-PyTorch-master/cpp_wrappers/cpp_neighbors/neighbors/neighbors.cpp",
"content": "\n#include \"neighbors.h\"\n\n\nvoid brute_neighbors(vector& queries, vector& supports, vector& neighbors_indices, float radius, int verbose)\n{\n\n\t// Initialize variables\n\t// ******************\n\n\t// square radius\n\tfloat r2 = radius * radius;\n\n\t// indices\n\tint i0 = 0;\n\n\t// Counting vector\n\tint max_count = 0;\n\tvector> tmp(queries.size());\n\n\t// Search neigbors indices\n\t// ***********************\n\n\tfor (auto& p0 : queries)\n\t{\n\t\tint i = 0;\n\t\tfor (auto& p : supports)\n\t\t{\n\t\t\tif ((p0 - p).sq_norm() < r2)\n\t\t\t{\n\t\t\t\ttmp[i0].push_back(i);\n\t\t\t\tif (tmp[i0].size() > max_count)\n\t\t\t\t\tmax_count = tmp[i0].size();\n\t\t\t}\n\t\t\ti++;\n\t\t}\n\t\ti0++;\n\t}\n\n\t// Reserve the memory\n\tneighbors_indices.resize(queries.size() * max_count);\n\ti0 = 0;\n\tfor (auto& inds : tmp)\n\t{\n\t\tfor (int j = 0; j < max_count; j++)\n\t\t{\n\t\t\tif (j < inds.size())\n\t\t\t\tneighbors_indices[i0 * max_count + j] = inds[j];\n\t\t\telse\n\t\t\t\tneighbors_indices[i0 * max_count + j] = -1;\n\t\t}\n\t\ti0++;\n\t}\n\n\treturn;\n}\n\nvoid ordered_neighbors(vector& queries,\n vector& supports,\n vector& neighbors_indices,\n float radius)\n{\n\n\t// Initialize variables\n\t// ******************\n\n\t// square radius\n\tfloat r2 = radius * radius;\n\n\t// indices\n\tint i0 = 0;\n\n\t// Counting vector\n\tint max_count = 0;\n\tfloat d2;\n\tvector> tmp(queries.size());\n\tvector> dists(queries.size());\n\n\t// Search neigbors indices\n\t// ***********************\n\n\tfor (auto& p0 : queries)\n\t{\n\t\tint i = 0;\n\t\tfor (auto& p : supports)\n\t\t{\n\t\t d2 = (p0 - p).sq_norm();\n\t\t\tif (d2 < r2)\n\t\t\t{\n\t\t\t // Find order of the new point\n\t\t\t auto it = std::upper_bound(dists[i0].begin(), dists[i0].end(), d2);\n\t\t\t int index = std::distance(dists[i0].begin(), it);\n\n\t\t\t // Insert element\n dists[i0].insert(it, d2);\n tmp[i0].insert(tmp[i0].begin() + index, i);\n\n\t\t\t // Update max count\n\t\t\t\tif (tmp[i0].size() > max_count)\n\t\t\t\t\tmax_count = tmp[i0].size();\n\t\t\t}\n\t\t\ti++;\n\t\t}\n\t\ti0++;\n\t}\n\n\t// Reserve the memory\n\tneighbors_indices.resize(queries.size() * max_count);\n\ti0 = 0;\n\tfor (auto& inds : tmp)\n\t{\n\t\tfor (int j = 0; j < max_count; j++)\n\t\t{\n\t\t\tif (j < inds.size())\n\t\t\t\tneighbors_indices[i0 * max_count + j] = inds[j];\n\t\t\telse\n\t\t\t\tneighbors_indices[i0 * max_count + j] = -1;\n\t\t}\n\t\ti0++;\n\t}\n\n\treturn;\n}\n\nvoid batch_ordered_neighbors(vector& queries,\n vector& supports,\n vector& q_batches,\n vector& s_batches,\n vector& neighbors_indices,\n float radius)\n{\n\n\t// Initialize variables\n\t// ******************\n\n\t// square radius\n\tfloat r2 = radius * radius;\n\n\t// indices\n\tint i0 = 0;\n\n\t// Counting vector\n\tint max_count = 0;\n\tfloat d2;\n\tvector> tmp(queries.size());\n\tvector> dists(queries.size());\n\n\t// batch index\n\tint b = 0;\n\tint sum_qb = 0;\n\tint sum_sb = 0;\n\n\n\t// Search neigbors indices\n\t// ***********************\n\n\tfor (auto& p0 : queries)\n\t{\n\t // Check if we changed batch\n\t if (i0 == sum_qb + q_batches[b])\n\t {\n\t sum_qb += q_batches[b];\n\t sum_sb += s_batches[b];\n\t b++;\n\t }\n\n\t // Loop only over the supports of current batch\n\t vector::iterator p_it;\n\t\tint i = 0;\n for(p_it = supports.begin() + sum_sb; p_it < supports.begin() + sum_sb + s_batches[b]; p_it++ )\n {\n\t\t d2 = (p0 - *p_it).sq_norm();\n\t\t\tif (d2 < r2)\n\t\t\t{\n\t\t\t // Find order of the new point\n\t\t\t auto it = std::upper_bound(dists[i0].begin(), dists[i0].end(), d2);\n\t\t\t int index = std::distance(dists[i0].begin(), it);\n\n\t\t\t // Insert element\n dists[i0].insert(it, d2);\n tmp[i0].insert(tmp[i0].begin() + index, sum_sb + i);\n\n\t\t\t // Update max count\n\t\t\t\tif (tmp[i0].size() > max_count)\n\t\t\t\t\tmax_count = tmp[i0].size();\n\t\t\t}\n\t\t\ti++;\n\t\t}\n\t\ti0++;\n\t}\n\n\t// Reserve the memory\n\tneighbors_indices.resize(queries.size() * max_count);\n\ti0 = 0;\n\tfor (auto& inds : tmp)\n\t{\n\t\tfor (int j = 0; j < max_count; j++)\n\t\t{\n\t\t\tif (j < inds.size())\n\t\t\t\tneighbors_indices[i0 * max_count + j] = inds[j];\n\t\t\telse\n\t\t\t\tneighbors_indices[i0 * max_count + j] = supports.size();\n\t\t}\n\t\ti0++;\n\t}\n\n\treturn;\n}\n\n\nvoid batch_nanoflann_neighbors(vector& queries,\n vector& supports,\n vector& q_batches,\n vector& s_batches,\n vector& neighbors_indices,\n float radius)\n{\n\n\t// Initialize variables\n\t// ******************\n\n\t// indices\n\tint i0 = 0;\n\n\t// Square radius\n\tfloat r2 = radius * radius;\n\n\t// Counting vector\n\tint max_count = 0;\n\tfloat d2;\n\tvector>> all_inds_dists(queries.size());\n\n\t// batch index\n\tint b = 0;\n\tint sum_qb = 0;\n\tint sum_sb = 0;\n\n\t// Nanoflann related variables\n\t// ***************************\n\n\t// CLoud variable\n\tPointCloud current_cloud;\n\n\t// Tree parameters\n\tnanoflann::KDTreeSingleIndexAdaptorParams tree_params(10 /* max leaf */);\n\n\t// KDTree type definition\n typedef nanoflann::KDTreeSingleIndexAdaptor< nanoflann::L2_Simple_Adaptor ,\n PointCloud,\n 3 > my_kd_tree_t;\n\n // Pointer to trees\n my_kd_tree_t* index;\n\n // Build KDTree for the first batch element\n current_cloud.pts = vector(supports.begin() + sum_sb, supports.begin() + sum_sb + s_batches[b]);\n index = new my_kd_tree_t(3, current_cloud, tree_params);\n index->buildIndex();\n\n\n\t// Search neigbors indices\n\t// ***********************\n\n // Search params\n nanoflann::SearchParams search_params;\n search_params.sorted = true;\n\n\tfor (auto& p0 : queries)\n\t{\n\n\t // Check if we changed batch\n\t if (i0 == sum_qb + q_batches[b])\n\t {\n\t sum_qb += q_batches[b];\n\t sum_sb += s_batches[b];\n\t b++;\n\n\t // Change the points\n\t current_cloud.pts.clear();\n current_cloud.pts = vector(supports.begin() + sum_sb, supports.begin() + sum_sb + s_batches[b]);\n\n\t // Build KDTree of the current element of the batch\n delete index;\n index = new my_kd_tree_t(3, current_cloud, tree_params);\n index->buildIndex();\n\t }\n\n\t // Initial guess of neighbors size\n all_inds_dists[i0].reserve(max_count);\n\n\t // Find neighbors\n\t float query_pt[3] = { p0.x, p0.y, p0.z};\n\t\tsize_t nMatches = index->radiusSearch(query_pt, r2, all_inds_dists[i0], search_params);\n\n // Update max count\n if (nMatches > max_count)\n max_count = nMatches;\n\n // Increment query idx\n\t\ti0++;\n\t}\n\n\t// Reserve the memory\n\tneighbors_indices.resize(queries.size() * max_count);\n\ti0 = 0;\n\tsum_sb = 0;\n\tsum_qb = 0;\n\tb = 0;\n\tfor (auto& inds_dists : all_inds_dists)\n\t{\n\t // Check if we changed batch\n\t if (i0 == sum_qb + q_batches[b])\n\t {\n\t sum_qb += q_batches[b];\n\t sum_sb += s_batches[b];\n\t b++;\n\t }\n\n\t\tfor (int j = 0; j < max_count; j++)\n\t\t{\n\t\t\tif (j < inds_dists.size())\n\t\t\t\tneighbors_indices[i0 * max_count + j] = inds_dists[j].first + sum_sb;\n\t\t\telse\n\t\t\t\tneighbors_indices[i0 * max_count + j] = supports.size();\n\t\t}\n\t\ti0++;\n\t}\n\n\tdelete index;\n\n\treturn;\n}\n\n"
},
{
"path": "benchmarks/KPConv-PyTorch-master/cpp_wrappers/cpp_neighbors/neighbors/neighbors.h",
"content": "\n\n#include \"../../cpp_utils/cloud/cloud.h\"\n#include \"../../cpp_utils/nanoflann/nanoflann.hpp\"\n\n#include \n#include \n\nusing namespace std;\n\n\nvoid ordered_neighbors(vector& queries,\n vector& supports,\n vector& neighbors_indices,\n float radius);\n\nvoid batch_ordered_neighbors(vector& queries,\n vector& supports,\n vector& q_batches,\n vector& s_batches,\n vector& neighbors_indices,\n float radius);\n\nvoid batch_nanoflann_neighbors(vector& queries,\n vector& supports,\n vector& q_batches,\n vector& s_batches,\n vector& neighbors_indices,\n float radius);\n"
},
{
"path": "benchmarks/KPConv-PyTorch-master/cpp_wrappers/cpp_neighbors/setup.py",
"content": "from distutils.core import setup, Extension\nimport numpy.distutils.misc_util\n\n# Adding OpenCV to project\n# ************************\n\n# Adding sources of the project\n# *****************************\n\nSOURCES = [\"../cpp_utils/cloud/cloud.cpp\",\n \"neighbors/neighbors.cpp\",\n \"wrapper.cpp\"]\n\nmodule = Extension(name=\"radius_neighbors\",\n sources=SOURCES,\n extra_compile_args=['-std=c++11',\n '-D_GLIBCXX_USE_CXX11_ABI=0'])\n\n\nsetup(ext_modules=[module], include_dirs=numpy.distutils.misc_util.get_numpy_include_dirs())\n\n\n\n\n\n\n\n\n"
},
{
"path": "benchmarks/KPConv-PyTorch-master/cpp_wrappers/cpp_neighbors/wrapper.cpp",
"content": "#include \n#include \n#include \"neighbors/neighbors.h\"\n#include \n\n\n\n// docstrings for our module\n// *************************\n\nstatic char module_docstring[] = \"This module provides two methods to compute radius neighbors from pointclouds or batch of pointclouds\";\n\nstatic char batch_query_docstring[] = \"Method to get radius neighbors in a batch of stacked pointclouds\";\n\n\n// Declare the functions\n// *********************\n\nstatic PyObject *batch_neighbors(PyObject *self, PyObject *args, PyObject *keywds);\n\n\n// Specify the members of the module\n// *********************************\n\nstatic PyMethodDef module_methods[] = \n{\n\t{ \"batch_query\", (PyCFunction)batch_neighbors, METH_VARARGS | METH_KEYWORDS, batch_query_docstring },\n\t{NULL, NULL, 0, NULL}\n};\n\n\n// Initialize the module\n// *********************\n\nstatic struct PyModuleDef moduledef = \n{\n PyModuleDef_HEAD_INIT,\n \"radius_neighbors\",\t\t// m_name\n module_docstring, // m_doc\n -1, // m_size\n module_methods, // m_methods\n NULL, // m_reload\n NULL, // m_traverse\n NULL, // m_clear\n NULL, // m_free\n};\n\nPyMODINIT_FUNC PyInit_radius_neighbors(void)\n{\n import_array();\n\treturn PyModule_Create(&moduledef);\n}\n\n\n// Definition of the batch_subsample method\n// **********************************\n\nstatic PyObject* batch_neighbors(PyObject* self, PyObject* args, PyObject* keywds)\n{\n\n\t// Manage inputs\n\t// *************\n\n\t// Args containers\n\tPyObject* queries_obj = NULL;\n\tPyObject* supports_obj = NULL;\n\tPyObject* q_batches_obj = NULL;\n\tPyObject* s_batches_obj = NULL;\n\n\t// Keywords containers\n\tstatic char* kwlist[] = { \"queries\", \"supports\", \"q_batches\", \"s_batches\", \"radius\", NULL };\n\tfloat radius = 0.1;\n\n\t// Parse the input \n\tif (!PyArg_ParseTupleAndKeywords(args, keywds, \"OOOO|$f\", kwlist, &queries_obj, &supports_obj, &q_batches_obj, &s_batches_obj, &radius))\n\t{\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Error parsing arguments\");\n\t\treturn NULL;\n\t}\n\n\n\t// Interpret the input objects as numpy arrays.\n\tPyObject* queries_array = PyArray_FROM_OTF(queries_obj, NPY_FLOAT, NPY_IN_ARRAY);\n\tPyObject* supports_array = PyArray_FROM_OTF(supports_obj, NPY_FLOAT, NPY_IN_ARRAY);\n\tPyObject* q_batches_array = PyArray_FROM_OTF(q_batches_obj, NPY_INT, NPY_IN_ARRAY);\n\tPyObject* s_batches_array = PyArray_FROM_OTF(s_batches_obj, NPY_INT, NPY_IN_ARRAY);\n\n\t// Verify data was load correctly.\n\tif (queries_array == NULL)\n\t{\n\t\tPy_XDECREF(queries_array);\n\t\tPy_XDECREF(supports_array);\n\t\tPy_XDECREF(q_batches_array);\n\t\tPy_XDECREF(s_batches_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Error converting query points to numpy arrays of type float32\");\n\t\treturn NULL;\n\t}\n\tif (supports_array == NULL)\n\t{\n\t\tPy_XDECREF(queries_array);\n\t\tPy_XDECREF(supports_array);\n\t\tPy_XDECREF(q_batches_array);\n\t\tPy_XDECREF(s_batches_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Error converting support points to numpy arrays of type float32\");\n\t\treturn NULL;\n\t}\n\tif (q_batches_array == NULL)\n\t{\n\t\tPy_XDECREF(queries_array);\n\t\tPy_XDECREF(supports_array);\n\t\tPy_XDECREF(q_batches_array);\n\t\tPy_XDECREF(s_batches_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Error converting query batches to numpy arrays of type int32\");\n\t\treturn NULL;\n\t}\n\tif (s_batches_array == NULL)\n\t{\n\t\tPy_XDECREF(queries_array);\n\t\tPy_XDECREF(supports_array);\n\t\tPy_XDECREF(q_batches_array);\n\t\tPy_XDECREF(s_batches_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Error converting support batches to numpy arrays of type int32\");\n\t\treturn NULL;\n\t}\n\n\t// Check that the input array respect the dims\n\tif ((int)PyArray_NDIM(queries_array) != 2 || (int)PyArray_DIM(queries_array, 1) != 3)\n\t{\n\t\tPy_XDECREF(queries_array);\n\t\tPy_XDECREF(supports_array);\n\t\tPy_XDECREF(q_batches_array);\n\t\tPy_XDECREF(s_batches_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Wrong dimensions : query.shape is not (N, 3)\");\n\t\treturn NULL;\n\t}\n\tif ((int)PyArray_NDIM(supports_array) != 2 || (int)PyArray_DIM(supports_array, 1) != 3)\n\t{\n\t\tPy_XDECREF(queries_array);\n\t\tPy_XDECREF(supports_array);\n\t\tPy_XDECREF(q_batches_array);\n\t\tPy_XDECREF(s_batches_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Wrong dimensions : support.shape is not (N, 3)\");\n\t\treturn NULL;\n\t}\n\tif ((int)PyArray_NDIM(q_batches_array) > 1)\n\t{\n\t\tPy_XDECREF(queries_array);\n\t\tPy_XDECREF(supports_array);\n\t\tPy_XDECREF(q_batches_array);\n\t\tPy_XDECREF(s_batches_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Wrong dimensions : queries_batches.shape is not (B,) \");\n\t\treturn NULL;\n\t}\n\tif ((int)PyArray_NDIM(s_batches_array) > 1)\n\t{\n\t\tPy_XDECREF(queries_array);\n\t\tPy_XDECREF(supports_array);\n\t\tPy_XDECREF(q_batches_array);\n\t\tPy_XDECREF(s_batches_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Wrong dimensions : supports_batches.shape is not (B,) \");\n\t\treturn NULL;\n\t}\n\tif ((int)PyArray_DIM(q_batches_array, 0) != (int)PyArray_DIM(s_batches_array, 0))\n\t{\n\t\tPy_XDECREF(queries_array);\n\t\tPy_XDECREF(supports_array);\n\t\tPy_XDECREF(q_batches_array);\n\t\tPy_XDECREF(s_batches_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Wrong number of batch elements: different for queries and supports \");\n\t\treturn NULL;\n\t}\n\n\t// Number of points\n\tint Nq = (int)PyArray_DIM(queries_array, 0);\n\tint Ns= (int)PyArray_DIM(supports_array, 0);\n\n\t// Number of batches\n\tint Nb = (int)PyArray_DIM(q_batches_array, 0);\n\n\t// Call the C++ function\n\t// *********************\n\n\t// Convert PyArray to Cloud C++ class\n\tvector queries;\n\tvector supports;\n\tvector q_batches;\n\tvector s_batches;\n\tqueries = vector((PointXYZ*)PyArray_DATA(queries_array), (PointXYZ*)PyArray_DATA(queries_array) + Nq);\n\tsupports = vector((PointXYZ*)PyArray_DATA(supports_array), (PointXYZ*)PyArray_DATA(supports_array) + Ns);\n\tq_batches = vector((int*)PyArray_DATA(q_batches_array), (int*)PyArray_DATA(q_batches_array) + Nb);\n\ts_batches = vector((int*)PyArray_DATA(s_batches_array), (int*)PyArray_DATA(s_batches_array) + Nb);\n\n\t// Create result containers\n\tvector neighbors_indices;\n\n\t// Compute results\n\t//batch_ordered_neighbors(queries, supports, q_batches, s_batches, neighbors_indices, radius);\n\tbatch_nanoflann_neighbors(queries, supports, q_batches, s_batches, neighbors_indices, radius);\n\n\t// Check result\n\tif (neighbors_indices.size() < 1)\n\t{\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Error\");\n\t\treturn NULL;\n\t}\n\n\t// Manage outputs\n\t// **************\n\n\t// Maximal number of neighbors\n\tint max_neighbors = neighbors_indices.size() / Nq;\n\n\t// Dimension of output containers\n\tnpy_intp* neighbors_dims = new npy_intp[2];\n\tneighbors_dims[0] = Nq;\n\tneighbors_dims[1] = max_neighbors;\n\n\t// Create output array\n\tPyObject* res_obj = PyArray_SimpleNew(2, neighbors_dims, NPY_INT);\n\tPyObject* ret = NULL;\n\n\t// Fill output array with values\n\tsize_t size_in_bytes = Nq * max_neighbors * sizeof(int);\n\tmemcpy(PyArray_DATA(res_obj), neighbors_indices.data(), size_in_bytes);\n\n\t// Merge results\n\tret = Py_BuildValue(\"N\", res_obj);\n\n\t// Clean up\n\t// ********\n\n\tPy_XDECREF(queries_array);\n\tPy_XDECREF(supports_array);\n\tPy_XDECREF(q_batches_array);\n\tPy_XDECREF(s_batches_array);\n\n\treturn ret;\n}\n"
},
{
"path": "benchmarks/KPConv-PyTorch-master/cpp_wrappers/cpp_subsampling/build.bat",
"content": "@echo off\npy setup.py build_ext --inplace\n\n\npause"
},
{
"path": "benchmarks/KPConv-PyTorch-master/cpp_wrappers/cpp_subsampling/grid_subsampling/grid_subsampling.cpp",
"content": "\n#include \"grid_subsampling.h\"\n\n\nvoid grid_subsampling(vector& original_points,\n vector& subsampled_points,\n vector& original_features,\n vector& subsampled_features,\n vector& original_classes,\n vector& subsampled_classes,\n float sampleDl,\n int verbose) {\n\n\t// Initialize variables\n\t// ******************\n\n\t// Number of points in the cloud\n\tsize_t N = original_points.size();\n\n\t// Dimension of the features\n\tsize_t fdim = original_features.size() / N;\n\tsize_t ldim = original_classes.size() / N;\n\n\t// Limits of the cloud\n\tPointXYZ minCorner = min_point(original_points);\n\tPointXYZ maxCorner = max_point(original_points);\n\tPointXYZ originCorner = floor(minCorner * (1/sampleDl)) * sampleDl;\n\n\t// Dimensions of the grid\n\tsize_t sampleNX = (size_t)floor((maxCorner.x - originCorner.x) / sampleDl) + 1;\n\tsize_t sampleNY = (size_t)floor((maxCorner.y - originCorner.y) / sampleDl) + 1;\n\t//size_t sampleNZ = (size_t)floor((maxCorner.z - originCorner.z) / sampleDl) + 1;\n\n\t// Check if features and classes need to be processed\n\tbool use_feature = original_features.size() > 0;\n\tbool use_classes = original_classes.size() > 0;\n\n\n\t// Create the sampled map\n\t// **********************\n\n\t// Verbose parameters\n\tint i = 0;\n\tint nDisp = N / 100;\n\n\t// Initialize variables\n\tsize_t iX, iY, iZ, mapIdx;\n\tunordered_map data;\n\n\tfor (auto& p : original_points)\n\t{\n\t\t// Position of point in sample map\n\t\tiX = (size_t)floor((p.x - originCorner.x) / sampleDl);\n\t\tiY = (size_t)floor((p.y - originCorner.y) / sampleDl);\n\t\tiZ = (size_t)floor((p.z - originCorner.z) / sampleDl);\n\t\tmapIdx = iX + sampleNX*iY + sampleNX*sampleNY*iZ;\n\n\t\t// If not already created, create key\n\t\tif (data.count(mapIdx) < 1)\n\t\t\tdata.emplace(mapIdx, SampledData(fdim, ldim));\n\n\t\t// Fill the sample map\n\t\tif (use_feature && use_classes)\n\t\t\tdata[mapIdx].update_all(p, original_features.begin() + i * fdim, original_classes.begin() + i * ldim);\n\t\telse if (use_feature)\n\t\t\tdata[mapIdx].update_features(p, original_features.begin() + i * fdim);\n\t\telse if (use_classes)\n\t\t\tdata[mapIdx].update_classes(p, original_classes.begin() + i * ldim);\n\t\telse\n\t\t\tdata[mapIdx].update_points(p);\n\n\t\t// Display\n\t\ti++;\n\t\tif (verbose > 1 && i%nDisp == 0)\n\t\t\tstd::cout << \"\\rSampled Map : \" << std::setw(3) << i / nDisp << \"%\";\n\n\t}\n\n\t// Divide for barycentre and transfer to a vector\n\tsubsampled_points.reserve(data.size());\n\tif (use_feature)\n\t\tsubsampled_features.reserve(data.size() * fdim);\n\tif (use_classes)\n\t\tsubsampled_classes.reserve(data.size() * ldim);\n\tfor (auto& v : data)\n\t{\n\t\tsubsampled_points.push_back(v.second.point * (1.0 / v.second.count));\n\t\tif (use_feature)\n\t\t{\n\t\t float count = (float)v.second.count;\n\t\t transform(v.second.features.begin(),\n v.second.features.end(),\n v.second.features.begin(),\n [count](float f) { return f / count;});\n subsampled_features.insert(subsampled_features.end(),v.second.features.begin(),v.second.features.end());\n\t\t}\n\t\tif (use_classes)\n\t\t{\n\t\t for (int i = 0; i < ldim; i++)\n\t\t subsampled_classes.push_back(max_element(v.second.labels[i].begin(), v.second.labels[i].end(),\n\t\t [](const pair&a, const pair&b){return a.second < b.second;})->first);\n\t\t}\n\t}\n\n\treturn;\n}\n\n\nvoid batch_grid_subsampling(vector& original_points,\n vector& subsampled_points,\n vector& original_features,\n vector& subsampled_features,\n vector& original_classes,\n vector& subsampled_classes,\n vector& original_batches,\n vector& subsampled_batches,\n float sampleDl,\n int max_p)\n{\n\t// Initialize variables\n\t// ******************\n\n\tint b = 0;\n\tint sum_b = 0;\n\n\t// Number of points in the cloud\n\tsize_t N = original_points.size();\n\n\t// Dimension of the features\n\tsize_t fdim = original_features.size() / N;\n\tsize_t ldim = original_classes.size() / N;\n\n\t// Handle max_p = 0\n\tif (max_p < 1)\n\t max_p = N;\n\n\t// Loop over batches\n\t// *****************\n\n\tfor (b = 0; b < original_batches.size(); b++)\n\t{\n\n\t // Extract batch points features and labels\n\t vector b_o_points = vector(original_points.begin () + sum_b,\n\t original_points.begin () + sum_b + original_batches[b]);\n\n vector b_o_features;\n if (original_features.size() > 0)\n {\n b_o_features = vector(original_features.begin () + sum_b * fdim,\n original_features.begin () + (sum_b + original_batches[b]) * fdim);\n\t }\n\n\t vector b_o_classes;\n if (original_classes.size() > 0)\n {\n b_o_classes = vector(original_classes.begin () + sum_b * ldim,\n original_classes.begin () + sum_b + original_batches[b] * ldim);\n\t }\n\n\n // Create result containers\n vector b_s_points;\n vector b_s_features;\n vector b_s_classes;\n\n // Compute subsampling on current batch\n grid_subsampling(b_o_points,\n b_s_points,\n b_o_features,\n b_s_features,\n b_o_classes,\n b_s_classes,\n sampleDl,\n\t\t\t\t\t\t 0);\n\n // Stack batches points features and labels\n // ****************************************\n\n // If too many points remove some\n if (b_s_points.size() <= max_p)\n {\n subsampled_points.insert(subsampled_points.end(), b_s_points.begin(), b_s_points.end());\n\n if (original_features.size() > 0)\n subsampled_features.insert(subsampled_features.end(), b_s_features.begin(), b_s_features.end());\n\n if (original_classes.size() > 0)\n subsampled_classes.insert(subsampled_classes.end(), b_s_classes.begin(), b_s_classes.end());\n\n subsampled_batches.push_back(b_s_points.size());\n }\n else\n {\n subsampled_points.insert(subsampled_points.end(), b_s_points.begin(), b_s_points.begin() + max_p);\n\n if (original_features.size() > 0)\n subsampled_features.insert(subsampled_features.end(), b_s_features.begin(), b_s_features.begin() + max_p * fdim);\n\n if (original_classes.size() > 0)\n subsampled_classes.insert(subsampled_classes.end(), b_s_classes.begin(), b_s_classes.begin() + max_p * ldim);\n\n subsampled_batches.push_back(max_p);\n }\n\n // Stack new batch lengths\n sum_b += original_batches[b];\n\t}\n\n\treturn;\n}\n"
},
{
"path": "benchmarks/KPConv-PyTorch-master/cpp_wrappers/cpp_subsampling/grid_subsampling/grid_subsampling.h",
"content": "\n\n#include \"../../cpp_utils/cloud/cloud.h\"\n\n#include \n#include \n\nusing namespace std;\n\nclass SampledData\n{\npublic:\n\n\t// Elements\n\t// ********\n\n\tint count;\n\tPointXYZ point;\n\tvector features;\n\tvector> labels;\n\n\n\t// Methods\n\t// *******\n\n\t// Constructor\n\tSampledData() \n\t{ \n\t\tcount = 0; \n\t\tpoint = PointXYZ();\n\t}\n\n\tSampledData(const size_t fdim, const size_t ldim)\n\t{\n\t\tcount = 0;\n\t\tpoint = PointXYZ();\n\t features = vector(fdim);\n\t labels = vector>(ldim);\n\t}\n\n\t// Method Update\n\tvoid update_all(const PointXYZ p, vector::iterator f_begin, vector::iterator l_begin)\n\t{\n\t\tcount += 1;\n\t\tpoint += p;\n\t\ttransform (features.begin(), features.end(), f_begin, features.begin(), plus());\n\t\tint i = 0;\n\t\tfor(vector::iterator it = l_begin; it != l_begin + labels.size(); ++it)\n\t\t{\n\t\t labels[i][*it] += 1;\n\t\t i++;\n\t\t}\n\t\treturn;\n\t}\n\tvoid update_features(const PointXYZ p, vector::iterator f_begin)\n\t{\n\t\tcount += 1;\n\t\tpoint += p;\n\t\ttransform (features.begin(), features.end(), f_begin, features.begin(), plus());\n\t\treturn;\n\t}\n\tvoid update_classes(const PointXYZ p, vector::iterator l_begin)\n\t{\n\t\tcount += 1;\n\t\tpoint += p;\n\t\tint i = 0;\n\t\tfor(vector::iterator it = l_begin; it != l_begin + labels.size(); ++it)\n\t\t{\n\t\t labels[i][*it] += 1;\n\t\t i++;\n\t\t}\n\t\treturn;\n\t}\n\tvoid update_points(const PointXYZ p)\n\t{\n\t\tcount += 1;\n\t\tpoint += p;\n\t\treturn;\n\t}\n};\n\nvoid grid_subsampling(vector& original_points,\n vector& subsampled_points,\n vector& original_features,\n vector& subsampled_features,\n vector& original_classes,\n vector& subsampled_classes,\n float sampleDl,\n int verbose);\n\nvoid batch_grid_subsampling(vector& original_points,\n vector& subsampled_points,\n vector& original_features,\n vector& subsampled_features,\n vector& original_classes,\n vector& subsampled_classes,\n vector& original_batches,\n vector& subsampled_batches,\n float sampleDl,\n int max_p);\n\n"
},
{
"path": "benchmarks/KPConv-PyTorch-master/cpp_wrappers/cpp_subsampling/setup.py",
"content": "from distutils.core import setup, Extension\nimport numpy.distutils.misc_util\n\n# Adding OpenCV to project\n# ************************\n\n# Adding sources of the project\n# *****************************\n\nSOURCES = [\"../cpp_utils/cloud/cloud.cpp\",\n \"grid_subsampling/grid_subsampling.cpp\",\n \"wrapper.cpp\"]\n\nmodule = Extension(name=\"grid_subsampling\",\n sources=SOURCES,\n extra_compile_args=['-std=c++11',\n '-D_GLIBCXX_USE_CXX11_ABI=0'])\n\n\nsetup(ext_modules=[module], include_dirs=numpy.distutils.misc_util.get_numpy_include_dirs())\n\n\n\n\n\n\n\n\n"
},
{
"path": "benchmarks/KPConv-PyTorch-master/cpp_wrappers/cpp_subsampling/wrapper.cpp",
"content": "#include \n#include \n#include \"grid_subsampling/grid_subsampling.h\"\n#include \n\n\n\n// docstrings for our module\n// *************************\n\nstatic char module_docstring[] = \"This module provides an interface for the subsampling of a batch of stacked pointclouds\";\n\nstatic char subsample_docstring[] = \"function subsampling a pointcloud\";\n\nstatic char subsample_batch_docstring[] = \"function subsampling a batch of stacked pointclouds\";\n\n\n// Declare the functions\n// *********************\n\nstatic PyObject *cloud_subsampling(PyObject* self, PyObject* args, PyObject* keywds);\nstatic PyObject *batch_subsampling(PyObject *self, PyObject *args, PyObject *keywds);\n\n\n// Specify the members of the module\n// *********************************\n\nstatic PyMethodDef module_methods[] = \n{\n\t{ \"subsample\", (PyCFunction)cloud_subsampling, METH_VARARGS | METH_KEYWORDS, subsample_docstring },\n\t{ \"subsample_batch\", (PyCFunction)batch_subsampling, METH_VARARGS | METH_KEYWORDS, subsample_batch_docstring },\n\t{NULL, NULL, 0, NULL}\n};\n\n\n// Initialize the module\n// *********************\n\nstatic struct PyModuleDef moduledef = \n{\n PyModuleDef_HEAD_INIT,\n \"grid_subsampling\", // m_name\n module_docstring, // m_doc\n -1, // m_size\n module_methods, // m_methods\n NULL, // m_reload\n NULL, // m_traverse\n NULL, // m_clear\n NULL, // m_free\n};\n\nPyMODINIT_FUNC PyInit_grid_subsampling(void)\n{\n import_array();\n\treturn PyModule_Create(&moduledef);\n}\n\n\n// Definition of the batch_subsample method\n// **********************************\n\nstatic PyObject* batch_subsampling(PyObject* self, PyObject* args, PyObject* keywds)\n{\n\n\t// Manage inputs\n\t// *************\n\n\t// Args containers\n\tPyObject* points_obj = NULL;\n\tPyObject* features_obj = NULL;\n\tPyObject* classes_obj = NULL;\n\tPyObject* batches_obj = NULL;\n\n\t// Keywords containers\n\tstatic char* kwlist[] = { \"points\", \"batches\", \"features\", \"classes\", \"sampleDl\", \"method\", \"max_p\", \"verbose\", NULL };\n\tfloat sampleDl = 0.1;\n\tconst char* method_buffer = \"barycenters\";\n\tint verbose = 0;\n\tint max_p = 0;\n\n\t// Parse the input \n\tif (!PyArg_ParseTupleAndKeywords(args, keywds, \"OO|$OOfsii\", kwlist, &points_obj, &batches_obj, &features_obj, &classes_obj, &sampleDl, &method_buffer, &max_p, &verbose))\n\t{\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Error parsing arguments\");\n\t\treturn NULL;\n\t}\n\n\t// Get the method argument\n\tstring method(method_buffer);\n\n\t// Interpret method\n\tif (method.compare(\"barycenters\") && method.compare(\"voxelcenters\"))\n\t{\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Error parsing method. Valid method names are \\\"barycenters\\\" and \\\"voxelcenters\\\" \");\n\t\treturn NULL;\n\t}\n\n\t// Check if using features or classes\n\tbool use_feature = true, use_classes = true;\n\tif (features_obj == NULL)\n\t\tuse_feature = false;\n\tif (classes_obj == NULL)\n\t\tuse_classes = false;\n\n\t// Interpret the input objects as numpy arrays.\n\tPyObject* points_array = PyArray_FROM_OTF(points_obj, NPY_FLOAT, NPY_IN_ARRAY);\n\tPyObject* batches_array = PyArray_FROM_OTF(batches_obj, NPY_INT, NPY_IN_ARRAY);\n\tPyObject* features_array = NULL;\n\tPyObject* classes_array = NULL;\n\tif (use_feature)\n\t\tfeatures_array = PyArray_FROM_OTF(features_obj, NPY_FLOAT, NPY_IN_ARRAY);\n\tif (use_classes)\n\t\tclasses_array = PyArray_FROM_OTF(classes_obj, NPY_INT, NPY_IN_ARRAY);\n\n\t// Verify data was load correctly.\n\tif (points_array == NULL)\n\t{\n\t\tPy_XDECREF(points_array);\n\t\tPy_XDECREF(batches_array);\n\t\tPy_XDECREF(classes_array);\n\t\tPy_XDECREF(features_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Error converting input points to numpy arrays of type float32\");\n\t\treturn NULL;\n\t}\n\tif (batches_array == NULL)\n\t{\n\t\tPy_XDECREF(points_array);\n\t\tPy_XDECREF(batches_array);\n\t\tPy_XDECREF(classes_array);\n\t\tPy_XDECREF(features_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Error converting input batches to numpy arrays of type int32\");\n\t\treturn NULL;\n\t}\n\tif (use_feature && features_array == NULL)\n\t{\n\t\tPy_XDECREF(points_array);\n\t\tPy_XDECREF(batches_array);\n\t\tPy_XDECREF(classes_array);\n\t\tPy_XDECREF(features_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Error converting input features to numpy arrays of type float32\");\n\t\treturn NULL;\n\t}\n\tif (use_classes && classes_array == NULL)\n\t{\n\t\tPy_XDECREF(points_array);\n\t\tPy_XDECREF(batches_array);\n\t\tPy_XDECREF(classes_array);\n\t\tPy_XDECREF(features_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Error converting input classes to numpy arrays of type int32\");\n\t\treturn NULL;\n\t}\n\n\t// Check that the input array respect the dims\n\tif ((int)PyArray_NDIM(points_array) != 2 || (int)PyArray_DIM(points_array, 1) != 3)\n\t{\n\t\tPy_XDECREF(points_array);\n\t\tPy_XDECREF(batches_array);\n\t\tPy_XDECREF(classes_array);\n\t\tPy_XDECREF(features_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Wrong dimensions : points.shape is not (N, 3)\");\n\t\treturn NULL;\n\t}\n\tif ((int)PyArray_NDIM(batches_array) > 1)\n\t{\n\t\tPy_XDECREF(points_array);\n\t\tPy_XDECREF(batches_array);\n\t\tPy_XDECREF(classes_array);\n\t\tPy_XDECREF(features_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Wrong dimensions : batches.shape is not (B,) \");\n\t\treturn NULL;\n\t}\n\tif (use_feature && ((int)PyArray_NDIM(features_array) != 2))\n\t{\n\t\tPy_XDECREF(points_array);\n\t\tPy_XDECREF(batches_array);\n\t\tPy_XDECREF(classes_array);\n\t\tPy_XDECREF(features_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Wrong dimensions : features.shape is not (N, d)\");\n\t\treturn NULL;\n\t}\n\n\tif (use_classes && (int)PyArray_NDIM(classes_array) > 2)\n\t{\n\t\tPy_XDECREF(points_array);\n\t\tPy_XDECREF(batches_array);\n\t\tPy_XDECREF(classes_array);\n\t\tPy_XDECREF(features_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Wrong dimensions : classes.shape is not (N,) or (N, d)\");\n\t\treturn NULL;\n\t}\n\n\t// Number of points\n\tint N = (int)PyArray_DIM(points_array, 0);\n\n\t// Number of batches\n\tint Nb = (int)PyArray_DIM(batches_array, 0);\n\n\t// Dimension of the features\n\tint fdim = 0;\n\tif (use_feature)\n\t\tfdim = (int)PyArray_DIM(features_array, 1);\n\n\t//Dimension of labels\n\tint ldim = 1;\n\tif (use_classes && (int)PyArray_NDIM(classes_array) == 2)\n\t\tldim = (int)PyArray_DIM(classes_array, 1);\n\n\t// Check that the input array respect the number of points\n\tif (use_feature && (int)PyArray_DIM(features_array, 0) != N)\n\t{\n\t\tPy_XDECREF(points_array);\n\t\tPy_XDECREF(batches_array);\n\t\tPy_XDECREF(classes_array);\n\t\tPy_XDECREF(features_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Wrong dimensions : features.shape is not (N, d)\");\n\t\treturn NULL;\n\t}\n\tif (use_classes && (int)PyArray_DIM(classes_array, 0) != N)\n\t{\n\t\tPy_XDECREF(points_array);\n\t\tPy_XDECREF(batches_array);\n\t\tPy_XDECREF(classes_array);\n\t\tPy_XDECREF(features_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Wrong dimensions : classes.shape is not (N,) or (N, d)\");\n\t\treturn NULL;\n\t}\n\n\n\t// Call the C++ function\n\t// *********************\n\n\t// Create pyramid\n\tif (verbose > 0)\n\t\tcout << \"Computing cloud pyramid with support points: \" << endl;\n\n\n\t// Convert PyArray to Cloud C++ class\n\tvector original_points;\n\tvector original_batches;\n\tvector original_features;\n\tvector original_classes;\n\toriginal_points = vector((PointXYZ*)PyArray_DATA(points_array), (PointXYZ*)PyArray_DATA(points_array) + N);\n\toriginal_batches = vector((int*)PyArray_DATA(batches_array), (int*)PyArray_DATA(batches_array) + Nb);\n\tif (use_feature)\n\t\toriginal_features = vector((float*)PyArray_DATA(features_array), (float*)PyArray_DATA(features_array) + N * fdim);\n\tif (use_classes)\n\t\toriginal_classes = vector((int*)PyArray_DATA(classes_array), (int*)PyArray_DATA(classes_array) + N * ldim);\n\n\t// Subsample\n\tvector subsampled_points;\n\tvector subsampled_features;\n\tvector subsampled_classes;\n\tvector subsampled_batches;\n\tbatch_grid_subsampling(original_points,\n\t\t\t\t\t\t\tsubsampled_points,\n\t\t\t\t\t\t\toriginal_features,\n\t\t\t\t\t\t\tsubsampled_features,\n\t\t\t\t\t\t\toriginal_classes,\n\t\t\t\t\t\t\tsubsampled_classes,\n\t\t\t\t\t\t\toriginal_batches,\n\t\t\t\t\t\t\tsubsampled_batches,\n\t\t\t\t\t\t\tsampleDl,\n\t\t\t\t\t\t\tmax_p);\n\n\t// Check result\n\tif (subsampled_points.size() < 1)\n\t{\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Error\");\n\t\treturn NULL;\n\t}\n\n\t// Manage outputs\n\t// **************\n\n\t// Dimension of input containers\n\tnpy_intp* point_dims = new npy_intp[2];\n\tpoint_dims[0] = subsampled_points.size();\n\tpoint_dims[1] = 3;\n\tnpy_intp* feature_dims = new npy_intp[2];\n\tfeature_dims[0] = subsampled_points.size();\n\tfeature_dims[1] = fdim;\n\tnpy_intp* classes_dims = new npy_intp[2];\n\tclasses_dims[0] = subsampled_points.size();\n\tclasses_dims[1] = ldim;\n\tnpy_intp* batches_dims = new npy_intp[1];\n\tbatches_dims[0] = Nb;\n\n\t// Create output array\n\tPyObject* res_points_obj = PyArray_SimpleNew(2, point_dims, NPY_FLOAT);\n\tPyObject* res_batches_obj = PyArray_SimpleNew(1, batches_dims, NPY_INT);\n\tPyObject* res_features_obj = NULL;\n\tPyObject* res_classes_obj = NULL;\n\tPyObject* ret = NULL;\n\n\t// Fill output array with values\n\tsize_t size_in_bytes = subsampled_points.size() * 3 * sizeof(float);\n\tmemcpy(PyArray_DATA(res_points_obj), subsampled_points.data(), size_in_bytes);\n\tsize_in_bytes = Nb * sizeof(int);\n\tmemcpy(PyArray_DATA(res_batches_obj), subsampled_batches.data(), size_in_bytes);\n\tif (use_feature)\n\t{\n\t\tsize_in_bytes = subsampled_points.size() * fdim * sizeof(float);\n\t\tres_features_obj = PyArray_SimpleNew(2, feature_dims, NPY_FLOAT);\n\t\tmemcpy(PyArray_DATA(res_features_obj), subsampled_features.data(), size_in_bytes);\n\t}\n\tif (use_classes)\n\t{\n\t\tsize_in_bytes = subsampled_points.size() * ldim * sizeof(int);\n\t\tres_classes_obj = PyArray_SimpleNew(2, classes_dims, NPY_INT);\n\t\tmemcpy(PyArray_DATA(res_classes_obj), subsampled_classes.data(), size_in_bytes);\n\t}\n\n\n\t// Merge results\n\tif (use_feature && use_classes)\n\t\tret = Py_BuildValue(\"NNNN\", res_points_obj, res_batches_obj, res_features_obj, res_classes_obj);\n\telse if (use_feature)\n\t\tret = Py_BuildValue(\"NNN\", res_points_obj, res_batches_obj, res_features_obj);\n\telse if (use_classes)\n\t\tret = Py_BuildValue(\"NNN\", res_points_obj, res_batches_obj, res_classes_obj);\n\telse\n\t\tret = Py_BuildValue(\"NN\", res_points_obj, res_batches_obj);\n\n\t// Clean up\n\t// ********\n\n\tPy_DECREF(points_array);\n\tPy_DECREF(batches_array);\n\tPy_XDECREF(features_array);\n\tPy_XDECREF(classes_array);\n\n\treturn ret;\n}\n\n// Definition of the subsample method\n// ****************************************\n\nstatic PyObject* cloud_subsampling(PyObject* self, PyObject* args, PyObject* keywds)\n{\n\n\t// Manage inputs\n\t// *************\n\n\t// Args containers\n\tPyObject* points_obj = NULL;\n\tPyObject* features_obj = NULL;\n\tPyObject* classes_obj = NULL;\n\n\t// Keywords containers\n\tstatic char* kwlist[] = { \"points\", \"features\", \"classes\", \"sampleDl\", \"method\", \"verbose\", NULL };\n\tfloat sampleDl = 0.1;\n\tconst char* method_buffer = \"barycenters\";\n\tint verbose = 0;\n\n\t// Parse the input \n\tif (!PyArg_ParseTupleAndKeywords(args, keywds, \"O|$OOfsi\", kwlist, &points_obj, &features_obj, &classes_obj, &sampleDl, &method_buffer, &verbose))\n\t{\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Error parsing arguments\");\n\t\treturn NULL;\n\t}\n\n\t// Get the method argument\n\tstring method(method_buffer);\n\n\t// Interpret method\n\tif (method.compare(\"barycenters\") && method.compare(\"voxelcenters\"))\n\t{\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Error parsing method. Valid method names are \\\"barycenters\\\" and \\\"voxelcenters\\\" \");\n\t\treturn NULL;\n\t}\n\n\t// Check if using features or classes\n\tbool use_feature = true, use_classes = true;\n\tif (features_obj == NULL)\n\t\tuse_feature = false;\n\tif (classes_obj == NULL)\n\t\tuse_classes = false;\n\n\t// Interpret the input objects as numpy arrays.\n\tPyObject* points_array = PyArray_FROM_OTF(points_obj, NPY_FLOAT, NPY_IN_ARRAY);\n\tPyObject* features_array = NULL;\n\tPyObject* classes_array = NULL;\n\tif (use_feature)\n\t\tfeatures_array = PyArray_FROM_OTF(features_obj, NPY_FLOAT, NPY_IN_ARRAY);\n\tif (use_classes)\n\t\tclasses_array = PyArray_FROM_OTF(classes_obj, NPY_INT, NPY_IN_ARRAY);\n\n\t// Verify data was load correctly.\n\tif (points_array == NULL)\n\t{\n\t\tPy_XDECREF(points_array);\n\t\tPy_XDECREF(classes_array);\n\t\tPy_XDECREF(features_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Error converting input points to numpy arrays of type float32\");\n\t\treturn NULL;\n\t}\n\tif (use_feature && features_array == NULL)\n\t{\n\t\tPy_XDECREF(points_array);\n\t\tPy_XDECREF(classes_array);\n\t\tPy_XDECREF(features_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Error converting input features to numpy arrays of type float32\");\n\t\treturn NULL;\n\t}\n\tif (use_classes && classes_array == NULL)\n\t{\n\t\tPy_XDECREF(points_array);\n\t\tPy_XDECREF(classes_array);\n\t\tPy_XDECREF(features_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Error converting input classes to numpy arrays of type int32\");\n\t\treturn NULL;\n\t}\n\n\t// Check that the input array respect the dims\n\tif ((int)PyArray_NDIM(points_array) != 2 || (int)PyArray_DIM(points_array, 1) != 3)\n\t{\n\t\tPy_XDECREF(points_array);\n\t\tPy_XDECREF(classes_array);\n\t\tPy_XDECREF(features_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Wrong dimensions : points.shape is not (N, 3)\");\n\t\treturn NULL;\n\t}\n\tif (use_feature && ((int)PyArray_NDIM(features_array) != 2))\n\t{\n\t\tPy_XDECREF(points_array);\n\t\tPy_XDECREF(classes_array);\n\t\tPy_XDECREF(features_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Wrong dimensions : features.shape is not (N, d)\");\n\t\treturn NULL;\n\t}\n\n\tif (use_classes && (int)PyArray_NDIM(classes_array) > 2)\n\t{\n\t\tPy_XDECREF(points_array);\n\t\tPy_XDECREF(classes_array);\n\t\tPy_XDECREF(features_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Wrong dimensions : classes.shape is not (N,) or (N, d)\");\n\t\treturn NULL;\n\t}\n\n\t// Number of points\n\tint N = (int)PyArray_DIM(points_array, 0);\n\n\t// Dimension of the features\n\tint fdim = 0;\n\tif (use_feature)\n\t\tfdim = (int)PyArray_DIM(features_array, 1);\n\n\t//Dimension of labels\n\tint ldim = 1;\n\tif (use_classes && (int)PyArray_NDIM(classes_array) == 2)\n\t\tldim = (int)PyArray_DIM(classes_array, 1);\n\n\t// Check that the input array respect the number of points\n\tif (use_feature && (int)PyArray_DIM(features_array, 0) != N)\n\t{\n\t\tPy_XDECREF(points_array);\n\t\tPy_XDECREF(classes_array);\n\t\tPy_XDECREF(features_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Wrong dimensions : features.shape is not (N, d)\");\n\t\treturn NULL;\n\t}\n\tif (use_classes && (int)PyArray_DIM(classes_array, 0) != N)\n\t{\n\t\tPy_XDECREF(points_array);\n\t\tPy_XDECREF(classes_array);\n\t\tPy_XDECREF(features_array);\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Wrong dimensions : classes.shape is not (N,) or (N, d)\");\n\t\treturn NULL;\n\t}\n\n\n\t// Call the C++ function\n\t// *********************\n\n\t// Create pyramid\n\tif (verbose > 0)\n\t\tcout << \"Computing cloud pyramid with support points: \" << endl;\n\n\n\t// Convert PyArray to Cloud C++ class\n\tvector original_points;\n\tvector original_features;\n\tvector original_classes;\n\toriginal_points = vector((PointXYZ*)PyArray_DATA(points_array), (PointXYZ*)PyArray_DATA(points_array) + N);\n\tif (use_feature)\n\t\toriginal_features = vector((float*)PyArray_DATA(features_array), (float*)PyArray_DATA(features_array) + N * fdim);\n\tif (use_classes)\n\t\toriginal_classes = vector((int*)PyArray_DATA(classes_array), (int*)PyArray_DATA(classes_array) + N * ldim);\n\n\t// Subsample\n\tvector subsampled_points;\n\tvector subsampled_features;\n\tvector subsampled_classes;\n\tgrid_subsampling(original_points,\n\t\tsubsampled_points,\n\t\toriginal_features,\n\t\tsubsampled_features,\n\t\toriginal_classes,\n\t\tsubsampled_classes,\n\t\tsampleDl,\n\t\tverbose);\n\n\t// Check result\n\tif (subsampled_points.size() < 1)\n\t{\n\t\tPyErr_SetString(PyExc_RuntimeError, \"Error\");\n\t\treturn NULL;\n\t}\n\n\t// Manage outputs\n\t// **************\n\n\t// Dimension of input containers\n\tnpy_intp* point_dims = new npy_intp[2];\n\tpoint_dims[0] = subsampled_points.size();\n\tpoint_dims[1] = 3;\n\tnpy_intp* feature_dims = new npy_intp[2];\n\tfeature_dims[0] = subsampled_points.size();\n\tfeature_dims[1] = fdim;\n\tnpy_intp* classes_dims = new npy_intp[2];\n\tclasses_dims[0] = subsampled_points.size();\n\tclasses_dims[1] = ldim;\n\n\t// Create output array\n\tPyObject* res_points_obj = PyArray_SimpleNew(2, point_dims, NPY_FLOAT);\n\tPyObject* res_features_obj = NULL;\n\tPyObject* res_classes_obj = NULL;\n\tPyObject* ret = NULL;\n\n\t// Fill output array with values\n\tsize_t size_in_bytes = subsampled_points.size() * 3 * sizeof(float);\n\tmemcpy(PyArray_DATA(res_points_obj), subsampled_points.data(), size_in_bytes);\n\tif (use_feature)\n\t{\n\t\tsize_in_bytes = subsampled_points.size() * fdim * sizeof(float);\n\t\tres_features_obj = PyArray_SimpleNew(2, feature_dims, NPY_FLOAT);\n\t\tmemcpy(PyArray_DATA(res_features_obj), subsampled_features.data(), size_in_bytes);\n\t}\n\tif (use_classes)\n\t{\n\t\tsize_in_bytes = subsampled_points.size() * ldim * sizeof(int);\n\t\tres_classes_obj = PyArray_SimpleNew(2, classes_dims, NPY_INT);\n\t\tmemcpy(PyArray_DATA(res_classes_obj), subsampled_classes.data(), size_in_bytes);\n\t}\n\n\n\t// Merge results\n\tif (use_feature && use_classes)\n\t\tret = Py_BuildValue(\"NNN\", res_points_obj, res_features_obj, res_classes_obj);\n\telse if (use_feature)\n\t\tret = Py_BuildValue(\"NN\", res_points_obj, res_features_obj);\n\telse if (use_classes)\n\t\tret = Py_BuildValue(\"NN\", res_points_obj, res_classes_obj);\n\telse\n\t\tret = Py_BuildValue(\"N\", res_points_obj);\n\n\t// Clean up\n\t// ********\n\n\tPy_DECREF(points_array);\n\tPy_XDECREF(features_array);\n\tPy_XDECREF(classes_array);\n\n\treturn ret;\n}"
},
{
"path": "benchmarks/KPConv-PyTorch-master/cpp_wrappers/cpp_utils/cloud/cloud.cpp",
"content": "//\n//\n//\t\t0==========================0\n//\t\t| Local feature test |\n//\t\t0==========================0\n//\n//\t\tversion 1.0 : \n//\t\t\t> \n//\n//---------------------------------------------------\n//\n//\t\tCloud source :\n//\t\tDefine usefull Functions/Methods\n//\n//----------------------------------------------------\n//\n//\t\tHugues THOMAS - 10/02/2017\n//\n\n\n#include \"cloud.h\"\n\n\n// Getters\n// *******\n\nPointXYZ max_point(std::vector points)\n{\n\t// Initialize limits\n\tPointXYZ maxP(points[0]);\n\n\t// Loop over all points\n\tfor (auto p : points)\n\t{\n\t\tif (p.x > maxP.x)\n\t\t\tmaxP.x = p.x;\n\n\t\tif (p.y > maxP.y)\n\t\t\tmaxP.y = p.y;\n\n\t\tif (p.z > maxP.z)\n\t\t\tmaxP.z = p.z;\n\t}\n\n\treturn maxP;\n}\n\nPointXYZ min_point(std::vector points)\n{\n\t// Initialize limits\n\tPointXYZ minP(points[0]);\n\n\t// Loop over all points\n\tfor (auto p : points)\n\t{\n\t\tif (p.x < minP.x)\n\t\t\tminP.x = p.x;\n\n\t\tif (p.y < minP.y)\n\t\t\tminP.y = p.y;\n\n\t\tif (p.z < minP.z)\n\t\t\tminP.z = p.z;\n\t}\n\n\treturn minP;\n}"
},
{
"path": "benchmarks/KPConv-PyTorch-master/cpp_wrappers/cpp_utils/cloud/cloud.h",
"content": "//\n//\n//\t\t0==========================0\n//\t\t| Local feature test |\n//\t\t0==========================0\n//\n//\t\tversion 1.0 : \n//\t\t\t> \n//\n//---------------------------------------------------\n//\n//\t\tCloud header\n//\n//----------------------------------------------------\n//\n//\t\tHugues THOMAS - 10/02/2017\n//\n\n\n# pragma once\n\n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n\n#include \n\n\n\n\n// Point class\n// ***********\n\n\nclass PointXYZ\n{\npublic:\n\n\t// Elements\n\t// ********\n\n\tfloat x, y, z;\n\n\n\t// Methods\n\t// *******\n\t\n\t// Constructor\n\tPointXYZ() { x = 0; y = 0; z = 0; }\n\tPointXYZ(float x0, float y0, float z0) { x = x0; y = y0; z = z0; }\n\t\n\t// array type accessor\n\tfloat operator [] (int i) const\n\t{\n\t\tif (i == 0) return x;\n\t\telse if (i == 1) return y;\n\t\telse return z;\n\t}\n\n\t// opperations\n\tfloat dot(const PointXYZ P) const\n\t{\n\t\treturn x * P.x + y * P.y + z * P.z;\n\t}\n\n\tfloat sq_norm()\n\t{\n\t\treturn x*x + y*y + z*z;\n\t}\n\n\tPointXYZ cross(const PointXYZ P) const\n\t{\n\t\treturn PointXYZ(y*P.z - z*P.y, z*P.x - x*P.z, x*P.y - y*P.x);\n\t}\t\n\n\tPointXYZ& operator+=(const PointXYZ& P)\n\t{\n\t\tx += P.x;\n\t\ty += P.y;\n\t\tz += P.z;\n\t\treturn *this;\n\t}\n\n\tPointXYZ& operator-=(const PointXYZ& P)\n\t{\n\t\tx -= P.x;\n\t\ty -= P.y;\n\t\tz -= P.z;\n\t\treturn *this;\n\t}\n\n\tPointXYZ& operator*=(const float& a)\n\t{\n\t\tx *= a;\n\t\ty *= a;\n\t\tz *= a;\n\t\treturn *this;\n\t}\n};\n\n\n// Point Opperations\n// *****************\n\ninline PointXYZ operator + (const PointXYZ A, const PointXYZ B)\n{\n\treturn PointXYZ(A.x + B.x, A.y + B.y, A.z + B.z);\n}\n\ninline PointXYZ operator - (const PointXYZ A, const PointXYZ B)\n{\n\treturn PointXYZ(A.x - B.x, A.y - B.y, A.z - B.z);\n}\n\ninline PointXYZ operator * (const PointXYZ P, const float a)\n{\n\treturn PointXYZ(P.x * a, P.y * a, P.z * a);\n}\n\ninline PointXYZ operator * (const float a, const PointXYZ P)\n{\n\treturn PointXYZ(P.x * a, P.y * a, P.z * a);\n}\n\ninline std::ostream& operator << (std::ostream& os, const PointXYZ P)\n{\n\treturn os << \"[\" << P.x << \", \" << P.y << \", \" << P.z << \"]\";\n}\n\ninline bool operator == (const PointXYZ A, const PointXYZ B)\n{\n\treturn A.x == B.x && A.y == B.y && A.z == B.z;\n}\n\ninline PointXYZ floor(const PointXYZ P)\n{\n\treturn PointXYZ(std::floor(P.x), std::floor(P.y), std::floor(P.z));\n}\n\n\nPointXYZ max_point(std::vector points);\nPointXYZ min_point(std::vector points);\n\n\nstruct PointCloud\n{\n\n\tstd::vector pts;\n\n\t// Must return the number of data points\n\tinline size_t kdtree_get_point_count() const { return pts.size(); }\n\n\t// Returns the dim'th component of the idx'th point in the class:\n\t// Since this is inlined and the \"dim\" argument is typically an immediate value, the\n\t// \"if/else's\" are actually solved at compile time.\n\tinline float kdtree_get_pt(const size_t idx, const size_t dim) const\n\t{\n\t\tif (dim == 0) return pts[idx].x;\n\t\telse if (dim == 1) return pts[idx].y;\n\t\telse return pts[idx].z;\n\t}\n\n\t// Optional bounding-box computation: return false to default to a standard bbox computation loop.\n\t// Return true if the BBOX was already computed by the class and returned in \"bb\" so it can be avoided to redo it again.\n\t// Look at bb.size() to find out the expected dimensionality (e.g. 2 or 3 for point clouds)\n\ttemplate \n\tbool kdtree_get_bbox(BBOX& /* bb */) const { return false; }\n\n};\n\n\n\n\n\n\n\n\n\n\n\n"
},
{
"path": "benchmarks/KPConv-PyTorch-master/cpp_wrappers/cpp_utils/nanoflann/nanoflann.hpp",
"content": "/***********************************************************************\n * Software License Agreement (BSD License)\n *\n * Copyright 2008-2009 Marius Muja (mariusm@cs.ubc.ca). All rights reserved.\n * Copyright 2008-2009 David G. Lowe (lowe@cs.ubc.ca). All rights reserved.\n * Copyright 2011-2016 Jose Luis Blanco (joseluisblancoc@gmail.com).\n * All rights reserved.\n *\n * THE BSD LICENSE\n *\n * Redistribution and use in source and binary forms, with or without\n * modification, are permitted provided that the following conditions\n * are met:\n *\n * 1. Redistributions of source code must retain the above copyright\n * notice, this list of conditions and the following disclaimer.\n * 2. Redistributions in binary form must reproduce the above copyright\n * notice, this list of conditions and the following disclaimer in the\n * documentation and/or other materials provided with the distribution.\n *\n * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR\n * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES\n * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.\n * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,\n * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT\n * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF\n * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n *************************************************************************/\n\n/** \\mainpage nanoflann C++ API documentation\n * nanoflann is a C++ header-only library for building KD-Trees, mostly\n * optimized for 2D or 3D point clouds.\n *\n * nanoflann does not require compiling or installing, just an\n * #include in your code.\n *\n * See:\n * - C++ API organized by modules \n * - Online README \n * - Doxygen\n * documentation \n */\n\n#ifndef NANOFLANN_HPP_\n#define NANOFLANN_HPP_\n\n#include \n#include \n#include \n#include // for abs()\n#include // for fwrite()\n#include // for abs()\n#include \n#include // std::reference_wrapper\n#include \n#include \n\n/** Library version: 0xMmP (M=Major,m=minor,P=patch) */\n#define NANOFLANN_VERSION 0x130\n\n// Avoid conflicting declaration of min/max macros in windows headers\n#if !defined(NOMINMAX) && \\\n (defined(_WIN32) || defined(_WIN32_) || defined(WIN32) || defined(_WIN64))\n#define NOMINMAX\n#ifdef max\n#undef max\n#undef min\n#endif\n#endif\n\nnamespace nanoflann {\n/** @addtogroup nanoflann_grp nanoflann C++ library for ANN\n * @{ */\n\n/** the PI constant (required to avoid MSVC missing symbols) */\ntemplate T pi_const() {\n return static_cast(3.14159265358979323846);\n}\n\n/**\n * Traits if object is resizable and assignable (typically has a resize | assign\n * method)\n */\ntemplate struct has_resize : std::false_type {};\n\ntemplate \nstruct has_resize().resize(1), 0)>\n : std::true_type {};\n\ntemplate struct has_assign : std::false_type {};\n\ntemplate \nstruct has_assign().assign(1, 0), 0)>\n : std::true_type {};\n\n/**\n * Free function to resize a resizable object\n */\ntemplate \ninline typename std::enable_if::value, void>::type\nresize(Container &c, const size_t nElements) {\n c.resize(nElements);\n}\n\n/**\n * Free function that has no effects on non resizable containers (e.g.\n * std::array) It raises an exception if the expected size does not match\n */\ntemplate \ninline typename std::enable_if::value, void>::type\nresize(Container &c, const size_t nElements) {\n if (nElements != c.size())\n throw std::logic_error(\"Try to change the size of a std::array.\");\n}\n\n/**\n * Free function to assign to a container\n */\ntemplate \ninline typename std::enable_if::value, void>::type\nassign(Container &c, const size_t nElements, const T &value) {\n c.assign(nElements, value);\n}\n\n/**\n * Free function to assign to a std::array\n */\ntemplate \ninline typename std::enable_if::value, void>::type\nassign(Container &c, const size_t nElements, const T &value) {\n for (size_t i = 0; i < nElements; i++)\n c[i] = value;\n}\n\n/** @addtogroup result_sets_grp Result set classes\n * @{ */\ntemplate \nclass KNNResultSet {\npublic:\n typedef _DistanceType DistanceType;\n typedef _IndexType IndexType;\n typedef _CountType CountType;\n\nprivate:\n IndexType *indices;\n DistanceType *dists;\n CountType capacity;\n CountType count;\n\npublic:\n inline KNNResultSet(CountType capacity_)\n : indices(0), dists(0), capacity(capacity_), count(0) {}\n\n inline void init(IndexType *indices_, DistanceType *dists_) {\n indices = indices_;\n dists = dists_;\n count = 0;\n if (capacity)\n dists[capacity - 1] = (std::numeric_limits::max)();\n }\n\n inline CountType size() const { return count; }\n\n inline bool full() const { return count == capacity; }\n\n /**\n * Called during search to add an element matching the criteria.\n * @return true if the search should be continued, false if the results are\n * sufficient\n */\n inline bool addPoint(DistanceType dist, IndexType index) {\n CountType i;\n for (i = count; i > 0; --i) {\n#ifdef NANOFLANN_FIRST_MATCH // If defined and two points have the same\n // distance, the one with the lowest-index will be\n // returned first.\n if ((dists[i - 1] > dist) ||\n ((dist == dists[i - 1]) && (indices[i - 1] > index))) {\n#else\n if (dists[i - 1] > dist) {\n#endif\n if (i < capacity) {\n dists[i] = dists[i - 1];\n indices[i] = indices[i - 1];\n }\n } else\n break;\n }\n if (i < capacity) {\n dists[i] = dist;\n indices[i] = index;\n }\n if (count < capacity)\n count++;\n\n // tell caller that the search shall continue\n return true;\n }\n\n inline DistanceType worstDist() const { return dists[capacity - 1]; }\n};\n\n/** operator \"<\" for std::sort() */\nstruct IndexDist_Sorter {\n /** PairType will be typically: std::pair */\n template \n inline bool operator()(const PairType &p1, const PairType &p2) const {\n return p1.second < p2.second;\n }\n};\n\n/**\n * A result-set class used when performing a radius based search.\n */\ntemplate \nclass RadiusResultSet {\npublic:\n typedef _DistanceType DistanceType;\n typedef _IndexType IndexType;\n\npublic:\n const DistanceType radius;\n\n std::vector> &m_indices_dists;\n\n inline RadiusResultSet(\n DistanceType radius_,\n std::vector> &indices_dists)\n : radius(radius_), m_indices_dists(indices_dists) {\n init();\n }\n\n inline void init() { clear(); }\n inline void clear() { m_indices_dists.clear(); }\n\n inline size_t size() const { return m_indices_dists.size(); }\n\n inline bool full() const { return true; }\n\n /**\n * Called during search to add an element matching the criteria.\n * @return true if the search should be continued, false if the results are\n * sufficient\n */\n inline bool addPoint(DistanceType dist, IndexType index) {\n if (dist < radius)\n m_indices_dists.push_back(std::make_pair(index, dist));\n return true;\n }\n\n inline DistanceType worstDist() const { return radius; }\n\n /**\n * Find the worst result (furtherest neighbor) without copying or sorting\n * Pre-conditions: size() > 0\n */\n std::pair worst_item() const {\n if (m_indices_dists.empty())\n throw std::runtime_error(\"Cannot invoke RadiusResultSet::worst_item() on \"\n \"an empty list of results.\");\n typedef\n typename std::vector>::const_iterator\n DistIt;\n DistIt it = std::max_element(m_indices_dists.begin(), m_indices_dists.end(),\n IndexDist_Sorter());\n return *it;\n }\n};\n\n/** @} */\n\n/** @addtogroup loadsave_grp Load/save auxiliary functions\n * @{ */\ntemplate \nvoid save_value(FILE *stream, const T &value, size_t count = 1) {\n fwrite(&value, sizeof(value), count, stream);\n}\n\ntemplate \nvoid save_value(FILE *stream, const std::vector &value) {\n size_t size = value.size();\n fwrite(&size, sizeof(size_t), 1, stream);\n fwrite(&value[0], sizeof(T), size, stream);\n}\n\ntemplate \nvoid load_value(FILE *stream, T &value, size_t count = 1) {\n size_t read_cnt = fread(&value, sizeof(value), count, stream);\n if (read_cnt != count) {\n throw std::runtime_error(\"Cannot read from file\");\n }\n}\n\ntemplate void load_value(FILE *stream, std::vector &value) {\n size_t size;\n size_t read_cnt = fread(&size, sizeof(size_t), 1, stream);\n if (read_cnt != 1) {\n throw std::runtime_error(\"Cannot read from file\");\n }\n value.resize(size);\n read_cnt = fread(&value[0], sizeof(T), size, stream);\n if (read_cnt != size) {\n throw std::runtime_error(\"Cannot read from file\");\n }\n}\n/** @} */\n\n/** @addtogroup metric_grp Metric (distance) classes\n * @{ */\n\nstruct Metric {};\n\n/** Manhattan distance functor (generic version, optimized for\n * high-dimensionality data sets). Corresponding distance traits:\n * nanoflann::metric_L1 \\tparam T Type of the elements (e.g. double, float,\n * uint8_t) \\tparam _DistanceType Type of distance variables (must be signed)\n * (e.g. float, double, int64_t)\n */\ntemplate \nstruct L1_Adaptor {\n typedef T ElementType;\n typedef _DistanceType DistanceType;\n\n const DataSource &data_source;\n\n L1_Adaptor(const DataSource &_data_source) : data_source(_data_source) {}\n\n inline DistanceType evalMetric(const T *a, const size_t b_idx, size_t size,\n DistanceType worst_dist = -1) const {\n DistanceType result = DistanceType();\n const T *last = a + size;\n const T *lastgroup = last - 3;\n size_t d = 0;\n\n /* Process 4 items with each loop for efficiency. */\n while (a < lastgroup) {\n const DistanceType diff0 =\n std::abs(a[0] - data_source.kdtree_get_pt(b_idx, d++));\n const DistanceType diff1 =\n std::abs(a[1] - data_source.kdtree_get_pt(b_idx, d++));\n const DistanceType diff2 =\n std::abs(a[2] - data_source.kdtree_get_pt(b_idx, d++));\n const DistanceType diff3 =\n std::abs(a[3] - data_source.kdtree_get_pt(b_idx, d++));\n result += diff0 + diff1 + diff2 + diff3;\n a += 4;\n if ((worst_dist > 0) && (result > worst_dist)) {\n return result;\n }\n }\n /* Process last 0-3 components. Not needed for standard vector lengths. */\n while (a < last) {\n result += std::abs(*a++ - data_source.kdtree_get_pt(b_idx, d++));\n }\n return result;\n }\n\n template \n inline DistanceType accum_dist(const U a, const V b, const size_t) const {\n return std::abs(a - b);\n }\n};\n\n/** Squared Euclidean distance functor (generic version, optimized for\n * high-dimensionality data sets). Corresponding distance traits:\n * nanoflann::metric_L2 \\tparam T Type of the elements (e.g. double, float,\n * uint8_t) \\tparam _DistanceType Type of distance variables (must be signed)\n * (e.g. float, double, int64_t)\n */\ntemplate \nstruct L2_Adaptor {\n typedef T ElementType;\n typedef _DistanceType DistanceType;\n\n const DataSource &data_source;\n\n L2_Adaptor(const DataSource &_data_source) : data_source(_data_source) {}\n\n inline DistanceType evalMetric(const T *a, const size_t b_idx, size_t size,\n DistanceType worst_dist = -1) const {\n DistanceType result = DistanceType();\n const T *last = a + size;\n const T *lastgroup = last - 3;\n size_t d = 0;\n\n /* Process 4 items with each loop for efficiency. */\n while (a < lastgroup) {\n const DistanceType diff0 = a[0] - data_source.kdtree_get_pt(b_idx, d++);\n const DistanceType diff1 = a[1] - data_source.kdtree_get_pt(b_idx, d++);\n const DistanceType diff2 = a[2] - data_source.kdtree_get_pt(b_idx, d++);\n const DistanceType diff3 = a[3] - data_source.kdtree_get_pt(b_idx, d++);\n result += diff0 * diff0 + diff1 * diff1 + diff2 * diff2 + diff3 * diff3;\n a += 4;\n if ((worst_dist > 0) && (result > worst_dist)) {\n return result;\n }\n }\n /* Process last 0-3 components. Not needed for standard vector lengths. */\n while (a < last) {\n const DistanceType diff0 = *a++ - data_source.kdtree_get_pt(b_idx, d++);\n result += diff0 * diff0;\n }\n return result;\n }\n\n template \n inline DistanceType accum_dist(const U a, const V b, const size_t) const {\n return (a - b) * (a - b);\n }\n};\n\n/** Squared Euclidean (L2) distance functor (suitable for low-dimensionality\n * datasets, like 2D or 3D point clouds) Corresponding distance traits:\n * nanoflann::metric_L2_Simple \\tparam T Type of the elements (e.g. double,\n * float, uint8_t) \\tparam _DistanceType Type of distance variables (must be\n * signed) (e.g. float, double, int64_t)\n */\ntemplate \nstruct L2_Simple_Adaptor {\n typedef T ElementType;\n typedef _DistanceType DistanceType;\n\n const DataSource &data_source;\n\n L2_Simple_Adaptor(const DataSource &_data_source)\n : data_source(_data_source) {}\n\n inline DistanceType evalMetric(const T *a, const size_t b_idx,\n size_t size) const {\n DistanceType result = DistanceType();\n for (size_t i = 0; i < size; ++i) {\n const DistanceType diff = a[i] - data_source.kdtree_get_pt(b_idx, i);\n result += diff * diff;\n }\n return result;\n }\n\n template \n inline DistanceType accum_dist(const U a, const V b, const size_t) const {\n return (a - b) * (a - b);\n }\n};\n\n/** SO2 distance functor\n * Corresponding distance traits: nanoflann::metric_SO2\n * \\tparam T Type of the elements (e.g. double, float)\n * \\tparam _DistanceType Type of distance variables (must be signed) (e.g.\n * float, double) orientation is constrained to be in [-pi, pi]\n */\ntemplate \nstruct SO2_Adaptor {\n typedef T ElementType;\n typedef _DistanceType DistanceType;\n\n const DataSource &data_source;\n\n SO2_Adaptor(const DataSource &_data_source) : data_source(_data_source) {}\n\n inline DistanceType evalMetric(const T *a, const size_t b_idx,\n size_t size) const {\n return accum_dist(a[size - 1], data_source.kdtree_get_pt(b_idx, size - 1),\n size - 1);\n }\n\n /** Note: this assumes that input angles are already in the range [-pi,pi] */\n template \n inline DistanceType accum_dist(const U a, const V b, const size_t) const {\n DistanceType result = DistanceType(), PI = pi_const();\n result = b - a;\n if (result > PI)\n result -= 2 * PI;\n else if (result < -PI)\n result += 2 * PI;\n return result;\n }\n};\n\n/** SO3 distance functor (Uses L2_Simple)\n * Corresponding distance traits: nanoflann::metric_SO3\n * \\tparam T Type of the elements (e.g. double, float)\n * \\tparam _DistanceType Type of distance variables (must be signed) (e.g.\n * float, double)\n */\ntemplate \nstruct SO3_Adaptor {\n typedef T ElementType;\n typedef _DistanceType DistanceType;\n\n L2_Simple_Adaptor distance_L2_Simple;\n\n SO3_Adaptor(const DataSource &_data_source)\n : distance_L2_Simple(_data_source) {}\n\n inline DistanceType evalMetric(const T *a, const size_t b_idx,\n size_t size) const {\n return distance_L2_Simple.evalMetric(a, b_idx, size);\n }\n\n template \n inline DistanceType accum_dist(const U a, const V b, const size_t idx) const {\n return distance_L2_Simple.accum_dist(a, b, idx);\n }\n};\n\n/** Metaprogramming helper traits class for the L1 (Manhattan) metric */\nstruct metric_L1 : public Metric {\n template struct traits {\n typedef L1_Adaptor distance_t;\n };\n};\n/** Metaprogramming helper traits class for the L2 (Euclidean) metric */\nstruct metric_L2 : public Metric {\n template struct traits {\n typedef L2_Adaptor distance_t;\n };\n};\n/** Metaprogramming helper traits class for the L2_simple (Euclidean) metric */\nstruct metric_L2_Simple : public Metric {\n template struct traits {\n typedef L2_Simple_Adaptor distance_t;\n };\n};\n/** Metaprogramming helper traits class for the SO3_InnerProdQuat metric */\nstruct metric_SO2 : public Metric {\n template struct traits {\n typedef SO2_Adaptor distance_t;\n };\n};\n/** Metaprogramming helper traits class for the SO3_InnerProdQuat metric */\nstruct metric_SO3 : public Metric {\n template struct traits {\n typedef SO3_Adaptor distance_t;\n };\n};\n\n/** @} */\n\n/** @addtogroup param_grp Parameter structs\n * @{ */\n\n/** Parameters (see README.md) */\nstruct KDTreeSingleIndexAdaptorParams {\n KDTreeSingleIndexAdaptorParams(size_t _leaf_max_size = 10)\n : leaf_max_size(_leaf_max_size) {}\n\n size_t leaf_max_size;\n};\n\n/** Search options for KDTreeSingleIndexAdaptor::findNeighbors() */\nstruct SearchParams {\n /** Note: The first argument (checks_IGNORED_) is ignored, but kept for\n * compatibility with the FLANN interface */\n SearchParams(int checks_IGNORED_ = 32, float eps_ = 0, bool sorted_ = true)\n : checks(checks_IGNORED_), eps(eps_), sorted(sorted_) {}\n\n int checks; //!< Ignored parameter (Kept for compatibility with the FLANN\n //!< interface).\n float eps; //!< search for eps-approximate neighbours (default: 0)\n bool sorted; //!< only for radius search, require neighbours sorted by\n //!< distance (default: true)\n};\n/** @} */\n\n/** @addtogroup memalloc_grp Memory allocation\n * @{ */\n\n/**\n * Allocates (using C's malloc) a generic type T.\n *\n * Params:\n * count = number of instances to allocate.\n * Returns: pointer (of type T*) to memory buffer\n */\ntemplate inline T *allocate(size_t count = 1) {\n T *mem = static_cast(::malloc(sizeof(T) * count));\n return mem;\n}\n\n/**\n * Pooled storage allocator\n *\n * The following routines allow for the efficient allocation of storage in\n * small chunks from a specified pool. Rather than allowing each structure\n * to be freed individually, an entire pool of storage is freed at once.\n * This method has two advantages over just using malloc() and free(). First,\n * it is far more efficient for allocating small objects, as there is\n * no overhead for remembering all the information needed to free each\n * object or consolidating fragmented memory. Second, the decision about\n * how long to keep an object is made at the time of allocation, and there\n * is no need to track down all the objects to free them.\n *\n */\n\nconst size_t WORDSIZE = 16;\nconst size_t BLOCKSIZE = 8192;\n\nclass PooledAllocator {\n /* We maintain memory alignment to word boundaries by requiring that all\n allocations be in multiples of the machine wordsize. */\n /* Size of machine word in bytes. Must be power of 2. */\n /* Minimum number of bytes requested at a time from\tthe system. Must be\n * multiple of WORDSIZE. */\n\n size_t remaining; /* Number of bytes left in current block of storage. */\n void *base; /* Pointer to base of current block of storage. */\n void *loc; /* Current location in block to next allocate memory. */\n\n void internal_init() {\n remaining = 0;\n base = NULL;\n usedMemory = 0;\n wastedMemory = 0;\n }\n\npublic:\n size_t usedMemory;\n size_t wastedMemory;\n\n /**\n Default constructor. Initializes a new pool.\n */\n PooledAllocator() { internal_init(); }\n\n /**\n * Destructor. Frees all the memory allocated in this pool.\n */\n ~PooledAllocator() { free_all(); }\n\n /** Frees all allocated memory chunks */\n void free_all() {\n while (base != NULL) {\n void *prev =\n *(static_cast(base)); /* Get pointer to prev block. */\n ::free(base);\n base = prev;\n }\n internal_init();\n }\n\n /**\n * Returns a pointer to a piece of new memory of the given size in bytes\n * allocated from the pool.\n */\n void *malloc(const size_t req_size) {\n /* Round size up to a multiple of wordsize. The following expression\n only works for WORDSIZE that is a power of 2, by masking last bits of\n incremented size to zero.\n */\n const size_t size = (req_size + (WORDSIZE - 1)) & ~(WORDSIZE - 1);\n\n /* Check whether a new block must be allocated. Note that the first word\n of a block is reserved for a pointer to the previous block.\n */\n if (size > remaining) {\n\n wastedMemory += remaining;\n\n /* Allocate new storage. */\n const size_t blocksize =\n (size + sizeof(void *) + (WORDSIZE - 1) > BLOCKSIZE)\n ? size + sizeof(void *) + (WORDSIZE - 1)\n : BLOCKSIZE;\n\n // use the standard C malloc to allocate memory\n void *m = ::malloc(blocksize);\n if (!m) {\n fprintf(stderr, \"Failed to allocate memory.\\n\");\n return NULL;\n }\n\n /* Fill first word of new block with pointer to previous block. */\n static_cast(m)[0] = base;\n base = m;\n\n size_t shift = 0;\n // int size_t = (WORDSIZE - ( (((size_t)m) + sizeof(void*)) &\n // (WORDSIZE-1))) & (WORDSIZE-1);\n\n remaining = blocksize - sizeof(void *) - shift;\n loc = (static_cast(m) + sizeof(void *) + shift);\n }\n void *rloc = loc;\n loc = static_cast(loc) + size;\n remaining -= size;\n\n usedMemory += size;\n\n return rloc;\n }\n\n /**\n * Allocates (using this pool) a generic type T.\n *\n * Params:\n * count = number of instances to allocate.\n * Returns: pointer (of type T*) to memory buffer\n */\n template T *allocate(const size_t count = 1) {\n T *mem = static_cast(this->malloc(sizeof(T) * count));\n return mem;\n }\n};\n/** @} */\n\n/** @addtogroup nanoflann_metaprog_grp Auxiliary metaprogramming stuff\n * @{ */\n\n/** Used to declare fixed-size arrays when DIM>0, dynamically-allocated vectors\n * when DIM=-1. Fixed size version for a generic DIM:\n */\ntemplate struct array_or_vector_selector {\n typedef std::array container_t;\n};\n/** Dynamic size version */\ntemplate struct array_or_vector_selector<-1, T> {\n typedef std::vector container_t;\n};\n\n/** @} */\n\n/** kd-tree base-class\n *\n * Contains the member functions common to the classes KDTreeSingleIndexAdaptor\n * and KDTreeSingleIndexDynamicAdaptor_.\n *\n * \\tparam Derived The name of the class which inherits this class.\n * \\tparam DatasetAdaptor The user-provided adaptor (see comments above).\n * \\tparam Distance The distance metric to use, these are all classes derived\n * from nanoflann::Metric \\tparam DIM Dimensionality of data points (e.g. 3 for\n * 3D points) \\tparam IndexType Will be typically size_t or int\n */\n\ntemplate \nclass KDTreeBaseClass {\n\npublic:\n /** Frees the previously-built index. Automatically called within\n * buildIndex(). */\n void freeIndex(Derived &obj) {\n obj.pool.free_all();\n obj.root_node = NULL;\n obj.m_size_at_index_build = 0;\n }\n\n typedef typename Distance::ElementType ElementType;\n typedef typename Distance::DistanceType DistanceType;\n\n /*--------------------- Internal Data Structures --------------------------*/\n struct Node {\n /** Union used because a node can be either a LEAF node or a non-leaf node,\n * so both data fields are never used simultaneously */\n union {\n struct leaf {\n IndexType left, right; //!< Indices of points in leaf node\n } lr;\n struct nonleaf {\n int divfeat; //!< Dimension used for subdivision.\n DistanceType divlow, divhigh; //!< The values used for subdivision.\n } sub;\n } node_type;\n Node *child1, *child2; //!< Child nodes (both=NULL mean its a leaf node)\n };\n\n typedef Node *NodePtr;\n\n struct Interval {\n ElementType low, high;\n };\n\n /**\n * Array of indices to vectors in the dataset.\n */\n std::vector vind;\n\n NodePtr root_node;\n\n size_t m_leaf_max_size;\n\n size_t m_size; //!< Number of current points in the dataset\n size_t m_size_at_index_build; //!< Number of points in the dataset when the\n //!< index was built\n int dim; //!< Dimensionality of each data point\n\n /** Define \"BoundingBox\" as a fixed-size or variable-size container depending\n * on \"DIM\" */\n typedef\n typename array_or_vector_selector::container_t BoundingBox;\n\n /** Define \"distance_vector_t\" as a fixed-size or variable-size container\n * depending on \"DIM\" */\n typedef typename array_or_vector_selector::container_t\n distance_vector_t;\n\n /** The KD-tree used to find neighbours */\n\n BoundingBox root_bbox;\n\n /**\n * Pooled memory allocator.\n *\n * Using a pooled memory allocator is more efficient\n * than allocating memory directly when there is a large\n * number small of memory allocations.\n */\n PooledAllocator pool;\n\n /** Returns number of points in dataset */\n size_t size(const Derived &obj) const { return obj.m_size; }\n\n /** Returns the length of each point in the dataset */\n size_t veclen(const Derived &obj) {\n return static_cast(DIM > 0 ? DIM : obj.dim);\n }\n\n /// Helper accessor to the dataset points:\n inline ElementType dataset_get(const Derived &obj, size_t idx,\n int component) const {\n return obj.dataset.kdtree_get_pt(idx, component);\n }\n\n /**\n * Computes the inde memory usage\n * Returns: memory used by the index\n */\n size_t usedMemory(Derived &obj) {\n return obj.pool.usedMemory + obj.pool.wastedMemory +\n obj.dataset.kdtree_get_point_count() *\n sizeof(IndexType); // pool memory and vind array memory\n }\n\n void computeMinMax(const Derived &obj, IndexType *ind, IndexType count,\n int element, ElementType &min_elem,\n ElementType &max_elem) {\n min_elem = dataset_get(obj, ind[0], element);\n max_elem = dataset_get(obj, ind[0], element);\n for (IndexType i = 1; i < count; ++i) {\n ElementType val = dataset_get(obj, ind[i], element);\n if (val < min_elem)\n min_elem = val;\n if (val > max_elem)\n max_elem = val;\n }\n }\n\n /**\n * Create a tree node that subdivides the list of vecs from vind[first]\n * to vind[last]. The routine is called recursively on each sublist.\n *\n * @param left index of the first vector\n * @param right index of the last vector\n */\n NodePtr divideTree(Derived &obj, const IndexType left, const IndexType right,\n BoundingBox &bbox) {\n NodePtr node = obj.pool.template allocate(); // allocate memory\n\n /* If too few exemplars remain, then make this a leaf node. */\n if ((right - left) <= static_cast(obj.m_leaf_max_size)) {\n node->child1 = node->child2 = NULL; /* Mark as leaf node. */\n node->node_type.lr.left = left;\n node->node_type.lr.right = right;\n\n // compute bounding-box of leaf points\n for (int i = 0; i < (DIM > 0 ? DIM : obj.dim); ++i) {\n bbox[i].low = dataset_get(obj, obj.vind[left], i);\n bbox[i].high = dataset_get(obj, obj.vind[left], i);\n }\n for (IndexType k = left + 1; k < right; ++k) {\n for (int i = 0; i < (DIM > 0 ? DIM : obj.dim); ++i) {\n if (bbox[i].low > dataset_get(obj, obj.vind[k], i))\n bbox[i].low = dataset_get(obj, obj.vind[k], i);\n if (bbox[i].high < dataset_get(obj, obj.vind[k], i))\n bbox[i].high = dataset_get(obj, obj.vind[k], i);\n }\n }\n } else {\n IndexType idx;\n int cutfeat;\n DistanceType cutval;\n middleSplit_(obj, &obj.vind[0] + left, right - left, idx, cutfeat, cutval,\n bbox);\n\n node->node_type.sub.divfeat = cutfeat;\n\n BoundingBox left_bbox(bbox);\n left_bbox[cutfeat].high = cutval;\n node->child1 = divideTree(obj, left, left + idx, left_bbox);\n\n BoundingBox right_bbox(bbox);\n right_bbox[cutfeat].low = cutval;\n node->child2 = divideTree(obj, left + idx, right, right_bbox);\n\n node->node_type.sub.divlow = left_bbox[cutfeat].high;\n node->node_type.sub.divhigh = right_bbox[cutfeat].low;\n\n for (int i = 0; i < (DIM > 0 ? DIM : obj.dim); ++i) {\n bbox[i].low = std::min(left_bbox[i].low, right_bbox[i].low);\n bbox[i].high = std::max(left_bbox[i].high, right_bbox[i].high);\n }\n }\n\n return node;\n }\n\n void middleSplit_(Derived &obj, IndexType *ind, IndexType count,\n IndexType &index, int &cutfeat, DistanceType &cutval,\n const BoundingBox &bbox) {\n const DistanceType EPS = static_cast(0.00001);\n ElementType max_span = bbox[0].high - bbox[0].low;\n for (int i = 1; i < (DIM > 0 ? DIM : obj.dim); ++i) {\n ElementType span = bbox[i].high - bbox[i].low;\n if (span > max_span) {\n max_span = span;\n }\n }\n ElementType max_spread = -1;\n cutfeat = 0;\n for (int i = 0; i < (DIM > 0 ? DIM : obj.dim); ++i) {\n ElementType span = bbox[i].high - bbox[i].low;\n if (span > (1 - EPS) * max_span) {\n ElementType min_elem, max_elem;\n computeMinMax(obj, ind, count, i, min_elem, max_elem);\n ElementType spread = max_elem - min_elem;\n ;\n if (spread > max_spread) {\n cutfeat = i;\n max_spread = spread;\n }\n }\n }\n // split in the middle\n DistanceType split_val = (bbox[cutfeat].low + bbox[cutfeat].high) / 2;\n ElementType min_elem, max_elem;\n computeMinMax(obj, ind, count, cutfeat, min_elem, max_elem);\n\n if (split_val < min_elem)\n cutval = min_elem;\n else if (split_val > max_elem)\n cutval = max_elem;\n else\n cutval = split_val;\n\n IndexType lim1, lim2;\n planeSplit(obj, ind, count, cutfeat, cutval, lim1, lim2);\n\n if (lim1 > count / 2)\n index = lim1;\n else if (lim2 < count / 2)\n index = lim2;\n else\n index = count / 2;\n }\n\n /**\n * Subdivide the list of points by a plane perpendicular on axe corresponding\n * to the 'cutfeat' dimension at 'cutval' position.\n *\n * On return:\n * dataset[ind[0..lim1-1]][cutfeat]cutval\n */\n void planeSplit(Derived &obj, IndexType *ind, const IndexType count,\n int cutfeat, DistanceType &cutval, IndexType &lim1,\n IndexType &lim2) {\n /* Move vector indices for left subtree to front of list. */\n IndexType left = 0;\n IndexType right = count - 1;\n for (;;) {\n while (left <= right && dataset_get(obj, ind[left], cutfeat) < cutval)\n ++left;\n while (right && left <= right &&\n dataset_get(obj, ind[right], cutfeat) >= cutval)\n --right;\n if (left > right || !right)\n break; // \"!right\" was added to support unsigned Index types\n std::swap(ind[left], ind[right]);\n ++left;\n --right;\n }\n /* If either list is empty, it means that all remaining features\n * are identical. Split in the middle to maintain a balanced tree.\n */\n lim1 = left;\n right = count - 1;\n for (;;) {\n while (left <= right && dataset_get(obj, ind[left], cutfeat) <= cutval)\n ++left;\n while (right && left <= right &&\n dataset_get(obj, ind[right], cutfeat) > cutval)\n --right;\n if (left > right || !right)\n break; // \"!right\" was added to support unsigned Index types\n std::swap(ind[left], ind[right]);\n ++left;\n --right;\n }\n lim2 = left;\n }\n\n DistanceType computeInitialDistances(const Derived &obj,\n const ElementType *vec,\n distance_vector_t &dists) const {\n assert(vec);\n DistanceType distsq = DistanceType();\n\n for (int i = 0; i < (DIM > 0 ? DIM : obj.dim); ++i) {\n if (vec[i] < obj.root_bbox[i].low) {\n dists[i] = obj.distance.accum_dist(vec[i], obj.root_bbox[i].low, i);\n distsq += dists[i];\n }\n if (vec[i] > obj.root_bbox[i].high) {\n dists[i] = obj.distance.accum_dist(vec[i], obj.root_bbox[i].high, i);\n distsq += dists[i];\n }\n }\n return distsq;\n }\n\n void save_tree(Derived &obj, FILE *stream, NodePtr tree) {\n save_value(stream, *tree);\n if (tree->child1 != NULL) {\n save_tree(obj, stream, tree->child1);\n }\n if (tree->child2 != NULL) {\n save_tree(obj, stream, tree->child2);\n }\n }\n\n void load_tree(Derived &obj, FILE *stream, NodePtr &tree) {\n tree = obj.pool.template allocate();\n load_value(stream, *tree);\n if (tree->child1 != NULL) {\n load_tree(obj, stream, tree->child1);\n }\n if (tree->child2 != NULL) {\n load_tree(obj, stream, tree->child2);\n }\n }\n\n /** Stores the index in a binary file.\n * IMPORTANT NOTE: The set of data points is NOT stored in the file, so when\n * loading the index object it must be constructed associated to the same\n * source of data points used while building it. See the example:\n * examples/saveload_example.cpp \\sa loadIndex */\n void saveIndex_(Derived &obj, FILE *stream) {\n save_value(stream, obj.m_size);\n save_value(stream, obj.dim);\n save_value(stream, obj.root_bbox);\n save_value(stream, obj.m_leaf_max_size);\n save_value(stream, obj.vind);\n save_tree(obj, stream, obj.root_node);\n }\n\n /** Loads a previous index from a binary file.\n * IMPORTANT NOTE: The set of data points is NOT stored in the file, so the\n * index object must be constructed associated to the same source of data\n * points used while building the index. See the example:\n * examples/saveload_example.cpp \\sa loadIndex */\n void loadIndex_(Derived &obj, FILE *stream) {\n load_value(stream, obj.m_size);\n load_value(stream, obj.dim);\n load_value(stream, obj.root_bbox);\n load_value(stream, obj.m_leaf_max_size);\n load_value(stream, obj.vind);\n load_tree(obj, stream, obj.root_node);\n }\n};\n\n/** @addtogroup kdtrees_grp KD-tree classes and adaptors\n * @{ */\n\n/** kd-tree static index\n *\n * Contains the k-d trees and other information for indexing a set of points\n * for nearest-neighbor matching.\n *\n * The class \"DatasetAdaptor\" must provide the following interface (can be\n * non-virtual, inlined methods):\n *\n * \\code\n * // Must return the number of data poins\n * inline size_t kdtree_get_point_count() const { ... }\n *\n *\n * // Must return the dim'th component of the idx'th point in the class:\n * inline T kdtree_get_pt(const size_t idx, const size_t dim) const { ... }\n *\n * // Optional bounding-box computation: return false to default to a standard\n * bbox computation loop.\n * // Return true if the BBOX was already computed by the class and returned\n * in \"bb\" so it can be avoided to redo it again.\n * // Look at bb.size() to find out the expected dimensionality (e.g. 2 or 3\n * for point clouds) template bool kdtree_get_bbox(BBOX &bb) const\n * {\n * bb[0].low = ...; bb[0].high = ...; // 0th dimension limits\n * bb[1].low = ...; bb[1].high = ...; // 1st dimension limits\n * ...\n * return true;\n * }\n *\n * \\endcode\n *\n * \\tparam DatasetAdaptor The user-provided adaptor (see comments above).\n * \\tparam Distance The distance metric to use: nanoflann::metric_L1,\n * nanoflann::metric_L2, nanoflann::metric_L2_Simple, etc. \\tparam DIM\n * Dimensionality of data points (e.g. 3 for 3D points) \\tparam IndexType Will\n * be typically size_t or int\n */\ntemplate \nclass KDTreeSingleIndexAdaptor\n : public KDTreeBaseClass<\n KDTreeSingleIndexAdaptor,\n Distance, DatasetAdaptor, DIM, IndexType> {\npublic:\n /** Deleted copy constructor*/\n KDTreeSingleIndexAdaptor(\n const KDTreeSingleIndexAdaptor\n &) = delete;\n\n /**\n * The dataset used by this index\n */\n const DatasetAdaptor &dataset; //!< The source of our data\n\n const KDTreeSingleIndexAdaptorParams index_params;\n\n Distance distance;\n\n typedef typename nanoflann::KDTreeBaseClass<\n nanoflann::KDTreeSingleIndexAdaptor,\n Distance, DatasetAdaptor, DIM, IndexType>\n BaseClassRef;\n\n typedef typename BaseClassRef::ElementType ElementType;\n typedef typename BaseClassRef::DistanceType DistanceType;\n\n typedef typename BaseClassRef::Node Node;\n typedef Node *NodePtr;\n\n typedef typename BaseClassRef::Interval Interval;\n /** Define \"BoundingBox\" as a fixed-size or variable-size container depending\n * on \"DIM\" */\n typedef typename BaseClassRef::BoundingBox BoundingBox;\n\n /** Define \"distance_vector_t\" as a fixed-size or variable-size container\n * depending on \"DIM\" */\n typedef typename BaseClassRef::distance_vector_t distance_vector_t;\n\n /**\n * KDTree constructor\n *\n * Refer to docs in README.md or online in\n * https://github.com/jlblancoc/nanoflann\n *\n * The KD-Tree point dimension (the length of each point in the datase, e.g. 3\n * for 3D points) is determined by means of:\n * - The \\a DIM template parameter if >0 (highest priority)\n * - Otherwise, the \\a dimensionality parameter of this constructor.\n *\n * @param inputData Dataset with the input features\n * @param params Basically, the maximum leaf node size\n */\n KDTreeSingleIndexAdaptor(const int dimensionality,\n const DatasetAdaptor &inputData,\n const KDTreeSingleIndexAdaptorParams ¶ms =\n KDTreeSingleIndexAdaptorParams())\n : dataset(inputData), index_params(params), distance(inputData) {\n BaseClassRef::root_node = NULL;\n BaseClassRef::m_size = dataset.kdtree_get_point_count();\n BaseClassRef::m_size_at_index_build = BaseClassRef::m_size;\n BaseClassRef::dim = dimensionality;\n if (DIM > 0)\n BaseClassRef::dim = DIM;\n BaseClassRef::m_leaf_max_size = params.leaf_max_size;\n\n // Create a permutable array of indices to the input vectors.\n init_vind();\n }\n\n /**\n * Builds the index\n */\n void buildIndex() {\n BaseClassRef::m_size = dataset.kdtree_get_point_count();\n BaseClassRef::m_size_at_index_build = BaseClassRef::m_size;\n init_vind();\n this->freeIndex(*this);\n BaseClassRef::m_size_at_index_build = BaseClassRef::m_size;\n if (BaseClassRef::m_size == 0)\n return;\n computeBoundingBox(BaseClassRef::root_bbox);\n BaseClassRef::root_node =\n this->divideTree(*this, 0, BaseClassRef::m_size,\n BaseClassRef::root_bbox); // construct the tree\n }\n\n /** \\name Query methods\n * @{ */\n\n /**\n * Find set of nearest neighbors to vec[0:dim-1]. Their indices are stored\n * inside the result object.\n *\n * Params:\n * result = the result object in which the indices of the\n * nearest-neighbors are stored vec = the vector for which to search the\n * nearest neighbors\n *\n * \\tparam RESULTSET Should be any ResultSet\n * \\return True if the requested neighbors could be found.\n * \\sa knnSearch, radiusSearch\n */\n template \n bool findNeighbors(RESULTSET &result, const ElementType *vec,\n const SearchParams &searchParams) const {\n assert(vec);\n if (this->size(*this) == 0)\n return false;\n if (!BaseClassRef::root_node)\n throw std::runtime_error(\n \"[nanoflann] findNeighbors() called before building the index.\");\n float epsError = 1 + searchParams.eps;\n\n distance_vector_t\n dists; // fixed or variable-sized container (depending on DIM)\n auto zero = static_cast(0);\n assign(dists, (DIM > 0 ? DIM : BaseClassRef::dim),\n zero); // Fill it with zeros.\n DistanceType distsq = this->computeInitialDistances(*this, vec, dists);\n\n searchLevel(result, vec, BaseClassRef::root_node, distsq, dists,\n epsError); // \"count_leaf\" parameter removed since was neither\n // used nor returned to the user.\n\n return result.full();\n }\n\n /**\n * Find the \"num_closest\" nearest neighbors to the \\a query_point[0:dim-1].\n * Their indices are stored inside the result object. \\sa radiusSearch,\n * findNeighbors \\note nChecks_IGNORED is ignored but kept for compatibility\n * with the original FLANN interface. \\return Number `N` of valid points in\n * the result set. Only the first `N` entries in `out_indices` and\n * `out_distances_sq` will be valid. Return may be less than `num_closest`\n * only if the number of elements in the tree is less than `num_closest`.\n */\n size_t knnSearch(const ElementType *query_point, const size_t num_closest,\n IndexType *out_indices, DistanceType *out_distances_sq,\n const int /* nChecks_IGNORED */ = 10) const {\n nanoflann::KNNResultSet resultSet(num_closest);\n resultSet.init(out_indices, out_distances_sq);\n this->findNeighbors(resultSet, query_point, nanoflann::SearchParams());\n return resultSet.size();\n }\n\n /**\n * Find all the neighbors to \\a query_point[0:dim-1] within a maximum radius.\n * The output is given as a vector of pairs, of which the first element is a\n * point index and the second the corresponding distance. Previous contents of\n * \\a IndicesDists are cleared.\n *\n * If searchParams.sorted==true, the output list is sorted by ascending\n * distances.\n *\n * For a better performance, it is advisable to do a .reserve() on the vector\n * if you have any wild guess about the number of expected matches.\n *\n * \\sa knnSearch, findNeighbors, radiusSearchCustomCallback\n * \\return The number of points within the given radius (i.e. indices.size()\n * or dists.size() )\n */\n size_t\n radiusSearch(const ElementType *query_point, const DistanceType &radius,\n std::vector> &IndicesDists,\n const SearchParams &searchParams) const {\n RadiusResultSet resultSet(radius, IndicesDists);\n const size_t nFound =\n radiusSearchCustomCallback(query_point, resultSet, searchParams);\n if (searchParams.sorted)\n std::sort(IndicesDists.begin(), IndicesDists.end(), IndexDist_Sorter());\n return nFound;\n }\n\n /**\n * Just like radiusSearch() but with a custom callback class for each point\n * found in the radius of the query. See the source of RadiusResultSet<> as a\n * start point for your own classes. \\sa radiusSearch\n */\n template \n size_t radiusSearchCustomCallback(\n const ElementType *query_point, SEARCH_CALLBACK &resultSet,\n const SearchParams &searchParams = SearchParams()) const {\n this->findNeighbors(resultSet, query_point, searchParams);\n return resultSet.size();\n }\n\n /** @} */\n\npublic:\n /** Make sure the auxiliary list \\a vind has the same size than the current\n * dataset, and re-generate if size has changed. */\n void init_vind() {\n // Create a permutable array of indices to the input vectors.\n BaseClassRef::m_size = dataset.kdtree_get_point_count();\n if (BaseClassRef::vind.size() != BaseClassRef::m_size)\n BaseClassRef::vind.resize(BaseClassRef::m_size);\n for (size_t i = 0; i < BaseClassRef::m_size; i++)\n BaseClassRef::vind[i] = i;\n }\n\n void computeBoundingBox(BoundingBox &bbox) {\n resize(bbox, (DIM > 0 ? DIM : BaseClassRef::dim));\n if (dataset.kdtree_get_bbox(bbox)) {\n // Done! It was implemented in derived class\n } else {\n const size_t N = dataset.kdtree_get_point_count();\n if (!N)\n throw std::runtime_error(\"[nanoflann] computeBoundingBox() called but \"\n \"no data points found.\");\n for (int i = 0; i < (DIM > 0 ? DIM : BaseClassRef::dim); ++i) {\n bbox[i].low = bbox[i].high = this->dataset_get(*this, 0, i);\n }\n for (size_t k = 1; k < N; ++k) {\n for (int i = 0; i < (DIM > 0 ? DIM : BaseClassRef::dim); ++i) {\n if (this->dataset_get(*this, k, i) < bbox[i].low)\n bbox[i].low = this->dataset_get(*this, k, i);\n if (this->dataset_get(*this, k, i) > bbox[i].high)\n bbox[i].high = this->dataset_get(*this, k, i);\n }\n }\n }\n }\n\n /**\n * Performs an exact search in the tree starting from a node.\n * \\tparam RESULTSET Should be any ResultSet\n * \\return true if the search should be continued, false if the results are\n * sufficient\n */\n template \n bool searchLevel(RESULTSET &result_set, const ElementType *vec,\n const NodePtr node, DistanceType mindistsq,\n distance_vector_t &dists, const float epsError) const {\n /* If this is a leaf node, then do check and return. */\n if ((node->child1 == NULL) && (node->child2 == NULL)) {\n // count_leaf += (node->lr.right-node->lr.left); // Removed since was\n // neither used nor returned to the user.\n DistanceType worst_dist = result_set.worstDist();\n for (IndexType i = node->node_type.lr.left; i < node->node_type.lr.right;\n ++i) {\n const IndexType index = BaseClassRef::vind[i]; // reorder... : i;\n DistanceType dist = distance.evalMetric(\n vec, index, (DIM > 0 ? DIM : BaseClassRef::dim));\n if (dist < worst_dist) {\n if (!result_set.addPoint(dist, BaseClassRef::vind[i])) {\n // the resultset doesn't want to receive any more points, we're done\n // searching!\n return false;\n }\n }\n }\n return true;\n }\n\n /* Which child branch should be taken first? */\n int idx = node->node_type.sub.divfeat;\n ElementType val = vec[idx];\n DistanceType diff1 = val - node->node_type.sub.divlow;\n DistanceType diff2 = val - node->node_type.sub.divhigh;\n\n NodePtr bestChild;\n NodePtr otherChild;\n DistanceType cut_dist;\n if ((diff1 + diff2) < 0) {\n bestChild = node->child1;\n otherChild = node->child2;\n cut_dist = distance.accum_dist(val, node->node_type.sub.divhigh, idx);\n } else {\n bestChild = node->child2;\n otherChild = node->child1;\n cut_dist = distance.accum_dist(val, node->node_type.sub.divlow, idx);\n }\n\n /* Call recursively to search next level down. */\n if (!searchLevel(result_set, vec, bestChild, mindistsq, dists, epsError)) {\n // the resultset doesn't want to receive any more points, we're done\n // searching!\n return false;\n }\n\n DistanceType dst = dists[idx];\n mindistsq = mindistsq + cut_dist - dst;\n dists[idx] = cut_dist;\n if (mindistsq * epsError <= result_set.worstDist()) {\n if (!searchLevel(result_set, vec, otherChild, mindistsq, dists,\n epsError)) {\n // the resultset doesn't want to receive any more points, we're done\n // searching!\n return false;\n }\n }\n dists[idx] = dst;\n return true;\n }\n\npublic:\n /** Stores the index in a binary file.\n * IMPORTANT NOTE: The set of data points is NOT stored in the file, so when\n * loading the index object it must be constructed associated to the same\n * source of data points used while building it. See the example:\n * examples/saveload_example.cpp \\sa loadIndex */\n void saveIndex(FILE *stream) { this->saveIndex_(*this, stream); }\n\n /** Loads a previous index from a binary file.\n * IMPORTANT NOTE: The set of data points is NOT stored in the file, so the\n * index object must be constructed associated to the same source of data\n * points used while building the index. See the example:\n * examples/saveload_example.cpp \\sa loadIndex */\n void loadIndex(FILE *stream) { this->loadIndex_(*this, stream); }\n\n}; // class KDTree\n\n/** kd-tree dynamic index\n *\n * Contains the k-d trees and other information for indexing a set of points\n * for nearest-neighbor matching.\n *\n * The class \"DatasetAdaptor\" must provide the following interface (can be\n * non-virtual, inlined methods):\n *\n * \\code\n * // Must return the number of data poins\n * inline size_t kdtree_get_point_count() const { ... }\n *\n * // Must return the dim'th component of the idx'th point in the class:\n * inline T kdtree_get_pt(const size_t idx, const size_t dim) const { ... }\n *\n * // Optional bounding-box computation: return false to default to a standard\n * bbox computation loop.\n * // Return true if the BBOX was already computed by the class and returned\n * in \"bb\" so it can be avoided to redo it again.\n * // Look at bb.size() to find out the expected dimensionality (e.g. 2 or 3\n * for point clouds) template bool kdtree_get_bbox(BBOX &bb) const\n * {\n * bb[0].low = ...; bb[0].high = ...; // 0th dimension limits\n * bb[1].low = ...; bb[1].high = ...; // 1st dimension limits\n * ...\n * return true;\n * }\n *\n * \\endcode\n *\n * \\tparam DatasetAdaptor The user-provided adaptor (see comments above).\n * \\tparam Distance The distance metric to use: nanoflann::metric_L1,\n * nanoflann::metric_L2, nanoflann::metric_L2_Simple, etc. \\tparam DIM\n * Dimensionality of data points (e.g. 3 for 3D points) \\tparam IndexType Will\n * be typically size_t or int\n */\ntemplate \nclass KDTreeSingleIndexDynamicAdaptor_\n : public KDTreeBaseClass,\n Distance, DatasetAdaptor, DIM, IndexType> {\npublic:\n /**\n * The dataset used by this index\n */\n const DatasetAdaptor &dataset; //!< The source of our data\n\n KDTreeSingleIndexAdaptorParams index_params;\n\n std::vector &treeIndex;\n\n Distance distance;\n\n typedef typename nanoflann::KDTreeBaseClass<\n nanoflann::KDTreeSingleIndexDynamicAdaptor_,\n Distance, DatasetAdaptor, DIM, IndexType>\n BaseClassRef;\n\n typedef typename BaseClassRef::ElementType ElementType;\n typedef typename BaseClassRef::DistanceType DistanceType;\n\n typedef typename BaseClassRef::Node Node;\n typedef Node *NodePtr;\n\n typedef typename BaseClassRef::Interval Interval;\n /** Define \"BoundingBox\" as a fixed-size or variable-size container depending\n * on \"DIM\" */\n typedef typename BaseClassRef::BoundingBox BoundingBox;\n\n /** Define \"distance_vector_t\" as a fixed-size or variable-size container\n * depending on \"DIM\" */\n typedef typename BaseClassRef::distance_vector_t distance_vector_t;\n\n /**\n * KDTree constructor\n *\n * Refer to docs in README.md or online in\n * https://github.com/jlblancoc/nanoflann\n *\n * The KD-Tree point dimension (the length of each point in the datase, e.g. 3\n * for 3D points) is determined by means of:\n * - The \\a DIM template parameter if >0 (highest priority)\n * - Otherwise, the \\a dimensionality parameter of this constructor.\n *\n * @param inputData Dataset with the input features\n * @param params Basically, the maximum leaf node size\n */\n KDTreeSingleIndexDynamicAdaptor_(\n const int dimensionality, const DatasetAdaptor &inputData,\n std::vector &treeIndex_,\n const KDTreeSingleIndexAdaptorParams ¶ms =\n KDTreeSingleIndexAdaptorParams())\n : dataset(inputData), index_params(params), treeIndex(treeIndex_),\n distance(inputData) {\n BaseClassRef::root_node = NULL;\n BaseClassRef::m_size = 0;\n BaseClassRef::m_size_at_index_build = 0;\n BaseClassRef::dim = dimensionality;\n if (DIM > 0)\n BaseClassRef::dim = DIM;\n BaseClassRef::m_leaf_max_size = params.leaf_max_size;\n }\n\n /** Assignment operator definiton */\n KDTreeSingleIndexDynamicAdaptor_\n operator=(const KDTreeSingleIndexDynamicAdaptor_ &rhs) {\n KDTreeSingleIndexDynamicAdaptor_ tmp(rhs);\n std::swap(BaseClassRef::vind, tmp.BaseClassRef::vind);\n std::swap(BaseClassRef::m_leaf_max_size, tmp.BaseClassRef::m_leaf_max_size);\n std::swap(index_params, tmp.index_params);\n std::swap(treeIndex, tmp.treeIndex);\n std::swap(BaseClassRef::m_size, tmp.BaseClassRef::m_size);\n std::swap(BaseClassRef::m_size_at_index_build,\n tmp.BaseClassRef::m_size_at_index_build);\n std::swap(BaseClassRef::root_node, tmp.BaseClassRef::root_node);\n std::swap(BaseClassRef::root_bbox, tmp.BaseClassRef::root_bbox);\n std::swap(BaseClassRef::pool, tmp.BaseClassRef::pool);\n return *this;\n }\n\n /**\n * Builds the index\n */\n void buildIndex() {\n BaseClassRef::m_size = BaseClassRef::vind.size();\n this->freeIndex(*this);\n BaseClassRef::m_size_at_index_build = BaseClassRef::m_size;\n if (BaseClassRef::m_size == 0)\n return;\n computeBoundingBox(BaseClassRef::root_bbox);\n BaseClassRef::root_node =\n this->divideTree(*this, 0, BaseClassRef::m_size,\n BaseClassRef::root_bbox); // construct the tree\n }\n\n /** \\name Query methods\n * @{ */\n\n /**\n * Find set of nearest neighbors to vec[0:dim-1]. Their indices are stored\n * inside the result object.\n *\n * Params:\n * result = the result object in which the indices of the\n * nearest-neighbors are stored vec = the vector for which to search the\n * nearest neighbors\n *\n * \\tparam RESULTSET Should be any ResultSet\n * \\return True if the requested neighbors could be found.\n * \\sa knnSearch, radiusSearch\n */\n template ![\"Texas](\"images/tamu_logo.png\"){kind=logo}
![\"The](\"images/arl_logo.png\"){kind=logo}
![](\"./resources/gscnn.gif\")
\n ![](\"./resources/intro.jpg\")
![](\"./resources/top.jpg\"/)
![](\"./resources/seg.jpg\")
![](\"./resources/edges.jpg\")
![](\"./resources/table.png\")
![](\"./resources/crop.jpg\")
![\"hrnet\"](\"./figures/seg-hrnet.png\" "\\"\\"")
\n\n![\"OCR\"](\"./figures/OCR.PNG\" "\\"\\"")
\n![](\"./resources/architecture.jpg\"/)