Repository: AnTao97/PointCloudDatasets
Branch: master
Commit: 54b1bfd4f53a
Files: 5
Total size: 19.1 KB
Directory structure:
gitextract_c1bfgm6a/
├── .gitignore
├── LICENSE
├── README.md
├── dataset.py
└── visualize.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
.DS_Store
*/.DS_Store
================================================
FILE: LICENSE
================================================
MIT License
Copyright (c) 2019 An Tao
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: README.md
================================================
# Point Cloud Datasets
This repository provides ShapeNetCore.v2, ShapeNetPart, ModelNet40 and ModelNet10 datasets in HDF5 format. For each shape in these datasets, we use farthest point sampling algorithm to uniformly sample 2,048 points from shape surface. All points are then centered and scaled. We follow the train/val/test split in official documents.
We also provide code to load and visualize our datasets with PyTorch 1.2 and Python 3.7. See `dataset.py` and run it to have a try.
To visualize, run `visualize.py` to generate XML file and use [Mitsuba](https://www.mitsuba-renderer.org/index.html) to render it. Our code is from this [repo](https://github.com/zekunhao1995/PointFlowRenderer).
## Download link:
- ShapeNetCore.v2 (0.98G) [[TsinghuaCloud]](https://cloud.tsinghua.edu.cn/f/06a3c383dc474179b97d/) [[BaiduDisk]](https://pan.baidu.com/s/154As2kzHZczMipuoZIc0kg)
- ShapeNetPart (338M) [[TsinghuaCloud]](https://cloud.tsinghua.edu.cn/f/c25d94e163454196a26b/) [[BaiduDisk]](https://pan.baidu.com/s/1yi4bMVBE2mV8NqVRtNLoqw)
- ModelNet40 (194M) [[TsinghuaCloud]](https://cloud.tsinghua.edu.cn/f/b3d9fe3e2a514def8097/) [[BaiduDisk]](https://pan.baidu.com/s/1NQZgN8tvHVqQntxefcdVAg)
- ModelNet10 (72.5M) [[TsinghuaCloud]](https://cloud.tsinghua.edu.cn/f/5414376f6afd41ce9b6d/) [[BaiduDisk]](https://pan.baidu.com/s/1tfnKQ_yg3SfIgyLSwQ2E0g)
## ShapeNetCore.v2
ShapeNetCore.v2 datset contains 51,127 pre-aligned shapes from 55 categories, which are split into 35,708 (70%) for training, 5,158 (10%) shapes for validation and 10,261 (20%) shapes for testing. In official document there should be 51,190 shapes in total, but 63 shapes are missing in original downloaded ShapeNetCore.v2 dataset from [here](https://www.shapenet.org/download/shapenetcore).
The 55 categories include: `airplane`, `bag`, `basket`, `bathtub`, `bed`, `bench`, `birdhouse`, `bookshelf`, `bottle`, `bowl`, `bus`, `cabinet`, `camera`, `can`, `cap`, `car`, `cellphone`, `chair`, `clock`, `dishwasher`, `earphone`, `faucet`, `file`, `guitar`, `helmet`, `jar`, `keyboard`, `knife`, `lamp`, `laptop`, `mailbox`, `microphone`, `microwave`, `monitor`, `motorcycle`, `mug`, `piano`, `pillow`, `pistol`, `pot`, `printer`, `remote_control`, `rifle`, `rocket`, `skateboard`, `sofa`, `speaker`, `stove`, `table`, `telephone`, `tin_can`, `tower`, `train`, `vessel`, `washer`.
Some visualized point clouds in our ShapeNetCore.v2 dataset:
<p float="left">
<img src="image/shapenetcorev2_test37_earphone.png" height="170"/>
<img src="image/shapenetcorev2_test59_lamp.png" height="170"/>
<img src="image/shapenetcorev2_train4_tower.png" height="170"/>
</p>
     earphone           lamp            tower
## ShapeNetPart
ShapeNetPart dataset contains 16,881 pre-aligned shapes from 16 categories, annotated with 50 segmentation parts in total. Most object categories are labeled with two to five segmentation parts. There are 12,137 (70%) shapes for training, 1,870 (10%) shapes for validation, and 2,874 (20%) shapes for testing. We also pack the segementation label in our dataset. The link for official dataset is [here](https://shapenet.cs.stanford.edu/media/shapenet_part_seg_hdf5_data.zip).
The 16 categories include: `airplane`, `bag`, `cap`, `car`, `chair`, `earphone`, `guitar`, `knife`, `lamp`, `laptop`, `motorbike`, `mug`, `pistol`, `rocket`, `skateboard`, `table`.
Although ShapeNetPart is made from ShapeNetCore, the number of points per shape in official ShapeNetPart dataset is not very large and sometimes less than 2,048. Thus the uniform sampling quality of our ShapeNetPart dataset is lower than our ShapeNetCore.v2 dataset.
In this dataset, we change segmentation label for each point into range 0~49 according to its category. You can find a index mapping list in `dataset.py`.
Some visualized point clouds in our ShapeNetPart dataset:
<p float="left">
<img src="image/shapenetpart_train4_airplane.png" height="170"/>
<img src="image/shapenetpart_train2_table.png" height="170"/>
<img src="image/shapenetpart_train13_chair.png" height="170"/>
</p>
     airplane            table             chair
## ModelNet40
ModelNet40 dataset contains 12,311 pre-aligned shapes from 40 categories, which are split into 9,843 (80%) for training and 2,468 (20%) for testing. The link for official dataset is [here](http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip).
The 40 categories include: `airplane`, `bathtub`, `bed`, `bench`, `bookshelf`, `bottle`, `bowl`, `car`, `chair`, `cone`, `cup`, `curtain`, `desk`, `door`, `dresser`, `flower_pot`, `glass_box`, `guitar`, `keyboard`, `lamp`, `laptop`, `mantel`, `monitor`, `night_stand`, `person`, `piano`, `plant`, `radio`, `range_hood`, `sink`, `sofa`, `stairs`, `stool`, `table`, `tent`, `toilet`, `tv_stand`, `vase`, `wardrobe`, `xbox`.
**Note**: The widely used 2,048 points sampled ModelNet40 dataset ([link](https://shapenet.cs.stanford.edu/media/modelnet40_ply_hdf5_2048.zip)) only contains 9,840 shapes for training, not 9,843 in official. Our ModelNet40 dataset fixs this problem and can substitute the above mentioned dataset perfectly.
Some visualized point clouds in our ModelNet40 dataset:
<p float="left">
<img src="image/modelnet40_train7_vase.png" height="170"/>
<img src="image/modelnet40_train10_bookshelf.png" height="170"/>
<img src="image/modelnet40_train14_plant.png" height="170" hspace="10"/>
</p>
      vase           bookshelf            plant
## ModelNet10
ModelNet10 dataset is a part of ModelNet40 dataset, containing 4,899 pre-aligned shapes from 10 categories. There are 3,991 (80%) shapes for training and 908 (20%) shapes for testing. The link for official dataset is [here](http://modelnet.cs.princeton.edu/ModelNet40.zip).
The 10 categories include: `bathtub`, `bed`, `chair`, `desk`, `dresser`, `monitor`, `night_stand`, `sofa`, `table`, `toilet`.
## Dataset performance
Repos below use our datasets:
- [antao97/UnsupervisedPointCloudReconstruction](https://github.com/antao97/UnsupervisedPointCloudReconstruction)
- coming soon ...
#### Reference repos:
- [charlesq34/pointnet](https://github.com/charlesq34/pointnet)
- [charlesq34/pointnet2](https://github.com/charlesq34/pointnet2)
- [stevenygd/PointFlow](https://github.com/stevenygd/PointFlow)
- [zekunhao1995/PointFlowRenderer](https://github.com/zekunhao1995/PointFlowRenderer)
- [WangYueFt/dgcnn](https://github.com/WangYueFt/dgcnn)
================================================
FILE: dataset.py
================================================
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
@Author: An Tao
@Contact: ta19@mails.tsinghua.edu.cn
@File: dataset.py
@Time: 2020/1/2 10:26 AM
"""
import os
import torch
import json
import h5py
from glob import glob
import numpy as np
import torch.utils.data as data
shapenetpart_cat2id = {'airplane': 0, 'bag': 1, 'cap': 2, 'car': 3, 'chair': 4,
'earphone': 5, 'guitar': 6, 'knife': 7, 'lamp': 8, 'laptop': 9,
'motor': 10, 'mug': 11, 'pistol': 12, 'rocket': 13, 'skateboard': 14, 'table': 15}
shapenetpart_seg_num = [4, 2, 2, 4, 4, 3, 3, 2, 4, 2, 6, 2, 3, 3, 3, 3]
shapenetpart_seg_start_index = [0, 4, 6, 8, 12, 16, 19, 22, 24, 28, 30, 36, 38, 41, 44, 47]
def translate_pointcloud(pointcloud):
xyz1 = np.random.uniform(low=2./3., high=3./2., size=[3])
xyz2 = np.random.uniform(low=-0.2, high=0.2, size=[3])
translated_pointcloud = np.add(np.multiply(pointcloud, xyz1), xyz2).astype('float32')
return translated_pointcloud
def jitter_pointcloud(pointcloud, sigma=0.01, clip=0.02):
N, C = pointcloud.shape
pointcloud += np.clip(sigma * np.random.randn(N, C), -1*clip, clip)
return pointcloud
def rotate_pointcloud(pointcloud):
theta = np.pi*2 * np.random.rand()
rotation_matrix = np.array([[np.cos(theta), -np.sin(theta)],[np.sin(theta), np.cos(theta)]])
pointcloud[:,[0,2]] = pointcloud[:,[0,2]].dot(rotation_matrix) # random rotation (x,z)
return pointcloud
class Dataset(data.Dataset):
def __init__(self, root, dataset_name='modelnet40', class_choice=None,
num_points=2048, split='train', load_name=True, load_file=True,
segmentation=False, random_rotate=False, random_jitter=False,
random_translate=False):
assert dataset_name.lower() in ['shapenetcorev2', 'shapenetpart',
'modelnet10', 'modelnet40', 'shapenetpartpart']
assert num_points <= 2048
if dataset_name in ['shapenetcorev2', 'shapenetpart', 'shapenetpartpart']:
assert split.lower() in ['train', 'test', 'val', 'trainval', 'all']
else:
assert split.lower() in ['train', 'test', 'all']
if dataset_name not in ['shapenetpart'] and segmentation == True:
raise AssertionError
self.root = os.path.join(root, dataset_name + '_hdf5_2048')
self.dataset_name = dataset_name
self.class_choice = class_choice
self.num_points = num_points
self.split = split
self.load_name = load_name
self.load_file = load_file
self.segmentation = segmentation
self.random_rotate = random_rotate
self.random_jitter = random_jitter
self.random_translate = random_translate
self.path_h5py_all = []
self.path_name_all = []
self.path_file_all = []
if self.split in ['train', 'trainval', 'all']:
self.get_path('train')
if self.dataset_name in ['shapenetcorev2', 'shapenetpart', 'shapenetpartpart']:
if self.split in ['val', 'trainval', 'all']:
self.get_path('val')
if self.split in ['test', 'all']:
self.get_path('test')
data, label, seg = self.load_h5py(self.path_h5py_all)
if self.load_name or self.class_choice != None:
self.name = np.array(self.load_json(self.path_name_all)) # load label name
if self.load_file:
self.file = np.array(self.load_json(self.path_file_all)) # load file name
self.data = np.concatenate(data, axis=0)
self.label = np.concatenate(label, axis=0)
if self.segmentation:
self.seg = np.concatenate(seg, axis=0)
if self.class_choice != None:
indices = (self.name == class_choice)
self.data = self.data[indices]
self.label = self.label[indices]
self.name = self.name[indices]
if self.segmentation:
self.seg = self.seg[indices]
id_choice = shapenetpart_cat2id[class_choice]
self.seg_num_all = shapenetpart_seg_num[id_choice]
self.seg_start_index = shapenetpart_seg_start_index[id_choice]
if self.load_file:
self.file = self.file[indices]
elif self.segmentation:
self.seg_num_all = 50
self.seg_start_index = 0
def get_path(self, type):
path_h5py = os.path.join(self.root, '%s*.h5'%type)
paths = glob(path_h5py)
paths_sort = [os.path.join(self.root, type + str(i) + '.h5') for i in range(len(paths))]
self.path_h5py_all += paths_sort
if self.load_name:
paths_json = [os.path.join(self.root, type + str(i) + '_id2name.json') for i in range(len(paths))]
self.path_name_all += paths_json
if self.load_file:
paths_json = [os.path.join(self.root, type + str(i) + '_id2file.json') for i in range(len(paths))]
self.path_file_all += paths_json
return
def load_h5py(self, path):
all_data = []
all_label = []
all_seg = []
for h5_name in path:
f = h5py.File(h5_name, 'r+')
data = f['data'][:].astype('float32')
label = f['label'][:].astype('int64')
if self.segmentation:
seg = f['seg'][:].astype('int64')
f.close()
all_data.append(data)
all_label.append(label)
if self.segmentation:
all_seg.append(seg)
return all_data, all_label, all_seg
def load_json(self, path):
all_data = []
for json_name in path:
j = open(json_name, 'r+')
data = json.load(j)
all_data += data
return all_data
def __getitem__(self, item):
point_set = self.data[item][:self.num_points]
label = self.label[item]
if self.load_name:
name = self.name[item] # get label name
if self.load_file:
file = self.file[item] # get file name
if self.random_rotate:
point_set = rotate_pointcloud(point_set)
if self.random_jitter:
point_set = jitter_pointcloud(point_set)
if self.random_translate:
point_set = translate_pointcloud(point_set)
# convert numpy array to pytorch Tensor
point_set = torch.from_numpy(point_set)
label = torch.from_numpy(np.array([label]).astype(np.int64))
label = label.squeeze(0)
if self.segmentation:
seg = self.seg[item]
seg = torch.from_numpy(seg)
return point_set, label, seg, name, file
else:
return point_set, label, name, file
def __len__(self):
return self.data.shape[0]
if __name__ == '__main__':
root = os.getcwd()
# choose dataset name from 'shapenetcorev2', 'shapenetpart', 'modelnet40' and 'modelnet10'
dataset_name = 'shapenetcorev2'
# choose split type from 'train', 'test', 'all', 'trainval' and 'val'
# only shapenetcorev2 and shapenetpart dataset support 'trainval' and 'val'
split = 'train'
d = Dataset(root=root, dataset_name=dataset_name, num_points=2048, split=split)
print("datasize:", d.__len__())
item = 0
ps, lb, n, f = d[item]
print(ps.size(), ps.type(), lb.size(), lb.type(), n, f)
================================================
FILE: visualize.py
================================================
import os
import numpy as np
def standardize_bbox(pcl, points_per_object):
pt_indices = np.random.choice(pcl.shape[0], points_per_object, replace=False)
np.random.shuffle(pt_indices)
pcl = pcl[pt_indices] # n by 3
mins = np.amin(pcl, axis=0)
maxs = np.amax(pcl, axis=0)
center = ( mins + maxs ) / 2.
scale = np.amax(maxs-mins)
print("Center: {}, Scale: {}".format(center, scale))
result = ((pcl - center)/scale).astype(np.float32) # [-0.5, 0.5]
return result
xml_head = \
"""
<scene version="0.5.0">
<integrator type="path">
<integer name="maxDepth" value="-1"/>
</integrator>
<sensor type="perspective">
<float name="farClip" value="100"/>
<float name="nearClip" value="0.1"/>
<transform name="toWorld">
<lookat origin="3,3,3" target="0,0,0" up="0,0,1"/>
</transform>
<float name="fov" value="25"/>
<sampler type="ldsampler">
<integer name="sampleCount" value="256"/>
</sampler>
<film type="ldrfilm">
<integer name="width" value="1600"/>
<integer name="height" value="1200"/>
<rfilter type="gaussian"/>
<boolean name="banner" value="false"/>
</film>
</sensor>
<bsdf type="roughplastic" id="surfaceMaterial">
<string name="distribution" value="ggx"/>
<float name="alpha" value="0.05"/>
<float name="intIOR" value="1.46"/>
<rgb name="diffuseReflectance" value="1,1,1"/> <!-- default 0.5 -->
</bsdf>
"""
xml_ball_segment = \
"""
<shape type="sphere">
<float name="radius" value="0.02"/>
<transform name="toWorld">
<translate x="{}" y="{}" z="{}"/>
<scale value="0.7"/>
</transform>
<bsdf type="diffuse">
<rgb name="reflectance" value="{},{},{}"/>
</bsdf>
</shape>
"""
xml_tail = \
"""
<shape type="rectangle">
<ref name="bsdf" id="surfaceMaterial"/>
<transform name="toWorld">
<scale x="10" y="10" z="10"/>
<translate x="0" y="0" z="-0.5"/>
</transform>
</shape>
<shape type="rectangle">
<transform name="toWorld">
<scale x="10" y="10" z="1"/>
<lookat origin="-4,4,20" target="0,0,0" up="0,0,1"/>
</transform>
<emitter type="area">
<rgb name="radiance" value="6,6,6"/>
</emitter>
</shape>
</scene>
"""
def colormap(x,y,z):
vec = np.array([x,y,z])
vec = np.clip(vec, 0.001,1.0)
norm = np.sqrt(np.sum(vec**2))
vec /= norm
return [vec[0], vec[1], vec[2]]
def mitsuba(pcl, path, clr=None):
xml_segments = [xml_head]
# pcl = standardize_bbox(pcl, 2048)
pcl = pcl[:,[2,0,1]]
pcl[:,0] *= -1
h = np.min(pcl[:,2])
for i in range(pcl.shape[0]):
if clr == None:
color = colormap(pcl[i,0]+0.5,pcl[i,1]+0.5,pcl[i,2]+0.5)
else:
color = clr
if h < -0.25:
xml_segments.append(xml_ball_segment.format(pcl[i,0],pcl[i,1],pcl[i,2]-h-0.6875, *color))
else:
xml_segments.append(xml_ball_segment.format(pcl[i,0],pcl[i,1],pcl[i,2], *color))
xml_segments.append(xml_tail)
xml_content = str.join('', xml_segments)
with open(path, 'w') as f:
f.write(xml_content)
if __name__ == '__main__':
item = 0
split = 'train'
dataset_name = 'shapenetcorev2'
root = os.getcwd()
save_root = os.path.join("image", dataset_name)
if not os.path.exists(save_root):
os.makedirs(save_root)
from dataset import Dataset
d = Dataset(root=root, dataset_name=dataset_name,
num_points=2048, split=split, random_rotation=False, load_name=True)
print("datasize:", d.__len__())
pts, lb, n = d[item]
print(pts.size(), pts.type(), lb.size(), lb.type(), n)
path = os.path.join(save_root, dataset_name + '_' + split + str(item) + '_' + str(n) + '.xml')
mitsuba(pts.numpy(), path)
gitextract_c1bfgm6a/ ├── .gitignore ├── LICENSE ├── README.md ├── dataset.py └── visualize.py
SYMBOL INDEX (13 symbols across 2 files)
FILE: dataset.py
function translate_pointcloud (line 26) | def translate_pointcloud(pointcloud):
function jitter_pointcloud (line 34) | def jitter_pointcloud(pointcloud, sigma=0.01, clip=0.02):
function rotate_pointcloud (line 40) | def rotate_pointcloud(pointcloud):
class Dataset (line 47) | class Dataset(data.Dataset):
method __init__ (line 48) | def __init__(self, root, dataset_name='modelnet40', class_choice=None,
method get_path (line 118) | def get_path(self, type):
method load_h5py (line 131) | def load_h5py(self, path):
method load_json (line 148) | def load_json(self, path):
method __getitem__ (line 156) | def __getitem__(self, item):
method __len__ (line 183) | def __len__(self):
FILE: visualize.py
function standardize_bbox (line 4) | def standardize_bbox(pcl, points_per_object):
function colormap (line 86) | def colormap(x,y,z):
function mitsuba (line 93) | def mitsuba(pcl, path, clr=None):
Condensed preview — 5 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (20K chars).
[
{
"path": ".gitignore",
"chars": 22,
"preview": ".DS_Store\n*/.DS_Store\n"
},
{
"path": "LICENSE",
"chars": 1063,
"preview": "MIT License\n\nCopyright (c) 2019 An Tao\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof "
},
{
"path": "README.md",
"chars": 6965,
"preview": "# Point Cloud Datasets\n\nThis repository provides ShapeNetCore.v2, ShapeNetPart, ModelNet40 and ModelNet10 datasets in HD"
},
{
"path": "dataset.py",
"chars": 7433,
"preview": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\"\"\"\n@Author: An Tao\n@Contact: ta19@mails.tsinghua.edu.cn\n@File: dataset.py"
},
{
"path": "visualize.py",
"chars": 4053,
"preview": "import os\nimport numpy as np\n\ndef standardize_bbox(pcl, points_per_object):\n pt_indices = np.random.choice(pcl.shape["
}
]
About this extraction
This page contains the full source code of the AnTao97/PointCloudDatasets GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 5 files (19.1 KB), approximately 5.8k tokens, and a symbol index with 13 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.