Documentation | Blog | Demo
Learning3D is an open-source library that supports the development of deep learning algorithms that deal with 3D data. The Learning3D exposes a set of state of art deep neural networks in python. A modular code has been provided for further development. We welcome contributions from the open-source community.
Sr. No. | Tasks | Algorithms |
---|---|---|
1 | Classification | PointNet, DGCNN, PPFNet |
2 | Segmentation | PointNet, DGCNN |
3 | Reconstruction | Point Completion Network (PCN) |
4 | Registration | PointNetLK, PCRNet, DCP, PRNet, RPM-Net |
5 | Flow Estimation | FlowNet3D |
- PointNet
- PCN
- PointNetLK
- PCRNet
- DCP
- PRNet
- FlowNet3D
- RPM-Net (clean-trained.pth, noisy-trained.pth, partial-pretrained.pth)
- ModelNet40
- Classification Loss (Cross Entropy)
- Registration Losses (FrobeniusNormLoss, RMSEFeaturesLoss)
- Distance Losses (Chamfer Distance, Earth Mover's Distance)
- Ubuntu 16.04
- Ubuntu 18.04
- Linux Mint
- CUDA 10.0 or higher
- Pytorch 1.3 or higher
Important Note: Clone this repository in your project. Please don't add your codes in "learning3d" folder.
- All networks are defined in the module "models".
- All loss functions are defined in the module "losses".
- Data loaders are pre-defined in data_utils/dataloaders.py file.
- All pretrained models are provided in learning3d/pretrained folder.
B: Batch Size, N: No. of points and C: Channels.
from learning3d.models import PointNet, DGCNN, PPFNet
pn = PointNet(emb_dims=1024, input_shape='bnc', use_bn=False)
dgcnn = DGCNN(emb_dims=1024, input_shape='bnc')
ppf = PPFNet(features=['ppf', 'dxyz', 'xyz'], emb_dims=96, radius='0.3', num_neighbours=64)
Sr. No. | Variable | Data type | Shape | Choices | Use |
---|---|---|---|---|---|
1. | emb_dims | Integer | Scalar | 1024, 512 | Size of feature vector for the each point |
2. | input_shape | String | - | 'bnc', 'bcn' | Shape of input point cloud |
3. | output | tensor | BxCxN | - | High dimensional embeddings for each point |
4. | features | List of Strings | - | ['ppf', 'dxyz', 'xyz'] | Use of various features |
5. | radius | Float | Scalar | 0.3 | Radius of cluster for computing local features |
6. | num_neighbours | Integer | Scalar | 64 | Maximum number of points to consider per cluster |
from learning3d.models import Classifier, PointNet, Segmentation
classifier = Classifier(feature_model=PointNet(), num_classes=40)
seg = Segmentation(feature_model=PointNet(), num_classes=40)
Sr. No. | Variable | Data type | Shape | Choices | Use |
---|---|---|---|---|---|
1. | feature_model | Object | - | PointNet / DGCNN | Point cloud embedding network |
2. | num_classes | Integer | Scalar | 10, 40 | Number of object categories to be classified |
3. | output | tensor | Classification: Bx40, Segmentation: BxNx40 | 10, 40 | Probabilities of each category or each point |
from learning3d.models import PointNet, PointNetLK, DCP, iPCRNet, PRNet, PPFNet, RPMNet
pnlk = PointNetLK(feature_model=PointNet(), delta=1e-02, xtol=1e-07, p0_zero_mean=True, p1_zero_mean=True, pooling='max')
dcp = DCP(feature_model=PointNet(), pointer_='transformer', head='svd')
pcrnet = iPCRNet(feature_moodel=PointNet(), pooling='max')
rpmnet = RPMNet(feature_model=PPFNet())
Sr. No. | Variable | Data type | Choices | Use | Algorithm |
---|---|---|---|---|---|
1. | feature_model | Object | PointNet / DGCNN | Point cloud embedding network | PointNetLK |
2. | delta | Float | Scalar | Parameter to calculate approximate jacobian | PointNetLK |
3. | xtol | Float | Scalar | Check tolerance to stop iterations | PointNetLK |
4. | p0_zero_mean | Boolean | True/False | Subtract mean from template point cloud | PointNetLK |
5. | p1_zero_mean | Boolean | True/False | Subtract mean from source point cloud | PointNetLK |
6. | pooling | String | 'max' / 'avg' | Type of pooling used to get global feature vectror | PointNetLK |
7. | pointer_ | String | 'transformer' / 'identity' | Choice for Transformer/Attention network | DCP |
8. | head | String | 'svd' / 'mlp' | Choice of module to estimate registration params | DCP |
from learning3d.models import PCN
pcn = PCN(emb_dims=1024, input_shape='bnc', num_coarse=1024, grid_size=4, detailed_output=True)
Sr. No. | Variable | Data type | Choices | Use |
---|---|---|---|---|
1. | emb_dims | Integer | 1024, 512 | Size of feature vector for each point |
2. | input_shape | String | 'bnc' / 'bcn' | Shape of input point cloud |
3. | num_coarse | Integer | 1024 | Shape of output point cloud |
4. | grid_size | Integer | 4, 8, 16 | Size of grid used to produce detailed output |
5. | detailed_output | Boolean | True / False | Choice for additional module to create detailed output point cloud |
from learning3d.models import FlowNet3D
flownet = FlowNet3D()
from learning3d.data_utils import ModelNet40Data, ClassificationData, RegistrationData, FlowData
modelnet40 = ModelNet40Data(train=True, num_points=1024, download=True)
classification_data = ClassificationData(data_class=ModelNet40Data())
registration_data = RegistrationData(algorithm='PointNetLK', data_class=ModelNet40Data(), partial_source=False, partial_template=False, noise=False)
flow_data = FlowData()
Sr. No. | Variable | Data type | Choices | Use |
---|---|---|---|---|
1. | train | Boolean | True / False | Split data as train/test set |
2. | num_points | Integer | 1024 | Number of points in each point cloud |
3. | download | Boolean | True / False | If data not available then download it |
4. | data_class | Object | - | Specify which dataset to use |
5. | algorithm | String | 'PointNetLK', 'PCRNet', 'DCP', 'iPCRNet' | Algorithm used for registration |
6. | partial_source | Boolean | True / False | Create partial source point cloud |
7. | partial_template | Boolean | True / False | Create partial template point cloud |
8. | noise | Boolean | True / False | Add noise in source point cloud |
from learning3d.losses import RMSEFeaturesLoss, FrobeniusNormLoss, ClassificationLoss, EMDLoss, ChamferDistanceLoss
rmse = RMSEFeaturesLoss()
fn_loss = FrobeniusNormLoss()
classification_loss = ClassificationLoss()
emd = EMDLoss()
cd = ChamferDistanceLoss()
Sr. No. | Loss Type | Use |
---|---|---|
1. | RMSEFeaturesLoss | Used to find root mean square value between two global feature vectors of point clouds |
2. | FrobeniusNormLoss | Used to find frobenius norm between two transfromation matrices |
3. | ClassificationLoss | Used to calculate cross-entropy loss |
4. | EMDLoss | Earth Mover's distance between two given point clouds |
5. | ChamferDistanceLoss | Chamfer's distance between two given point clouds |
- Copy the file from "examples" folder outside of the directory "learning3d"
- Now, run the file. (ex. python test_pointnet.py)
- Your Directory/Location
- learning3d
- test_pointnet.py
- PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
- Dynamic Graph CNN for Learning on Point Clouds
- PPFNet: Global Context Aware Local Features for Robust 3D Point Matching
- PointNetLK: Robust & Efficient Point Cloud Registration using PointNet
- PCRNet: Point Cloud Registration Network using PointNet Encoding
- Deep Closest Point: Learning Representations for Point Cloud Registration
- PRNet: Self-Supervised Learning for Partial-to-Partial Registration
- FlowNet3D: Learning Scene Flow in 3D Point Clouds
- PCN: Point Completion Network
- RPM-Net: Robust Point Matching using Learned Features
- 3D ShapeNets: A Deep Representation for Volumetric Shapes