Plant 3D (P3D): A plant phenotyping toolkit for 3D point clouds
Developing methods to efficiently analyze 3D point cloud data of plant architectures remains challenging for many phenotyping applications. Here, we describe a tool that tackles four core phenotyping tasks: classification of cloud points into stem and lamina points; graph skeletonization of the stem points; segmentation of individual lamina; and whole leaf labeling. These four tasks are critical for numerous downstream phenotyping goals, such as quantifying plant biomass, performing morphological analyses of plant shapes, and uncovering genotype to phenotype relationships. The P3D tool provides an intuitive graphical user interface, a fast 3D rendering engine for visualizing plants with millions of cloud points, and several graph-theoretic and machine learning algorithms for 3D architecture analyses. As 3D point clouds become a standard data type for digitizing plant architectures in the lab and in the field, we hope the P3D tool can help accelerate next-generation plant phenotyping.
Plant 3D (P3D) automatically extracts common phenotyping features of interest from high-resolution 3D scans of plant architectures. P3D is open-source and is bundled with a stand-alone Windows application. P3D is written in C++ using OpenGL, QT, TensorFlow, and the point cloud library (PCL). P3D can visualize and process data imported as a 3D point cloud (pcd or txt formats) or a mesh (obj format).
- Lamina vs. stem classification
- Lamina counting and segmentation
- Stem skeletonization
- Whole leaf labeling
Cloud points are classified by a trained deep learning model. The model uses binary classification to separate lamina points (e.g., cotyledons, leaves) and stem points (e.g., branches, petioles). Prior to training, a 33-dimensional feature vector was extracted for each 3D point. These features correspond to FPFH (Fast Point Feature Historgram) features, computed using the Point Cloud Library. P3D comes with a few pre-trained models. Models can be found in TF_Models folder with ".pb" tensorflow file extension. To use a model, click on the classify button on the sidebar menu; controls panel on the lower right should appear. In the controls panel, click Browse button to provide a path to ".pb" file and then click Run Classification. For details on why FPFH was selected and what parameters were used for training, please refer to our publication (link below).
Lamina points are clustered into individual lamina. Clicking on the segmentaion icon on the sidebar menu will bring lamina segmentation controls panel on the lower right. The method is an enhanced version of conditional region growing. To run lamina segmentaion on a user's file that has already been classified elsewhere and only contains lamina points (no stem points), a particular naming convetion must be followed (described below).
Creates a graph-theoretic tree skeleton of the stem points. Our method is an improved version of the stem skeletanization of the PypeTree framework. Stem file naming convetion needs to be followed to run this methods as well (described below).
Leaf labeling task consolidates the information gathered from lamina segmentation and skeletanization to produce a point cloud where each point is labeled as one of the following: main stem, leaf in the biological sense, or cotyledon. Currenlty this task in only available for tomato plants. To run this method, both the stem and lamina files need to named according to the naming convention (below) when imported to P3D.
For the tasks to function properly a particular naming conventions need to be followed. All tasks except for classification expect this format.plant_name_l.pcd or plant_name_l.txt // "_l" stands for lamina points
plant_name_s.pcd or plant_name_s.txt // "_s" stands for stem points
Running classification on any file will produce two sets of points; the names of these two files will automatically follow the expected convention.
The folder "Plant_Point_Clouds" contains some raw (unclassified) scanned point clouds from our dataset in PCL's ".pcd" format. These can be used to test P3D.
Windows installer folder contains a msi installer file. Please donwload the file and run the executable wizard.
After the wizard is done, there should be two folders P3D and Assets. Open P3D folder and run the executable file located there. If you don't have access you a Windows computer, consider using a virual machine with a tool such as virtual box.
Because plants show remarkable plasticity in their form, P3D provides a comprehensive classification model pre-trained on diverse architectures grown in different environments (ambient, high-light, high-heat, drought, shade) and at different developmental time-points (from day 5 to day 20). In addition, P3D allows users to provide a path to their own pre-trained network.
The TF_Models folder contains deep learning models trained on our dataset using FPFH features. To use a model, supply a path to it during classification. For inference, P3D will compute FPHF features for every point using the same FPFH parameters used during training.
For aditional technical details, please refer to our publication below:
I. Ziamtsov and S. Navlakha. Machine learning approaches to improve three basic plant phenotyping tasks using three-dimensional point clouds. Plant Phys., 2019.