Classifying 3-D Models of Coral Reefs using Structure-from-Motion Photogrammetry and Multi-view Semantic Segmentation
Authors: Jordan Pierce, Mark Butler, Yuri Rzhanov, Kim Lowell, Jennifer Dijkstra
​
My second publication came from the third chapter of my Master's thesis, which built off of the previous two chapters. Structure-from-Motion (SfM) photogrammetry is a common tool used by coral reef ecologists to create a digital 3-D model of a habitat. Previously, researchers were required to physically measure structural complexity metrics while in-situ, though these often times difficult to quantify while underwater and can be highly invasive. Although it is now relatively easy to create a precise 3-D model of a coral reef habitat using SfM, there is no inherent mechanism to classify parts of the 3-D model to the species or functional group that they belong to; the implication here is that structural complexity metrics can only be quantified at the model-level.
​
This publication looked into how dense labels created for each image used in the 3-D model reconstruction could be projected onto the model itself (i.e., dense point cloud, shaded and texture mesh) to create classified versions. The study outlines a procedure from start to finish on how sparse labels can be created, converted to dense labels, and then used with Agisoft Metashape to create classified 3-D models. We also created a Github repository that contains Python Notebooks to assist in the first part of the workflow; if you have any questions or comments, please feel free to create an issue!
​
Main takeaways:
​
-
Images used in the reconstruction of a 3-D photogrammetric model can be provided with dense labels, and then projected from image-space to model-space to create a classified 3-D model;
-
Classification accuracy of classified 3-D models can be quantified by comparing the classified 2-D texture atlas with one that was manually annotated;
-
Accuracy is dependent on the quality of the reconstructed model, and the accuracy of the dense labels;
-
Dense labels can be produced using a workflow that starts with sparse labels (i.e., CPCe annotations), and then made into dense labels using Fast-MSS. Alternatively, dense labels can be made by hand using image labeling software.
​
Below you can find the accepted version of our paper published in Frontiers of Marine Science under the Coral Reef Research section.