Classifying 3-D Models of Coral Reefs using Structure-from-Motion Photogrammetry and Multi-view Semantic Segmentation
Authors: Jordan Pierce, Mark Butler, Yuri Rzhanov, Kim Lowell, Jennifer Dijkstra
My second publication came from the third chapter of my Master's thesis, which built off of the previous two chapters. Structure-from-Motion (SfM) photogrammetry is a common tool used by coral reef ecologists to create a digital 3-D model of a habitat. Previously, researchers were required to physically measure structural complexity metrics while in-situ, though these often times difficult to quantify while underwater and can be highly invasive. Although it is now relatively easy to create a precise 3-D model of a coral reef habitat using SfM, there is no inherent mechanism to classify parts of the 3-D model to the species or functional group that they belong to; the implication here is that structural complexity metrics can only be quantified at the model-level.
This publication looked into how dense labels created for each image used in the 3-D model reconstruction could be projected onto the model itself (i.e., dense point cloud, shaded and texture mesh) to create classified versions. The study outlines a procedure from start to finish on how sparse labels can be created, converted to dense labels, and then used with Agisoft Metashape to create classified 3-D models. We also created a Github repository that contains Python Notebooks to assist in the first part of the workflow; if you have any questions or comments, please feel free to create an issue!
Images used in the reconstruction of a 3-D photogrammetric model can be provided with dense labels, and then projected from image-space to model-space to create a classified 3-D model;
Classification accuracy of classified 3-D models can be quantified by comparing the classified 2-D texture atlas with one that was manually annotated;
Accuracy is dependent on the quality of the reconstructed model, and the accuracy of the dense labels;
Dense labels can be produced using a workflow that starts with sparse labels (i.e., CPCe annotations), and then made into dense labels using Fast-MSS. Alternatively, dense labels can be made by hand using image labeling software.
Below you can find the accepted version of our paper published in Frontiers of Marine Science under the Coral Reef Research section.