Compute Confusion Matrix or Accuracy Assessment for Point Cloud Datasets

mattkramer — Fri, 22 Dec 2023 22:03:16 GMT

Accuracy Assessment and Confusion Matrix workflows exist for comparing a classified raster to a reference raster, but the same process does not exist for point cloud files (at least I can't find one). Being able to compare two point cloud datasets to each other would allow us to check on a model's performance after it has been trained, which would allow for us to ensure the accuracy assessment that gets generated during training is still true. This would also allow us to gauge the accuracy of out of the box auto classification methods, such as classify ground. Ensuring these out of the box classification methods are producing results within acceptable tolerances to a by hand approach is a necessary step before we can confidently switch to these processes in our workflows.

Relevant links:

Assess point cloud training results—ArcGIS Pro | Documentation -- Accuracy results reported during the training of a model, this same accuracy result would be the desired output of a tool, that does not require training the mode.

Accuracy Assessment—ArcGIS Pro | Documentation

Create Accuracy Assessment Points (Spatial Analyst)—ArcGIS Pro | Documentation

Compute Confusion Matrix (Spatial Analyst)—ArcGIS Pro | Documentation - These 3 reference tools that work for raster datasets, but do not work for point cloud datasets

Re: Compute Confusion Matrix or Accuracy Assessment for Point Cloud Datasets - Status changed to: Under Consideration

KhalidDuri1 — Thu, 08 Feb 2024 18:26:05 GMT

Hello and thank you for sharing this suggestion. The Evaluate Point Cloud Classification Model tool provides a way to compare multiple classification models at once and it provides a confusion matrix for each model. However, this tool requires deep learning models as input and does not allow for comparison between two or more stand-alone point cloud datasets. We intend to introduce a tool that does so, but until then, I hope the Evaluate tool is useful for your needs.

Regards,

Khalid

idea Compute Confusion Matrix or Accuracy Assessment for Point Cloud Datasets in ArcGIS Pro Ideas

Compute Confusion Matrix or Accuracy Assessment for Point Cloud Datasets

Re: Compute Confusion Matrix or Accuracy Assessment for Point Cloud Datasets - Status changed to: Under Consideration