topic Re: How to train and use custom deep learning model in 3D Exploratory Analysis toolbar in Pro? in ArcGIS Pro Questions

How to train and use custom deep learning model in 3D Exploratory Analysis toolbar in Pro?

kntr — Wed, 19 Feb 2025 06:40:30 GMT

The 3D interactive object detection tool in the Exploratory Analysis toolbar in Arcgis Pro has pre-trained models for windows and doors detection from mesh.

Ref: https://pro.arcgis.com/en/pro-app/latest/help/mapping/exploratory-analysis/interactive-object-detection-basics.htm

Are there ways to train custom deep learning models to use with this tool? For example, road crack detection.

Or are there plans for Esri to add more pre-trained models into this tool?

Cheers!

Re: How to train and use custom deep learning model in 3D Exploratory Analysis toolbar in Pro?

BobBooth1 — Wed, 19 Feb 2025 14:02:55 GMT

The Windows and Doors Extraction model item page (https://esriinc.maps.arcgis.com/home/item.html?id=8c0078cc7e314e31b20001d94daace5e) has a link to a developer documentation page on retraining the model:

https://developers.arcgis.com/python/latest/guide/retraining-windows-doors-extraction-model/

There is a crack detection model (for imagery) here:

https://esriinc.maps.arcgis.com/home/item.html?id=a9c3134e361e49a191efda169f5a337d

Re: How to train and use custom deep learning model in 3D Exploratory Analysis toolbar in Pro?

dkutkevicius_hb — Thu, 20 Mar 2025 07:45:52 GMT

Hello @BobBooth1 ,

This is very interesting. So if I understand your answer correctly, we can retrain Esri Windows and Doors Extraction model to detect other objects (like road cracks mentioned in original question)?

Also, are there any other ready to use .dlpk models for interactive detection tool in 3D scene?

Re: How to train and use custom deep learning model in 3D Exploratory Analysis toolbar in Pro?

BobBooth1 — Tue, 21 Oct 2025 14:40:57 GMT

The Windows and Doors model is more suitable for detecting windows and doors, but the important point is that you can train, or in some cases, retrain, existing models. That sample code is about training the model to better detect specific types of windows and doors that are most common in your area, but a similar approach would apply for retraining a model to detect cracks.

The generic object detection model mentioned in this help topic may be retrainable to detect cracks:

https://pro.arcgis.com/en/pro-app/latest/help/mapping/exploratory-analysis/interactive-object-detection-basics.htm

The tool accepts dlpk files where you select what model to use, so you should be able to train a model for your specific needs (crack detection in this case) and package it as a dlpk to use in the tool.

Check out the crack detection model here:

https://www.arcgis.com/home/item.html?id=a9c3134e361e49a191efda169f5a337d

[URL edited]

I have not tested it with the 3D interactive object detection tool in the Exploratory Analysis toolbar, but it is probably worth experimenting with, to determine if it suits your needs.

Other pretrained object detection models are available:

https://livingatlas.arcgis.com/en/browse/?q=dlpk%20detection#d=2&q=dlpk+detection

Re: How to train and use custom deep learning model in 3D Exploratory Analysis toolbar in Pro?

dkutkevicius_hb — Thu, 20 Mar 2025 12:20:34 GMT

Thank you for detailed explanation. I will look into the possibility of retraining the model to suit our needs. I have tried to add other pretrained object detection models from living atlas but it seems that none of them can be added.

Re: How to train and use custom deep learning model in 3D Exploratory Analysis toolbar in Pro?

BobBooth1 — Tue, 01 Apr 2025 13:12:33 GMT

I got some more information on this tool.

You cannot download other models directly from within the tool. You have to download other models to your machine and then you can browse/choose the downloaded model using the browse button for the Model input.
Only the following models are supported: FasterRCNN, YOLOv3, Single Shot Detector (SSD), and RetinaNet.

So, for a crack detection scenario, you would need to research which of those model types is likely to work well for cracks, download a pretrained model that is of one of those model types, do retraining following the general model of the developer doc, or try transfer learning, as shown in this tutorial (https://learn.arcgis.com/en/projects/improve-a-deep-learning-model-with-transfer-learning/).

Other tutorials that may be helpful in this process include:

https://learn.arcgis.com/en/projects/get-ready-for-deep-learning-in-arcgis-pro/

and

https://learn.arcgis.com/en/projects/train-a-model-using-automated-deep-learning/

Re: How to train and use custom deep learning model in 3D Exploratory Analysis toolbar in Pro?

JerradDringman1 — Tue, 21 Oct 2025 14:02:16 GMT

Your link to the crack detection model doesn't appear to be working for me. Asks for a login that won't let me login. Is there another way to view what you've linked here?

Thank you.

Re: How to train and use custom deep learning model in 3D Exploratory Analysis toolbar in Pro?

BobBooth1 — Tue, 21 Oct 2025 14:40:10 GMT

Oops, sorry. It is publicly shared here:

https://www.arcgis.com/home/item.html?id=a9c3134e361e49a191efda169f5a337d

Re: How to train and use custom deep learning model in 3D Exploratory Analysis toolbar in Pro?

BobBooth1 — Tue, 21 Oct 2025 14:41:16 GMT

I updated the URL in my post above too.

Re: How to train and use custom deep learning model in 3D Exploratory Analysis toolbar in Pro?

fmuresu — Wed, 29 Oct 2025 07:02:45 GMT

Hello everyone,

I'm jumping into this thread since it's one of the few that discusses the Object Detection tool within the Exploratory Analysis toolbar in ArcGIS.

I'm currently testing the tool in ArcGIS Pro 3.5.4 with the "Windows and Doors" model. Regardless of the model's actual performance, the resulting detected points always show an evident shift from the objects present in the image.

I managed to find another model that works with this tool, the "Pool Detection - USA" model, available at the following link:

https://ps-esriitalia.maps.arcgis.com/home/item.html?id=0e7dffe605c24bdfadf3c376bdf2d413

This model seems to be affected by the exact same problem. This leads me to believe that the issue is not with the model itself but rather with the too.

Please find two relevant screenshots attached.

Has anyone else experienced this same inconvenience?

Thanks!