Classify Pixels Using Deep Learning - how to run efficiently over many images?

[KeithDesertspring]

I'm attempting to analyze millions of JPG images stored in a filesystem using arcpy.ia.ClassifyPixelsUsingDeepLearning and am stuck on how to do this efficiently. 

My ideal script would instantiate the model into GPU memory and then feed it batches of images within a loop. Ideally I'd load these batches using my own parallel processing code, and the images would be provided to the model as numpy arrays or torch tensors, with the output being received in a similar object format. This is how you do it outside of the ESRI ecosystem, and it is very performant and flexible and easy to reason about. 

Is anything like this possible using Python + Image Analyst + DPLK models? 

I'm using ArcGIS Pro 3.3 on Windows 11

[David_McRitchie]

Hey Keith,

This is quite a specialised question and unfortunately, not one I think we will get an exact answer too, particularly if we are looking at automating this. You might have better luck posting this on the ArcGIS API for Python board.

I would recommend checking the Notebook samples site as well, as you might be able to reuse some code snippets that can assist.

The following documentation may also offer some assistance if you are looking to utilise this on numerous GPUs.

Hopefully some of that helps,

David

[PriyankaTuteja]

Hi @KeithDesertspring ,

Great question! While the workflow you're describing—leveraging GPU memory and processing batches of images efficiently—aligns with frameworks like PyTorch or TensorFlow, you can still achieve a similar result using ArcGIS Pro’s deep learning capabilities with Python and the Image Analyst extension.

A good starting point would be this sample notebook:
Count Cars in Aerial Imagery Using Deep Learning

This example demonstrates how to use the detect_objects API to process images in a loop with a DLPK model. Although it’s designed for object detection, the concepts can be adapted for pixel classification tasks.

Let me know if you’d like help tailoring this workflow further!