Help Needed with YOLO v8 Model Integration in ArcGIS Pro

[MuhammadElbagoury]

Hello Everyone,

I have trained a YOLO v8 model and saved the weights in .pt format for detecting objects in 360-degree images. I also converted the model to .onnx format. Here are my specific needs:

1. Integration with ArcGIS Pro: Can I use this model directly with ArcGIS Pro tools or through the ArcGIS API for Python to extract detected objects along with their GPS coordinates for mapping?

2. Conversion to .dlpk: Is there a way to convert the .onnx model to a .dlpk file for use in ArcGIS Pro?

3. Coordinate Conversion: How can I convert the pixel coordinates of the detected objects to GPS coordinates and export this data to a shapefile for import into ArcGIS Pro?

Any guidance or resources on these topics would be greatly appreciated!

Thank you!

[PriyankaTuteja]

Hello,
Integrating an external model into ArcGIS Pro involves multiple steps, including model integration, conversion to a compatible format. Below is a structured approach to help you achieve this:

1. Integrating with ArcGIS Pro

ArcGIS Pro supports deep learning workflows through its Image Analyst extension. To use your external model within ArcGIS Pro, follow these steps:

• Prepare Training Data: Use the Export Training Data for Deep Learning tool in ArcGIS Pro to generate the required training datasets.
• Train or Import the Model: If your model is already trained, ensure it is in a format compatible with ArcGIS Pro. You can also train an external model using the ModelExtension API. Refer here for more details.
• Detect Objects: Apply your trained model to imagery in ArcGIS Pro using the Detect Objects Using Deep Learning tool. This tool requires a model definition file (.emd) that contains information about the trained model.

2. Converting to .dlpk

A Deep Learning Model Package (.dlpk) is a compressed file that includes the model definition and necessary files for deployment in ArcGIS. You can create a .dlpk file using one of the following methods:

• Train Deep Learning Model Tool: The Train Deep Learning Model tool in ArcGIS Pro generates a .dlpk file after successfully training a model. This tool works with datasets prepared using the Export Training Data for Deep Learning tool.
• Manual Packaging: If your model is trained externally, you can manually package it into a .dlpk file. This requires organizing model files and metadata into a structured directory and compressing them. Ensure that the model definition file (.emd) is correctly configured. Refer to the Custom Models repo for detailed guidance.

3. Converting Pixel Coordinates to GPS Coordinates

To convert detected object locations from pixel coordinates to GPS coordinates and export the data as a shapefile:

• Georeferencing: Ensure that your input imagery is georeferenced, meaning each pixel has a corresponding real-world coordinate.
• Detection Output: The Detect Objects Using Deep Learning tool generates spatial features (e.g., bounding boxes) with coordinate references that match the input imagery.
• Export to Shapefile: Once object detection is complete, you can directly export the results to a shapefile, which will retain the GPS coordinates of the detected objects.

Let me know if you need further clarification!