AI-Powered Image and Text Analysis in ArcGIS Pro
Artificial intelligence is quickly becoming part of our everyday GIS workflows. It allows us to work faster, uncover insights more easily, and automate tasks that used to require hours of manual effort.
One of the newest additions to the ArcGIS Python API makes this easier than ever. The new arcgis.ai module provides simple, built-in methods for bringing AI directly into your GIS workflows — giving you what you need to quickly build AI capabilities into your own ArcGIS tools and models using Python. Please note that while this capability is in beta, ArcGIS Online credits are not charged. After general release, running this tool will consume credits.
The module focuses on 3 functions:
- analyze_text() for analyzing or transforming text with natural language prompts.
- analyze_image() for understanding and extracting information from images.
- translate() for translating text into one or more languages.
To explore what's possible, I built a small ArcGIS Pro toolbox that demonstrates how flexible these new capabilities can be. The toolbox includes two generic tools that can be adapted to almost any industry or dataset.
My goal wasn't to create a finished product for a single use case—it was to show how AI can become another analysis tool that's available wherever your data lives. Download the toolbox and start using the tools right away, or start hacking at the code and customize it for your problem.
AI-Powered Image Analysis
The toolbox includes two AI-powered image tools that help you extract information from the photos in your layers. One analyzes feature attachments stored with your GIS data, while the other analyzes linked images referenced by URLs or local file paths. These tools offer a flexible and powerful way to unlock the information contained in your photos.
Your organization may already be collecting photos during inspections, field work, surveys, and asset inventories. Those photos often sit unused until someone clicks a popup because reviewing each of these images manually simply isn't practical.
This tool changes that.
Simply select the features you want to analyze, provide a prompt describing what you're interested in, and the tool automatically analyzes each image and writes the response back into a field in your dataset.
The prompt is entirely up to you.
You might ask AI to:
- Describe the photo in 1 or 2 sentences
- Identify damage or maintenance issues
- Read what is on the sign
- Describe for safety hazards
- Read asset numbers
- Describe the accessibility of this location
- Classify images into defined categories
Because the prompt is completely customizable, the same tools can support countless workflows without changing a single line of code.
Imagine inspecting utility poles, documenting storm damage, reviewing trail conditions, cataloging museum collections, or evaluating construction progress. These tools simply become another way of enriching your GIS data using the information already contained in your photos.
AI-Powered Text Analysis
The second tool focuses on text instead of images.
Many GIS datasets contain notes, comments, inspection reports, feedback, incident descriptions, maintenance logs, or survey responses. While this information is incredibly valuable, it often exists as long-form text that's difficult to analyze at scale.
This tool sends the contents of a text field to AI using your custom prompt, then stores the response back into another field.
Again, the possibilities are wide open.
You can ask AI to:
- Summarize lengthy reports
- Extract key issues
- Categorize complaints
- Determine sentiment
- Rewrite notes into a standardized format
- Identify recurring themes
- Generate concise executive summaries
Instead of manually reviewing thousands of records, analysts can quickly transform unstructured text into consistent, searchable information that's ready for mapping, filtering, and analysis.
Show me the code!
For you developers - or even those that know enough code to be dangerous: crack open the code! Just open the .pyt in your IDE of choice or even a text editor, and you'll find plain, readable Python. Roll up your sleeves and start customizing for the problems you're trying to solve!
And that's really the point of the arcgis.ai module — the actual AI call is almost anticlimactic. Here's roughly what's happening under the hood every time the image tool runs:
from arcgis.gis import GISfrom arcgis.ai import analyze_image gis = GIS("home") response = analyze_image( image="https://myorg.maps.arcgis.com/sharing/rest/content/items/<item_id>/attachments/<attachment_id>", prompt="Describe any visible damage or maintenance issues.", to_language="en") print(response.results[0].value)
And the text tool is doing essentially the same thing, just swapping in analyze_text() and a string of text instead of an image:
from arcgis.gis import GISfrom arcgis.ai import analyze_text gis = GIS("home") response = analyze_text( text="Caller reported a large pothole on Redlands Blvd near the intersection with Orange St, says it's been growing for weeks and damaged a tire.", prompt="Summarize this in one sentence and identify the issue type.", to_language="en") print(response.results[0].value)
That's it — that's the whole call, in both cases. Point it at an image or hand it some text, give it a prompt, get back a result. Everything else in the toolbox — the batching, rate limit and error handling, etc — exists to make those two function calls safe to run against thousands of features unattended. The AI part was never the hard part.
Where This Could Go
The two tools in this toolbox are deliberately generic — the same image tool that reads utility pole inspection photos today can classify wildlife camera traps tomorrow, and the same text tool that summarizes 311 complaints for a city can just as easily standardize maintenance logs for a utility or triage incident narratives for public safety. Local government, utilities, transportation, environmental management, parks — if the data includes a photo or a paragraph of notes, there's a prompt for it. You don't need a new tool for each of those; you need a new question.
Try It Yourself
The toolbox is free to download and tinker with — pull it into ArcGIS Pro, point it at a layer, and see what a simple prompt can do with data you already have. With this new arcgis.ai module, AI stops being a separate project on the roadmap or another series of apps that you run, and starts being another tool sitting right there in your toolbox, ready whenever you need it.
Download it, try it out on your data, rebuild the tool it into something completely different — I'd love to hear what you end up doing with it!
