The AI workplace and ArcGIS Deep Learning Workflow
Authored By: Steve Snow
Esri Senior National Government Industry Strategy Specialist
Part 4 of a 7 part series exploring GIS and Artificial Intelligence
Welcome to part 4 of my AI and GeoAI Series that will cover the more technical aspects of GeoAI and ArcGIS. Previously, part 1 of this series covered the Future Impacts of AI on Mapping and Modernization which introduced the concept of GeoAI and why you should care about having an AI as a future coworker. Part 2 of the series, GIS, Artificial Intelligence, and Automation in the Workplace covered specific geospatial professions that will be drastically effected by introduction of GeoAI technology in the workplace. Part 3 addressed Teaming with the Machine - AI in the workplace the emergence of the new geospatial working relationship between information, humans, and artificial intelligence to be successful in an organizations mission.
For part 4, we will address 3 specific GeoAI areas in ArcGIS that will help you with your journey to developing your Deep Learning workflows.
- Tools to Generate Training Data
- Tools & APIs for Training Models
- Tools to perform inferencing, such as object detection, land cover classification and more...
Tools to Generate Training Data
ArcGIS has many tools that help with GeoAI workflows for data preparation and analysis, training deep learning models, inferencing, and sharing data for collaboration with web services and maps. The tools for creating AI training can be found in ArcGIS Pro Image Analyst and ArcGIS Image Server. ArcGIS Image Analyst has tools for labeling objects and exporting training model for deep learning and using trained models for feature extraction or classification. ArcGIS Image Server in the ArcGIS Enterprise 10.7 and above has the same capabilities and allows scaling through distributed computing using clouds on premise, off premise, or hybrid cloud approaches.
Besides using prebuilt tools within the ArcGIS products, users can also work through scripting and notebooks.
Tools & API’s for Training Models
Training models can be done either through ArcGIS notebooks, ArcGIS Learn API, or in ArcGIS Pro. ArcGIS Notebooks enable a one-click access to pre-configured Jupyter Notebooks. This includes time-saving deep learning libraries using a gallery of pre-configured notebooks that contain deep learning models for easy training and deployment.
The arcgis.learn is a module in the ArcGIS API for Python which enable organizations to easily adopt and apply deep learning in their workflows. arcgis.learn enables simple and intuitive training of state-of-the-art deep learning models. arcgis.learn allows for much faster training and removes the guesswork in the training process. It integrates seamlessly with the ArcGIS platform by consuming the exported training samples directly, and the models that it creates can be used directly for inferencing in ArcGIS Pro and ArcGIS Image Server.
ArcGIS Pro users can utilize Deep Learning training methods for classification of remotely sensed data. Pixel classification, image classification, object classification, and detection can all be done with ArcGIS pro. More detailed information can be found at deep learning in ArcGIS Pro help pages. This means with notebooks being available in ArcGIS Pro and ArcGIS Online people can start experimenting with Deep Learning in the Desktop.
One exciting area using ArcGIS for training is taking advantage of pretrained models. If you’re interested in these capabilities, please check out the following document for trained deep learning models and their backbones in the model_type and backbone_model parameters: Train Deep Learning Model.
Inferencing – should my organization use ArcGIS Image Analyst or ArcGIS Image Server?
Inferencing is the process of applying the trained model, developed from the training data, and applying it against a previously unused or new data set to create a new set of annotation and labeling. ArcGIS has long been utilized for machine learning capabilities and tools. It can be used to solve a variety of geospatial problems, such as Identification of objects and image classification.
With deep learning in ArcGIS, the tools are available in both the ArcGIS Image Analyst and ArcGIS Image Server which is part of ArcGIS Enterprise. You might be wondering when you should use a) ArcGIS Image Analyst vs b) ArcGIS Image Server for inferencing?
Inferencing with ArcGIS Pro Image Analyst
If you don’t have access to ArcGIS Enterprise, you can use the ArcGIS Image Analyst (desktop) to do the inferencing/processing with a graphics card meant for deep learning. ArcGIS Image Analyst works well with on average size projects, but for larger products staff have the option of scaling their workflows for larger datasets with ArcGIS Image Server.
Inferencing with ArcGIS Image Server
Automating Object detection takes time. When your project is not small and you need to scale and optimize your AI operations, then ArcGIS Image Server should be used when running large jobs. Using ArcGIS Imager Server will always be the fastest and most efficient option. One added benefit of using ArcGIS Image Server for inferencing will be by sharing inferenced outputs that are accessible and shared directly through your own spatial data infrastructure inside and outside the organization.
If you have a high end GPU and need more tips on when to use ArcGIS Pro vs ArcGIS Enterprise I recommend you read this great article Which is better for deep learning ArcGIS Pro or ArcGIS Enterprise? from my Esri Colleagues Vinay Vishwambharan and Emily Windahl.
Tools – a few popular inferencing tools
Inferencing tools
Classify Pixels Using Deep Learning - In a raster analysis deployment, this tool runs a trained deep learning model on an input image to produce a classified raster published as a hosted imagery layer in your portal.
- Object Detection Using Deep Learning - In a raster analysis deployment, this tool runs a trained deep learning model on an input raster to produce a feature class containing the objects it identifies. The feature class can be shared as a hosted feature layer in your portal. The features can be bounding boxes or polygons around the objects found, or points at the centers of the objects.
- Non-Maximum Suppression - Identifies duplicate features from the output of the Detect Objects Using Deep Learning tool as a postprocessing step and creates a new output with no duplicate features. The Detect Objects Using Deep Learning tool can return more than one bounding box or polygon for the same object, especially as a tiling side effect. If two features overlap more than a given maximum ratio, the feature with the lower confidence value will be removed.
I would like to thank everyone for following this series. For part 5 of the next installment I will be focusing on Providing National Scalability – Modernizing National Mapping Capabilities with AI and National 3D Basemaps.
If you missed the previous 3 sections of this 7-part series feel free to visit the previous articles.
Part 1: Future Impacts by AI on Mapping and Modernization
Part 2: GIS, Artificial Intelligence, and Automation in the Workplace
Part 3: Teaming with the Machine: AI in the Workplace