Migrating Defense Mapping Production Workflows from ArcMap to ArcGIS Pro

321
0
04-29-2021 03:56 PM
ShreeRajagopalan
Esri Contributor
0 0 321

ArcGIS Defense Mapping is now fully available in ArcGIS Pro. This blog provides information about the new topographic production workflows available in ArcGIS Defense Mapping Pro and the concepts and capabilities from ArcMap-based workflows that exist in this new workflow framework.

Overview of  Topographic Production Workflows

In ArcMap, Defense Mapping included ArcGIS Workflow Manager configurations (.jxls), additional Workflow Database schema tables, custom toolboxes, and other files that could be used to deploy workflows for data and map production. In general, the data production focused workflows could utilize any data schema, while the cartographic production focused workflows were usually used to create defense-specific products supported by Defense Mapping.

In Pro, the workflow configurations for use with Workflow Manager and necessary tools and files are installed with the ArcGIS Production Mapping Product Data. Data production workflows continue to be schema-agnostic, while cartographic workflows support defense products and additionally Esri’s Civilian Topographic Map (CTM) specification for civilian customers. The Data Ingest workflows have been improved and new workflows have been added to support Data Generalization, including updating an existing database with the newly generalized data.

What’s not changed?

The most frequently used production workflow capabilities from ArcMap have been integrated into the Pro workflows starting at ArcGIS Pro 2.6. This includes vector data editing, quality control (QC), field editing, and automated cartographic product creation and cartographic QC.  All of these workflows utilize geoprocessing tools within ArcGIS Pro to automate most workflow steps, including creating and deleting versions, extracting and loading data, creating cartographic data and layouts, and synchronizing replicas and posting versions. Workflow steps also open maps and layouts already sourced to the correct data and zoomed to the Location of Interest (LOI). Jobs can be still be created using desktop tools or using an updated version of the Production Manager web application – in either case, their LOI’s will match polygon map index/grid feature class(es) that you define. As before, the workflows use a shared folder location for job artifacts (working files) and shared configuration files for the environment. ArcGIS Workflow Manager (Classic) Administrator, now renamed and available as a standalone download from MyEsri, provides the same familiar administration experience as the ArcMap-based tool.

What’s changed/new?

Significant changes have been made to both the user interface and the workflow framework in ArcGIS Pro, with the goal of providing a more integrated and streamlined experience for end users. Most notably, ArcGIS Workflow Manager is integrated directly into the ArcGIS Pro application. Users create a workflow connection once in their project, then familiar Workflow Manager views like job queries, the job list and map views, and job properties are visible as panes. Additionally, the ribbon interface includes a Workflow tab used to create and manage jobs. More details can be found here: https://pro.arcgis.com/en/pro-app/latest/help/workflow-manager/what-is-workflow-manager.htm

From an end user perspective, the new workflow framework should make for a more straightforward, streamlined process. In ArcMap, one long workflow existed for each type of production, necessitating manual handoffs between users. The new framework relies on parent and child jobs, where the parent job creates smaller child jobs, with each child job capturing a focused task and designed to be executed by one user. For example, Data Production can include four child jobs/workflows for Creating checkout replica, Editing data in Pro, QC, and Publishing updates. Each child job workflow includes only the handful of steps needed for the specific tasks. These child jobs are created automatically by a background process at the right time, so end users only need to work with the specific child job types and steps necessary for their task.

Configuring the new workflow framework

This new framework requires different extended property tables in the Workflow Database than previously used in ArcMap. There is no automated migration of the previous tables to the new format, meaning that your existing Extraction, Cartography and Field Editing Operations (for example) must be recreated in Pro. These are called Production Types in the new workflow framework, and it is a straight-forward process using a geoprocessing tool to create them and set key paths and properties. See below:

ShreeRajagopalan_0-1619736467591.png

The new workflow framework has been made more flexible with the introduction of Task Groups, which dictate, for a given production type, exactly which child jobs need to be included, in what order, their expected level of effort, and which groups or users should be assigned. Like Production Types, Task Groups can also be created and modified using a geoproccesing tool, and you will need to create them for your production environment before using the workflows. See the image below that shows the process for creating a new Task Group that uses the TDS EXTRACTION Production Type from the previous image.

ShreeRajagopalan_1-1619736609650.png

In the example, we are defining that for TDS Extraction, Tasks (child jobs) will be created in order to create the replica, maintain (edit), QC, and finally to publish the data updates. Sample task groups like this one are included in ArcGIS Production Mapping to use as a starting point. You can add, remove, reorder, and modify the tasks using the tool above as you need. For example, if a specific type of work does not have require a formal QC process, the DATA-QUALITY CONTROL task can be removed. Or, if all QC should flow through a specific group, the Default Assignment for the QC task can be set to that group . Additionally, expected levels of effort (LOE) can be defined along with standard working days and hours. This enables the workflow system to automatically generate an overall expected level of effort for each Task Group (parent job) by aggregating those of each child task, and track how delays in any child task will affect start dates of tasks downstream and ultimately the expected due date of the parent job.

To summarize, Production Types (similar to the Extraction/Cartography/Field Editing Operations in ArcMap) and Task Groups are how you configure the system to your production tasks and files. Production Types dictate the key paths and files for the workflows to use, and Task Groups dictate which child workflows to actually utilize for a given Production Type. You must define these during the initial configuration of the environment using the geoprocessing tools pictured above.

More automation

The Production Mapping Product Data installer also includes scripts and .bat files to enable the automatic execution of key workflows. Specifically, the Execute Task Group parent job is designed to always run automatically using the supplied scripts and .bat file. This can be easily configured and scheduled using Windows Task Scheduler to scan for jobs that are ready to run and execute them. This ensures that as soon as a child job (say Data Editing) is completed by one user, the next job (Quality Control) is created and assigned automatically. Many child job types can also be configured to automatically execute their entire workflow, meaning tasks like creating replicas, publishing data updates, creating a cartographic product and generalizing data can run entirely without any user interaction.

Capturing metrics

The new framework captures valuable production information during and after the work is complete. At any time, users can see a job’s initial expected level of effort and remaining level of effort, which adjusts automatically if rework is required. Once work is completed, metrics are captured at the AOI, resource, and task (job type) levels. These include totals and averages of days spent on tasks, number of times rework was required, what percentage of tasks resulted in rework, and how many rework days were required. If you define a complexity value for your AOI’s, additional metrics will be captured including average complexity by user and task. These values could be used to better understand when existing work should be completed, assess resource performance, and forecast future production velocity.

Configuration tools and documentation

Finally, the deployment of the production workflows has been made easier with the inclusion of several geoprocessing tools installed with the Production Mapping Product Data that streamline the process. The Configure Workflow Database tool will automatically create the database, import the desired workflow files (.jxls) and create the necessary additional tables, automatically accounting for any RDBMS schema differences. The Manage Production Type and Manage Task Group tools shown above can be used to interactively create your Production Types and Task Groups as well as modify or delete them as needed. Standalone documentation is installed with the Production Mapping Product Data to walk you through this process, as well as the configuration of the background process. While this new framework is very different, these tools and documents will help you get up and running quickly.

For migrating generalization processes from ArcMap to ArcGIS Pro, refer to this blog

About the Author
I manage marketing strategy, communications and product marketing for an elite group of products at Esri. I also help draft/refine messaging and artifacts, and promote awareness, demand and adoption among Esri customers.