Select to view content in your preferred language

Reporting data quality using Attribute Rules and Python in ArcGIS Pro 3.7

70
0
yesterday
BaileyAOhlson
Esri Contributor
0 0 70

Table of Contents

Summary

In 2021, the Data Reviewer team authored a Python Toolbox that leverages ArcPy and OpenPyXL to generate an Excel data quality report by extracting data from attribute rule errors and feature classes. This reporting tool can be leveraged to quickly assess data quality and inform key stakeholders of the quality of their organization's data in an easy-to-read table report.

In the 2024 release of ArcGIS Pro 3.3, this Python script was updated to account for file and mobile geodatabases, and was enhanced to report data quality metrics for Visual Review Rules. For the 2026 release of ArcGIS Pro 3.7, this tool has been updated again to support two new Data Reviewer checks: Valency, and Duplicate Row. 

This blog is intended to share the updated script tool for download. To learn more about how data is compiled for Attribute Rules, please visit this 2021 blog post detailing important terms and methodology. To learn more about how data is compiled for Visual Review results, please visit this 2023 blog post detailing important terms and methodology.

Updates to the Data Quality Reporting Script

Since the ArcGIS Pro 3.3 version of the Data Quality Report script was shared Data Reviewer has added two new data quality checks to our lineup: Valency and Duplicate Row.

The Data Quality Report tool has been updated to support both of these two new Attribute Rules. It is worth noting that previous versions of this script will still work in ArcGIS Pro 3.7, but if you use the Valency and/or Duplicate Row checks as Attribute Rules, those error results will not appear in the output report. No changes were made to the logic supporting Visual Review.

Script Instructions

The Data Quality Reporting scripts can be downloaded from this blog and run as a Python Toolbox in ArcGIS Pro (DataReviewerDataQuality.pyt) or in the command line interface (CLI) using the stand-alone python script (AttributeRulesReport.py). The following CLI options are supported when calling the python script: 

OptionsNameDescriptionRequired/Optional

-h or --help 

Help 

Show this help message and exit 

Optional 

-i 

INPUT 

File location of your geodatabase containing attribute rules (only FGDB and MGDB are currently supported) 

Required 

-o 

OUTPUT 

File location to save the data quality report .xlsx file 

Required 

-hide {True,False} 

HIDDEN 

Enter True/False to choose if you would like the severity rows hidden from the Visual Review Rules worksheet 

Required 

-rn 

REPORTNAME 

Enter the name you would like to give the data quality report, otherwise the default format will be used (AttributeRulesReport_MMDDYYYY-TTTTTT) 

Optional 

 

List of required and optional inputs in Command Prompt.List of required and optional inputs in Command Prompt.

Python Toolbox UI in ArcGIS Pro: The tooltips and metadata will not appear for the custom python toolbox Data Quality Report tool unless the DataReviewerDataQualityReport.ReviewerErrorReport.pyt.xml is stored in the same folder as the DataReviewerDataQuality.pyt.Python Toolbox UI in ArcGIS Pro: The tooltips and metadata will not appear for the custom python toolbox Data Quality Report tool unless the DataReviewerDataQualityReport.ReviewerErrorReport.pyt.xml is stored in the same folder as the DataReviewerDataQuality.pyt.

Download

Contents of DQReport_Pro3.7.zip: 

  • AttributeRulesReport.py – The stand-alone python script.
  • DataReviewerDataQuality.pyt – The Python Toolbox. 
  • DataReviewerDataQualityReport.ReviewerErrorReport.pyt.xml – Contains the metadata and tooltips for the Python Toolbox. Must be stored in the same folder as DataReviewerDataQuality.pyt for tool tips to appear.
Contributors
About the Author
Bailey Ohlson is a Product Engineer on the Data Reviewer team in the Esri Professional Services R&D Center. In her role, she develops and maintains QA/QC automation scripts, identifies software requirements, defines software test cases, and validates software repairs. Bailey earned a BS in Geology from Texas A&M University and a MA in Geography from the University of Texas at Austin.