New Tool for Analyzing Attribute Rule Performance with ArcGIS Data Reviewer

[Jaspar_Saadi-Klein]

Why this tool was created

Understanding the performance of ArcGIS Data Reviewer-based attribute rules in ArcGIS Server can be challenging, particularly when evaluating large volumes of data. This tool was developed in response to a series of user troubleshooting engagements where users were trying to understand where time is actually being spent during validation attribute rule evaluation. By helping users analyze and break down the time for Data Reviewer attribute rule execution, the tool provides greater visibility into validation rule performance, making it easier to identify bottlenecks, troubleshoot issues, and optimize enterprise workflows.

How to use the tool

Complete the following steps to run the log parser tool:

1. Set your ArcGIS Server’s log level to Debug. Follow these instructions to specify the server log settings in ArcGIS Server Manager.
2. Evaluate your attribute rules
   1. As long as there are Data Reviewer-based validation attribute rules authored on your data, there should be log messages that this tool can parse.
3. Locate the folder on the ArcGIS Server machine where the server logs for the service are stored. There are two places where the logs are stored depending on if you evaluated your validation attribute rules synchronously or asynchronously.  
   1. Server logs for synchronous evaluations are located in logs > machine name > services > service name.MapServer
      1.        Windows: C:\arcgisserver\logs\<machine name>\services\<service name>.MapServer
      2.         Linux: <ArcGIS Server installation directory>/logs/<machine name>/services/<service name>.MapServer
   2. Server logs for asynchronous evaluations are located in logs > machine name > services > System > ValidationTools.GPServer
      1.        Windows: C:\arcgisserver\logs\<machine name>\services\System\ValidationTools.GPServer
      2.         Linux: <ArcGIS Server installation directory>/logs/<machine name>/services/System/ValidationTools.GPServer
4. Run the tool in ArcGIS Pro or from the terminal.
   1. To run the tool in ArcGIS Pro
      1. Save the ReviewerLogParser.pyt file to a known location
      2. Copy all of the server logs to the machine with ArcGIS Pro
      3. Open ArcGIS Pro and add the toolbox to a project
      4. Open the Reviewer Log Parser tool
      5. For the Log Folder parameter, browse to the folder containing the server logs and click OK
      6. Specify where to place the parsed excel output
      7. Run the tool

         

   2. To run the tool in the terminal via the python interactive shell on the server machine with the logs:
      1. Save the ReviewerLogParser.pyt file to a known location on the server machine
      2. Identify the server log location
      3. Launch the interactive python shell tied to ArcGIS Enterprise
         1. For Windows it is typically "C:\Program Files\ArcGIS\Server\framework\runtime\ArcGIS\bin\Python\envs\arcgispro-py3\python"
         2. For Linux it is typically <ArcGIS Server installation location>/arcgis/server/tools/python
      4. Import arcpy
      5. Import the toolbox
         1. arcpy.ImportToolbox(path/to/ReviewerLogParser.pyt)
      6. Run the tool
         1. arcpy.ReviewerLogParser_datareviewer(log_folder="/path/to/log/folder",output_excel="/output/excel/parsed_output.xlsx")
      7. Copy the output .xlsx and .csv to a machine that has Microsoft Excel (or an Excel viewer) installed and view the results.

How the tool works

At a high level, every evaluation consists of three phases: reading, validation, and writing. Reading includes the time spent syncing geodatabase features into the temporary SQLite tables preparing them for processing. Validation encompasses the running of Data Reviewer attribute rules. Writing represents the tasks needed to update the geodatabase with results, such as recording new errors, updating fixed errors, updating existing errors, and updating the validation status field.

When Data Reviewer-based validation attribute rules are evaluated, ArcGIS Server uses several optimizations to process rules more efficiently. One of these optimizations is the creation of a temporary SQLite database that contains only those features and fields needed for the evaluation, allowing multiple Data Reviewer attribute rules in the same evaluation to use the same cached data. Similar evaluations can also be combined to improve overall processing efficiency.

The Reviewer Log Parser helps determine where time is being spent during the evaluation process. It reads ArcGIS Server log files from a selected folder and looks for Data Reviewer validation messages written by the ReviewerUtilities method. The tool then groups related messages by request ID, allowing each validation to be analyzed separately. For every log message, the tool captures the message text, timestamp, elapsed time, source log file, and line number. It also calculates a Unix timestamp to support chronological sorting and downstream analysis.

To support general awareness and deeper troubleshooting, the tool produces two outputs. The first is a CSV file that contains the output details of each log message as described above. The second is an Excel workbook that summarizes each validation run in a more readable format. The workbook opens with a README worksheet that explains the report assumptions, followed by one worksheet per request ID (named using the first 31 characters of the request ID). Each run worksheet lists the parsed operations, describes what each operation represents, breaks elapsed time into Reading, Validation, and Writing time in seconds, and includes phase totals formatted as hours, minutes, and seconds.

Note: When building the run worksheets, the parser tries to avoid double-counting sync time. If a sync operation occurs during a validation operation, the tool subtracts that sync time from the validation total and reports it separately as Reading Time.

If you need additional detail, the CSV output lets you trace a summarized operation back to the original log message. Start with the worksheet name, which is based on the first 31 characters of the request ID. Then open the CSV, find the matching request ID, and compare the operation text from the workbook to the message field in the CSV. Because the CSV also includes the source log path and line number, you can use it to locate the original message in the ArcGIS Server log file.

Known limitations and expectations when running the tool

With this tool come some important items to be aware of. They are as follows:

1.  The tool only parses those messages (ReviewerUtilities) created when Reviewer-based validation attribute rules are evaluated. There are additional messages (both Reviewer and non-Reviewer) related to the validation process being generated that are not captured by this tool.
2. The tool does not parse Arcade-based validation attribute rules.
   1. See Troubleshoot attribute rule behavior with ArcGIS Diagnostic Monitor and ArcGIS Enterprise Server log... to learn more about troubleshooting Arcade-base attribute rules.
3. The current version of this tool does not capture error messages like timeouts or missing data source related logs.
   1. To track these back you may need to start with the excel, see where there are no entries or entries are zeroed out, track that back to the csv (which includes the log and line) and then from there examine the raw log around that message to see specifically what is going on.
4. When examining the validation time, specific Data Reviewer check types are only listed once even if there are multiple instances of that check being evaluated.
5. When examining the validation time, it is expected that some checks will take longer than others to complete due to their differing complexity or the number of features they are running on. For example, rules based on the Valency check are more time-intensive when compared to other checks due to the complexity of the spatial relationships under evaluation.
6. When evaluating attribute rules, the total rule evaluation time is only as fast as the slowest rule. This is especially important when using a mix of Arcade and Data Reviewer attribute rules

Feedback welcome

This is the first version of the tool, and we welcome your feedback. Whether you’re using it to troubleshoot performance issues or just better understand validation behavior, we’d like to hear your experience. Feedback, enhancement ideas, and bug reports will help guide future improvements and help make the tools even more valuable.

ReviewerLogParserV0.2.4.zip