Implementing ArcGIS Blog

Curious about the performance profile of the ArcGIS Server machine in your ArcGIS Enterprise?
This Article discusses an easy approach for using the Geometry Service to benchmark the underlying processing hardware.

Table of Contents

• Benchmark ArcGIS Enterprise...The Original Approach
• Benchmark ArcGIS Enterprise...An Alternative Approach
• Benefits of Using the Geometry service
• The Geometry_Functions_Benchmark Test Plan
• What Types of Functions Should be Tested?
• Expected Test and Operation Performance
• Deployment Architecture Example
• Benchmark Test Execution
  • Results
  • Calculate Percentage Change
  • Percentage Change Estimate Based on SPEC
• Future Hardware
• Final Thoughts 
  • A Fast Response Time Is All About CPU Speed, Right?
  • What About Other CPU Comparison Tools?
    
    

When people talk about load testing an ArcGIS Enterprise Site, such conversations typically involve the consumption of dedicated or hosted feature services.
For many years, dedicated and hosted services have provided a fast, dependable mechanism for consuming high-traffic map resources online. This has not changed.

However, there is another type of resource to provide maps to users: the Shared Service Instance Pool.
Introduced in 10.7, the shared instances pool make it easier to view and query services that are still important but where memory usage is favored over performance.
This allows for high service density publishing (e.g., being able to publish and have running many services) at the expense of some speed and throughput. It can be a good trade-off considering that for many organizations, there are generally more shared service candidates than dedicated or hosted.

From this advantageous characteristic, shared services have been a true game changer. But, from a load test perspective there are some considerations.

The purpose of this article is to discuss strategies and concepts for using a JMeter Test Plan to exercise branch versioning editing. The test takes a programmatic approach for creating and capturing the data of important key/value pairs for several of the critical requests.
While a GIS editing test can be complicated since there are many moving parts to keep track of, this Article attempts to simplify the process as much as possible.
The tested workflow utilizes the Natural Earth dataset that is published as a web feature layer with branch versioning enabled.

Table Of Contents

• Branch Versioning
• Why Test Branch Versioning?
• Branch Versioning Editing Testing Challenges
• How to Test Branch Versioning Editing?
  • The Natural Earth Dataset
  • The ArcGIS Pro Feature Cache
• The Editing Workflow via ArcGIS Pro 
• The Branch Versioning Editing JMeter Test Plan
  • Components of the Test Plan
    • CSV Data Set Config
  • (User Test Thread) Initialization Transaction
  • Open ArcGIS Pro Project Transaction
  • Create Version and Switch Version Transaction
  • Add Point Spatially Transaction
  • Update Point Attributes Transaction
  • Polygon Transactions
  • Save Edits (to the Named Version) Transaction
  • Reconcile With Default Transaction
  • Post To Default Transaction
    • Managing the Named Versions
  • Zoom Out (to Scale 1 Million) Transaction
• Final Thoughts
  • Running the Load Test
  • Historic Moment Testing Details
  • Operation Concurrency

Performance with your services and map are extremely important. Better performance means higher scalability which translate to your Site supporting more users. However, there are challenges GIS Administrators face when dealing with performance. This Article discusses some common performance challenges and strategies used to overcome them.

Table of Contents

• Performance: Challenges and Strategies
• What is Performance?
• Why is Performance Important?
• What is Acceptable Performance?
• How is Performance Measured?
  • Capturing Response Times -- Web Browser
  • Capturing Response Times -- ArcGIS Pro
• Are Goals Required for Improving Performance?
• Common Performance Challenges and Potential Strategies
  • Service Pool Types and Instances
    • Selecting the Appropriate Type
  • Focus the Map
    • Map Strategies
  • Software Releases
  • Resource Contention and Expansion
    • Scalability
    • Performance
    • A General Approach to Scaling
  • Observability
    • A Challenge with Common Observability Tools
• Conclusion
  
  

Optimizing the ArcSOC instance availability and utilization for your service is a good strategy for helping users obtain fast response times and lower wait times from their dynamic requests to your Site. It can also benefit server resource utilization like memory as the service is not running a lot of instances that it will never use.

But optimizing the minimum and maximum number of instances for your dedicated services is not a one-time job. Usage patterns of your services can change over time so the task of collecting this information is something that you will want and to revisit periodically as a GIS administrator.

Before diving into how to observe ArcSOC instance activity statistics, let’s review some of the key details of the two ArcSOC-based service types in ArcGIS Server and how they play into this discussion:

• Dedicated
• Shared

Table Of Contents

• ArcSOC Availability and Utilization
• Dedicated Instance Pool Services
• Limitations of Shared Instance Pool Services
• Configured ArcSOC Instance Availability vs Instance Demand
• ArcGIS Server Service Report
• Automating the Service Report Collection with Soccer
• Analyzing the CSV File
• Final Thoughts

GIS Enterprise Reporter now has a companion application “er_compare.exe” that allows you to compare two ArcGIS Enterprise systems.

Measured transaction throughput and response times are critical data of any ArcGIS Enterprise load test, but captured hardware utilization of the deployment machines provide vital information as well. Together, these test artifacts allow for the proper analysis of the capabilities and efficiencies of the Site.

Determining that an application or feature service has achieved a particular level of throughput is good, but confirming the scalability characteristics while also examining the captured processor utilization of the tested workload is even better.

This Article will discuss several ways to capture the machine hardware utilization. This resource usage is a great compliment to the results of an Apache JMeter load test of ArcGIS Enterprise and can help further the analysis. The Article will focus on the most common scenarios using free tools and utilities for Windows and Linux.

Table Of Contents

• Capturing Hardware Utilization During an Apache JMeter Load Test
• Capturing Strategies
  • What Information Should the Load Test Capture?
  • Sample Interval
  • The Difference Between Capturing and Monitoring
  • Test Clients (Load Generator Machines)
  • Understand Your Server Baselines
  • Common Utilization Capturing Examples
    • Using Perfmon (To Capture Utilization Outside of Load Test)
    • Using dstat (To Capture Utilization Outside of Load Test)
    • Using Apache JMeter (To Capture Utilization Directly From Load Test)
• The SampleWorldCities Test Plan With Metric Collection Support 
  • Components of the Test Plan
    • jp@gc - PerfMon Metrics Collector Extension
  • Validate the JMeter-to-ServerAgent Connectivity
• Test Execution
• General Methodology Guide
• Common Utilization Collection Challenges
• Final Thoughts

Looking for a quick way to summarize the details of all your ArcGIS Enterprise services? If so, then System Log Parser's ServiceDetails Analysis Type is highly recommended. This feature performs a scan of your Site through the REST Admin API and presents the gathered information into a spreadsheet report for easy digestion.

The report is a great resource to find details on your service provider statistics, service configuration info, or data source analysis.

Apache JMeter is a great load testing tool, but it's a fantastic automation framework too! This Article focuses on using JMeter to call the ArcGIS REST API in order to carry out user management tasks that would be tedious for large numbers of members. 

Table Of Contents

• Administration Automation with Apache JMeter
• The User Administration Test Plans
  • The CSV Data Set Config of Users
  • The Text File List of Users
  • Administrator Login
• Add a New User (portal_users_add1)
• Add a New User (portal_users_add2)
• Add a New User (portal_users_add3)
• Set the Security Question/Answer for New Users (portal_users_update_profile1)
• Disable a User (portal_users_disable1)
• Enable a User (portal_users_enable1)
• The Thread Group Configuration
• Test Execution
• Validating the Test Plans
  • Troubleshooting a Command-line Test Execution
• Final Thoughts
  • A Quick Word on Using Multiple Threads 
  • A Quick Word on Deleting Users

A geoprocessing (GP) services is essentially a resource that is setup to perform or automate any ArcGIS Enterprise task. The purpose of this tool could be something that works on a feature class or table and creates an output dataset. The work performed could be to take in a set of inputs and generate a buffer or project to another spatial reference.

They take the form of being either synchronous or asynchronous. This is known as the execution mode.
Synchronous geoprocessing services are ideal for small jobs where the required computation to complete the task is very quick (e.g. just a few seconds). In this case, all the requests involved in the process are short-lived and load testing such a service would be straight-forward: construct a test to send the requests as they (sequentially) appear when interacting with the service. Once the job request is submitted, the response returns as soon as it has completed.

Asynchronous geoprocessing services on the other hand, have more moving parts. Since the task could be very long running, once the job has been submitted, its up to the user (the tester) to periodically check on the status of job with a separate request. Status state checking logic inside a loop introduces complexity to the test.

This article can be used as a guide for load testing an Asynchronous GP service. While it is specific to the "Summarizing Invasive Species" example, the majority of the test and its logic can be applied to other geoprocessing services.

Table Of Contents

• Why Test a Asynchronous Geoprocessing Service?
• Asynchronous Geoprocessing Service Testing Challenges 
• How to Test an Asynchronous Geoprocessing Service?
  • The "Summarize Invasive Species" Geoprocessing Model and Dataset
  • Test Data Generation
• The Asynchronous Geoprocessing Service Test Plan
  • Components of the Test Plan
    • SubmitJob
    • CSV Data Set Config
    • InitialJobStatus
    • LoopJobStatus
    • WhileLoop
      • JobStatusCheck
      • ResponseAssertion
      • SleepWhileLoop
      • IfWhileLoopMax
    • IfJobSucceeded
    • DownloadOutput
    • IfTestIterationMax
• The Thread Group Configuration
• Validating the Test Plan
  • Transactions
  • Requests
• Test Execution
• JMeter Report
  • Throughput Curves
  • Performance Curves
• Final Thoughts
  • A Quick Word on Sizing
  • A Quick Word on Instances
  • A Quick Word on the Location of the arcgisjobs Folder