Implementing ArcGIS Blog

GIS Enterprise Reporter now has a companion application “er_compare.exe” that allows you to compare two ArcGIS Enterprise systems.

Measured transaction throughput and response times are critical data of any ArcGIS Enterprise load test, but captured hardware utilization of the deployment machines provide vital information as well. Together, these test artifacts allow for the proper analysis of the capabilities and efficiencies of the Site.

Determining that an application or feature service has achieved a particular level of throughput is good, but confirming the scalability characteristics while also examining the captured processor utilization of the tested workload is even better.

This Article will discuss several ways to capture the machine hardware utilization. This resource usage is a great compliment to the results of an Apache JMeter load test of ArcGIS Enterprise and can help further the analysis. The Article will focus on the most common scenarios using free tools and utilities for Windows and Linux.

Table Of Contents

• Capturing Hardware Utilization During an Apache JMeter Load Test
• Capturing Strategies
  • What Information Should the Load Test Capture?
  • Sample Interval
  • The Difference Between Capturing and Monitoring
  • Test Clients (Load Generator Machines)
  • Understand Your Server Baselines
  • Common Utilization Capturing Examples
    • Using Perfmon (To Capture Utilization Outside of Load Test)
    • Using dstat (To Capture Utilization Outside of Load Test)
    • Using Apache JMeter (To Capture Utilization Directly From Load Test)
• The SampleWorldCities Test Plan With Metric Collection Support 
  • Components of the Test Plan
    • jp@gc - PerfMon Metrics Collector Extension
  • Validate the JMeter-to-ServerAgent Connectivity
• Test Execution
• General Methodology Guide
• Common Utilization Collection Challenges
• Final Thoughts

Looking for a quick way to summarize the details of all your ArcGIS Enterprise services? If so, then System Log Parser's ServiceDetails Analysis Type is highly recommended. This feature performs a scan of your Site through the REST Admin API and presents the gathered information into a spreadsheet report for easy digestion.

The report is a great resource to find details on your service provider statistics, service configuration info, or data source analysis.

Apache JMeter is a great load testing tool, but it's a fantastic automation framework too! This Article focuses on using JMeter to call the ArcGIS REST API in order to carry out user management tasks that would be tedious for large numbers of members. 

Table Of Contents

• Administration Automation with Apache JMeter
• The User Administration Test Plans
  • The CSV Data Set Config of Users
  • The Text File List of Users
  • Administrator Login
• Add a New User (portal_users_add1)
• Add a New User (portal_users_add2)
• Add a New User (portal_users_add3)
• Set the Security Question/Answer for New Users (portal_users_update_profile1)
• Disable a User (portal_users_disable1)
• Enable a User (portal_users_enable1)
• The Thread Group Configuration
• Test Execution
• Validating the Test Plans
  • Troubleshooting a Command-line Test Execution
• Final Thoughts
  • A Quick Word on Using Multiple Threads 
  • A Quick Word on Deleting Users

A geoprocessing (GP) services is essentially a resource that is setup to perform or automate any ArcGIS Enterprise task. The purpose of this tool could be something that works on a feature class or table and creates an output dataset. The work performed could be to take in a set of inputs and generate a buffer or project to another spatial reference.

They take the form of being either synchronous or asynchronous. This is known as the execution mode.
Synchronous geoprocessing services are ideal for small jobs where the required computation to complete the task is very quick (e.g. just a few seconds). In this case, all the requests involved in the process are short-lived and load testing such a service would be straight-forward: construct a test to send the requests as they (sequentially) appear when interacting with the service. Once the job request is submitted, the response returns as soon as it has completed.

Asynchronous geoprocessing services on the other hand, have more moving parts. Since the task could be very long running, once the job has been submitted, its up to the user (the tester) to periodically check on the status of job with a separate request. Status state checking logic inside a loop introduces complexity to the test.

This article can be used as a guide for load testing an Asynchronous GP service. While it is specific to the "Summarizing Invasive Species" example, the majority of the test and its logic can be applied to other geoprocessing services.

Table Of Contents

• Why Test a Asynchronous Geoprocessing Service?
• Asynchronous Geoprocessing Service Testing Challenges 
• How to Test an Asynchronous Geoprocessing Service?
  • The "Summarize Invasive Species" Geoprocessing Model and Dataset
  • Test Data Generation
• The Asynchronous Geoprocessing Service Test Plan
  • Components of the Test Plan
    • SubmitJob
    • CSV Data Set Config
    • InitialJobStatus
    • LoopJobStatus
    • WhileLoop
      • JobStatusCheck
      • ResponseAssertion
      • SleepWhileLoop
      • IfWhileLoopMax
    • IfJobSucceeded
    • DownloadOutput
    • IfTestIterationMax
• The Thread Group Configuration
• Validating the Test Plan
  • Transactions
  • Requests
• Test Execution
• JMeter Report
  • Throughput Curves
  • Performance Curves
• Final Thoughts
  • A Quick Word on Sizing
  • A Quick Word on Instances
  • A Quick Word on the Location of the arcgisjobs Folder

Cached map services are a popular and recommended way to provide a fast performing presentation of static data. When map services are cached, the server draws the data at a set of map scale levels you define and saves the images. Then, as the server received requests for the map, it is much quicker (and requires less CPU resources) to retrieve the pre-created cached images than it is to dynamically draw the map again. This is the benefit of cached map services.

Generally speaking, cached map services can support simultaneous requests from many users. Such services can scale well. Despite this well known performance characteristic, it is still a common requirement to load test a cached map service to see first-hand how it behaves and scales on the hardware of a particular ArcGIS Enterprise deployment.

This Article contains a walk-through for load testing a cached map service with an Apache JMeter Test Plan. The examples use a public accessible dataset. Using datasets in Coordinate Systems other than WGS 1984 Web Mercator Auxiliary_Sphere (WKID: 3857) should work, but the generated test data will be based on a WKID of 3857.

Table Of Contents

• Why Test a Cached Map Service?
• Cached Map Service Testing Challenges
• How to Test a Cached Map Service?
  • The Natural Earth Dataset
  • Test Data Generation
    • The Generate Bounding Boxes (Precision) Tool
    • Visualizing the Generated Data in ArcGIS Pro
    • Visualizing the Generated Data in a Text Editor
• The Cached Map Service Test Plan
  • Components of the Test Plan
    • CSV Data Set Config
    • Levels Of Detail List Logic
    • GetMapTile Logic
    • Cache Tile Loop and Path Population
      • Loop Controller
      • Counter
      • JSR223 Sampler
      • HTTP Request
• The Thread Group Configuration
• Validating the Test Plan
  • Transactions
  • Requests
• Test Execution
• JMeter Report
  • Throughput Curve
  • Performance Curve
• Final Thoughts
  • Additional Items Worth Mentioning
  • Assumptions and Constraints

Network Analyst has always been in-demand capability of ArcGIS Enterprise as it provides popular problem solving analysis types such as Route, Service Area, Closest Facility and Origin-Destination Cost Matrix.

This Article contains a walk-through for load testing a route solver service (map-based service) with an Apache JMeter Test Plan. It uses a public accessible dataset in the examples for the data generation and test configuration process. However, the testing procedure should work when using any valid network dataset. This Article assumes you are familiar with Apache JMeter and some of the Test Plan strategies we have been using in other discussions.

Table Of Contents

• Network Analyst Route
• Load Testing a Network Analyst Route Service
• How to Test a Network Analyst Route Service?
  • Network Analyst ArcGIS Pro Tutorial Data
  • Test Data Generation
    • Making the Tools Available from ArcGIS Pro
    • Launch the Generate Data (Solve Route) Tool
    • The Generate Data (Solve Route) Tool Inputs
    • Spatially Visualize the Generated Points
• The Route Solver Test Plan
  • HTTP Request
  • The Thread Group Configuration
• Validating the Test Plan
  • Transactions
  • Requests
• Test Execution
• JMeter Report
  • Throughput Curve
  • Performance Curves
• Final Thoughts

If you have a multiple-machine Portal for ArcGIS or ArcGIS Server site, you have a dependency on a file share.  If you are having a problem with your site, how do know whether the file share might be causing it?  This article is intended to help with that challenge.

The load testing of ArcGIS Enterprise can be carried out using a variety of different tools like: JMeter, LoadRunner or Silk Performer. While all test tools essentially perform the same task of applying pressure to a particular resource and measuring its impact, having a listing of the meanings to common phrases can help ensure testers and analysts are on the same page.

This resource provides insight into some testing fundamentals that are mentioned in other Community Articles about performance and load testing of ArcGIS Enterprise. The vocabulary definitions are listed from a chronological point of view. It all starts with a request so this is the first item discussed.