Implementing ArcGIS Blog - Page 3

The load testing of ArcGIS Enterprise can be carried out using a variety of different tools like: JMeter, LoadRunner or Silk Performer. While all test tools essentially perform the same task of applying pressure to a particular resource and measuring its impact, having a listing of the meanings to common phrases can help ensure testers and analysts are on the same page.

This resource provides insight into some testing fundamentals that are mentioned in other Community Articles about performance and load testing of ArcGIS Enterprise. The vocabulary definitions are listed from a chronological point of view. It all starts with a request so this is the first item discussed.

We are excited to offer Data Health Check appointments at this year's User Conference.  It's been a popular offering at UC for the past 10 years.  Users get to meet with an Esri industry expert, who will assess the quality of their GIS data (geometry and attributes) for use cases such as getting ready to migrate to the Utility Network or Parcel Fabric, connectivity issues in their geometric network, report/submit data quality, understand inherited data, readiness to migrate to indoors data model, and more.

Customers with water, wastewater, stormwater, electric, gas, linear referenced pipelines, addressing, land records, street centerlines, airports (airfield operations), and Indoors datasets can sign up for a free, 45-min virtual appointment. 

Learn more about this awesome offering here.

If you are ready to sign-up, click here.

As the foundational software system for GIS, ArcGIS Enterprise performs many duties such mapping and visualization, analytics. If one function were to be used as a benchmark for testing an ArcGIS Enterprise deployment, a strong case can be made for the map service export function. Export map can be called easily in an Apache JMeter Test Plan by varying the spatial extents of the requests across several map scales.

This Article provides a discussion on the key components of running such a benchmark like the dataset, service instances and step load. For comparison purposes, results are also listed of the benchmark performed within an Esri test environment.

Table Of Contents

• Choosing a Capability of ArcGIS Enterprise to Benchmark
• What is a Benchmark of a Map Service?
• Benchmark Dataset
  • What is Public Domain Data?
  • Why Use Public Domain Data?
  • SampleWorldCities vs Natural Earth
    • The Benchmark Natural Earth Dataset
  • Deployment Architecture
  • Data Source Type and Location
  • Service Type and Number of Instances
  • Do the Request Options in a Benchmark Test Matter?
• The Map Service Benchmark Test Plan
  • The Thread Group Configuration
• Benchmark Test Execution
• Results and Analysis
  • The Ideal Throughput Curve
  • The Ideal Performance Curve
• JMeter Report
  • Actual Throughput Curve
  • Actual Performance Curve
• Comparing the Results

Performance Engineering: Load Testing ArcGIS Enterprise

Performance Engineering is the practice of proactively testing, monitoring and analyzing an ArcGIS Enterprise deployment or application from the perspective of performance and/or scalability. System performance and scalability are critical factors in the successful adoption, operation, and long-term use of an ArcGIS Enterprise deployment.  They are often key determinants of end-user satisfaction.

From this post can link you to all of our Articles related to strategies and practices with Load Testing ArcGIS Enterprise.

Apache JMeter is an open source Java software program for performance and scalability load testing. It is a free tool with GUI and command-line support for Windows, Linux and MacOSX that has many enterprise-level features.

This Article contains a walk-through for taking a SampleWorldCities map service Test Plan (created from previous a Article) and discussing how it can be extended to support ArcGIS Enterprise Authentication. It assumes you are familiar with Apache JMeter and some of the Test Plan strategies we have been using in other Articles. For a refresher, please see: 

• Performance Testing with Apache JMeter (An Introduction)
• Creating a Load Test in Apache JMeter against the SampleWorldCites Map Service (Beginner/Intermediat...
• Running an Apache JMeter Load Test from Command-line mode (Beginner/Intermediate)

Table Of Contents

• Why Add Authentication to a Test Plan?
• Getting Started
• Creating the sampleworldcities4 Test Plan
• Extending the Test Plan to Add Authentication Support
  • User Defined Variables
  • Authentication Requests to Acquire a Token
    • OAuthState HTTP Request
    • OAuthState Regular Expression Extractor
    • AccessToken HTTP Request
    • AccessToken Regular Expression Extractor
    • Token HTTP Request
    • Token Regular Expression Extractor
    • Token Response Assertion
  • Token Support in the Map Request Headers
• Validating the Test Plan

Apache JMeter is an open source Java software program for performance and scalability load testing. It is a free tool with GUI and command-line support for Windows, Linux and MacOSX with many enterprise-level features.

This Article contains strategies and procedures for running a Test Plan against an ArcGIS map service using the command-line mode in Windows. It assumes you are familiar with some of the basic JMeter Test Plan concepts. If you are new to Apache JMeter, please see our Performance Testing with Apache JMeter (An Introduction). 

Table Of Contents

• Why Run Test Plans from Command-line mode?
• Verifying the Pre-Test Checklist
• Batch Script Walkthrough
  • Variables
    • Memory
    • JMeter and Project Paths
    • Test Execution and Switches
  • Putting It All Together
• Running the Load Test
• Test Artifacts
• Preparing for Analysis

Apache JMeter is an open source Java software program for performance and scalability load testing. It is a free tool with GUI and command-line support for Windows, Linux and MacOSX with many enterprise-level features.

This Article contains strategies and procedures for creating a dynamic load test against an ArcGIS Server service and validating the response. It assumes you are familiar with some of the basics of Apache JMeter. If you are new to Apache JMeter, please see our Performance Testing with Apache JMeter (An Introduction). 

Table Of Contents

• Prerequisite
• JMeter Testing Strategies
  • A Good Test Plan Starts with a Proper Folder Structure
  • User Defined Variables
  • Choosing the Right Thread Group
    • Adding Support for Custom Thread Groups into JMeter
  • Using Transactions to Group Requests
  • Validate the Response is Being Returned
• Creating a Data Driven, Export Map Test in JMeter
  • Create and Save a New Test Plan
  • Adding Variables to the Test Plan
  • Adding the bzm – Concurrency Thread Group to the Test Plan
  • Adding a Transaction to a Test Plan
  • Adding an HTTP Request
  • Making the HTTP Request Dynamic
  • Adding a Response Assertion
  • Adding a CSV Data Set Config
• Validating the Test Plan
  • Saving Recent Changes
• Preparing to Run the Load Test

Apache JMeter is an open source Java software program for performance and scalability load testing. It is a free tool with GUI and command-line support for Windows, Linux and MacOSX with many enterprise-level features.

This Article will step you through the installation and setup of an Apache JMeter environment on Windows. Additionally, it will cover how to construct a very basic performance test that requests a static resource and how to visually validate the responses coming from the remote server.

Table Of Contents

• What is Apache JMeter?
  • What can I do with it?
    • JMeter Behavior
  • Why do I need it?
    • Performance Testing
    • Scalability Testing
• Installing Apache JMeter
  • Grab the Latest
  • OpenJDK or Oracle JDK?
  • Installing the Plugins Manager
• Running JMeter (GUI)
• Creating a Very Basic, Static Test Plan
• Validating the Test Plan

The load testing of ArcGIS Server and its services can be conducted for several reasons: to understand how published services perform, to determine if there are any known hardware limitations or bottlenecks, to observe the stability of a service over time or to measure the scalability of a service.

The following list of testing strategies is a common a practice within Esri Professional Services followed by many analysts. 
This list can be used to work with any load test software to help act as a guide for the tester/analyst to achieve the best results and to maximize the testing effort.

Table Of Contents

• Start with a test plan
• Tests should be conducted against application with no known major bugs or defects
• Interact with your application (and ArcGIS services) before load testing them
• Coordinate test execution
• Verify that the Test environment matches expectation
• Start a load test at step 1
• Collect hardware utilization from all machines involved in the test
• Test individual ArcGIS services first
• Test as physical close to the deployment as possible
• Step and test duration
• Be mindful of the ArcGIS Server log level
• Traditional ArcGIS services can be tuned within (ArcGIS) Server
• Not all ArcGIS services are CPU bound
• It’s all about throughput (not users)
• Verify that the test was successful
• Test results are not a guarantee of the support for X number of users
• Avoid testing shared resources like ArcGIS Online or Google Maps
• Repeatable test results
• Understand the Value
• Design of tests/workflows should be realistic and based on what would be expected from a user
• Keep it simple