Performance Testing with Apache JMeter (An Introduction)
What is Apache JMeter?
Apache JMeter is an open source Java software program (currently released under the Apache License, version 2.0) for performance and scalability load testing. It is a free tool with GUI and command-line support for Windows, Linux and MacOSX with many enterprise-level features.
What can I do with it?
Apache JMeter may be used to test the performance and scalability of both on static and dynamic resources like ArcGIS services as well as web applications within an ArcGIS Enterprise Site.
It can exercise many different applications and protocols, for example:
- Web - HTTP, HTTPS
- SOAP / REST Webservices
- Native commands or shell scripts
It provides a full featured IDE for constructing and debugging a test as well as the recording of workflow traffic (from web browsers) directly into a project. This Article refers to the IDE as "the GUI".
At the end of a test, an HTML report can be created automatically to assist with the analysis of the results.
JMeter Behavior
JMeter works at the protocol level (e.g. HTTPS) and does not try to mimic a web browser. As such, it does not execute any JavaScript logic found in web applications nor does it attempt to render the HTML pages like a browser.
Why do I need it?
Being able to verify system performance, scalability and/or stability are critical for the successful adoption of any GIS deployment. Having a modern testing tool to help quantify the ArcGIS capabilities can optimize the investment in and use of the deployment.
By using a free or open source application to the conduct tests, the analyst can work with the users to avoid the distraction of cost while just focusing on the tests and their results.
Performance Testing
Single user performance profiling and troubleshooting of an ArcGIS service covers most testing cases. JMeter can become a great compliment to the other tools in the GIS testing "tool chest":
- Web Browser – Chrome, Firefox, Safari
- HTTP Debugger -- Fiddler (Classic)
- ArcGIS Log analysis – System Log Parser
- Desktop -- PerfQAnalyzer/mxdperfstat/PerfTools
- Performance Testing/Troubleshooting Analysis
- Apache JMeter
Single user performance testing and analysis is typically a first step before applying load as it can help identify if a resource is suitable for the heavier testing. In other words, if something is “slow” with one user, it will be slow(er) with 20 users so, there is little value in moving to the load test until performance is improved through tuning.
Scalability Testing
Scalability testing typically conducted when necessary to understand performance under load (e.g. the simulation of multiple users).
However, load testing:
- Requires different expertise
- Has more options to configure
- Can be more complex
That said, having a broad community of support for a tool like proven tool like JMeter can help lessen this complexity:
- Lots of HTTPS test samples/walkthroughs online
- But, specific Test Plan examples for ArcGIS services might be hard to find
- Our Articles and resources will try to bridge this gap and will be geared toward the GIS community
- Knowledgeable colleagues
Installing Apache JMeter
Grab the Latest
Download the latest from the Apache JMeter site; January 2024 -– version 5.6.3
Note: JMeter should be installed on a dedicated test client machine (not on your ArcGIS Enterprise servers)
Once downloaded, unzip the package to a local directory
- C:\apache-jmeter-5.6.3
For more information and options, see the JMeter Getting Started resource, Installation section: https://jmeter.apache.org/usermanual/get-started.html
Apache JMeter is available for Windows, Linux and MacOSX environments. Our screenshots and examples are based on running JMeter (v5.3) from a Windows environment but also apply to v5.4.x -- v5.6.x setups.
OpenJDK or Oracle JDK?
A Java development kit (or runtime) is required to run Apache JMeter, and either the OpenJDK or Oracle JDK will accomplish this task.
- OpenJDKs – License
- July 2022 -– version 17GA
- Latest release supported by latest JMeter
- Version 17.0.2 (build 17.0.2+8) recommended for running our test examples
- July 2022 -– version 17GA
- Oracle JDK – License
- July 2022 -– version 8U333x64
Note: Neither Apache JMeter nor the OpenJDK have an actual “installer”. They are both just compressed downloads. In some ways, this makes their deployment easier.
Note: It is recommended to always run the latest version of JMeter. If you are also updating your JDK release, check that it is supported by the JMeter version you are using. From Apache JMeter: "JMeter 5.6.x requires Java 8 or later for execution (Java 17 or later recommended)."
Once the Java is downloaded (e.g. the OpenJDK), unzip the package to a local directory. For example:
- C:\jdk-17.0.2
or
- C:\OpenJDK
In either case, the Java included bin folder exists the directory listed above (along with several other folders like conf, include, jmods, lib, etc…).
Add the environment variable JAVA_HOME to your system that points to this path:
- JAVA_HOME= C:\jdk-17.0.2
Add the “JAVA_HOME bin” path to your system’s PATH environment variable:
- %JAVA_HOME%\bin
Installing the Plugins Manager
The JMeter Plugins Manager is a free component and highly recommended for adding functionality that is not included with the base tool. Our test examples will make use of technology that is added via the Plugins Manager.
Download the JAR file, put it into JMeter’s lib/ext directory then (re)start JMeter.
Running JMeter (GUI)
To start the JMeter GUI mode, run the appropriate “jmeter” script file for your environment:
- C:\apache-jmeter-5.6.3\bin\jmeter.bat
The GUI is a handy interface for constructing and verifying your test is working as expected
The JMeter GUI Article screenshots are based on the Metal theme found under Options-->Look and Feel-->Metal (the default theme is “Darklaf – Darcula”)
Note: Apache JMeter advises to not run a load test from the GUI. The GUI should only be used to build and debug the test.
Steps for running the test from the command line are covered in another Load Testing Article.
Creating a Very Basic, Static Test Plan
Before testing dynamic ArcGIS services, its best to become familiar with some of the inter-workings and behaviors of JMeter by building and validating the execution of a simple test with a single static request.
Subsequent tutorials will cover how to build more complex tests that exercise a dynamic ArcGIS service.
- Start Apache JMeter
- This automatically creates a new, empty Test Plan (e.g. a test project)
- Right-click on Test Plan and under Add-->Threads (Users), select Thread Group
- This is necessary for defining the test‘s pressure…if it will exert a step load or constant load
- “Thread Group” just includes basic logic for defining a test’s step load (e.g. the test pressure)
- Not every thread group will define a step...some are configured to just apply a constant pressure
- The default values of the Thread Group (Loop Count = 1) will have the test run and execute the added web URL resource only once
- Right-click on the Thread Group and under Add-->Sampler, select HTTP Request
- Copy and Paste a URL for a static resource into the Path field of the HTTP Request sampler.
- For example: https://jmeter.apache.org/images/jmeter.png
Validating the Test Plan
- Use the View Results Tree listener to assist with validating the response in the GUI from the remote server
- Right click on the Test Plan and under Add-->Listener, select View Results Tree
- Apache JMeter provides many other listeners for validating the payload responses within the GUI, but for good spot-check analysis the View Results Tree is highly recommended
- Right click on the Test Plan and under Add-->Listener, select View Results Tree
- Click back on the Test Plan and save your project
- File-->Save
- You will be prompted for a location to save your JMX file (your Test Plan)
- E.g., simpletest1.jmx
- File-->Save
- Visually validating a Test Plan can be easy thanks to the View Results Tree listener.
- Simply click on the green triangle at the top of the Apache JMeter GUI, then select View Results Tree
- This will just send only one request for the image we specified, then stop the test
- This is by design with the default configuration of the “Thread Group”
- A green shield with the words “HTTP Request” should be observed in the Text section
- The “green” denotes a successful request and response
Note: Sending only one request helps prevent the "load testing" of the jmeter.apache.org site, which should be avoided.
- Clicking on the green shield or HTTP Request, followed by Sampler result will show some key information by the remote server
- Load time (response time)
- Size in bytes (Header and Body)
- Error Count
- Response code
- ContentType
- Clicking on the green shield or HTTP Request, followed by Response data then Response Body will show the payload returned by the remote server
- Clicking on the green triangle again will send another, single request for the same image
- The Load time for the requested image will most likely be different
- You’ve just constructed, walkthrough and verified your first Apache JMeter test!
- In other tutorials, we will discuss how to create a Test Plan to point to a service in your ArcGIS deployment as well as executing a test from the command-line for a production quality run.
To download the Apache JMeter Test Plan used in this Article see: simpletest1.zip
For information on how to build an Apache JMeter Test Plan to “dynamically” call the SampleWorldCities ArcGIS service through the export map function, see: Creating a Load Test in Apache JMeter against the SampleWorldCites Map Service (Beginner/Intermediat.... To download the (updated) accompanying JMeter Test Plan, see: sampleworldcities2B.zip.
For a set of additional testing tutorials (of increasing complexity), see: Performance Engineering: Load Testing ArcGIS Enterprise
Apache JMeter released under the Apache License 2.0. Apache, Apache JMeter, JMeter, the Apache feather, and the Apache JMeter logo are trademarks of the Apache Software Foundation. Oracle®, Java, and MySQL are registered trademarks of Oracle and/or its affiliates.