ArcGIS Enterprise User Administration Automation with Apache JMeter (Intermediate)

[AaronLopez]

Administration Automation with Apache JMeter

Apache JMeter is a great load testing tool, but it's a fantastic automation framework too! There are many ArcGIS Enterprise administrative workflows and automation solutions for your portal. This Article focuses on using JMeter to call the ArcGIS REST API in order to carry out user management tasks that would be tedious for large numbers of members. Thankfully, the JMeter's GUI makes the test setup and REST request building easy.

The User Administration Test Plans

• To download the Apache JMeter Test Plan used in this Article see: portal_administration1.zip 

This project includes 6 Test Plans for ArcGIS Enterprise 10.9/10.9.1:

• Add a new user
  • A simple, basic test that just adds new members
• Add a new user with a few options
  • A test that adds new members but allows the Start page and a Portal Group to be specified
• Add a new user with more options
  • A test that adds new members (Start page and Group) but can set Add-on licenses
• Set the security question/answer for new users
  • Set the security question and answer for newly added members that have not logged in
• Disable a user
  • A test that disables users
• Enable a user
  • A test that enables users

The CSV Data Set Config of Users

For convenience, all the Test Plans in the project work off the same list of users from the same file. In the Test Plans, this is referenced by the CSV Data Set Config element named "Users File".

The Text File List of Users

 The included text file contains user information for working with 10 different members. However, it can be adjusted and/or expanded to suite the needs of your organization.

• There are many different options for the role and userLicenseTypeId fields
• These choices can also impact the Add-on licenses as some automatically include specific entitlements

Note: It is recommended to first run the test plans with a small list of users in order to see if everything is configured correctly for your Site.

Administrator Login

With the exception of one, all the included Test Plans have logic to log in as a built-in Portal for ArcGIS administrator at the beginning of the test. For efficiency this action is only executed once (at the start) per each test thread.

Note: When connecting to the Portal for ArcGIS component of ArcGIS Enterprise, the Test Plans will be sending requests directly to the "arcgis" instance on port 7443.

Add a New User (portal_users_add1)

The portal_users_add1 Test Plan is simple way to add new members to Portal for ArcGIS. 

• The administrator credentials are specified from the User Defined Variables section of the Test Plan
  • Except where noted, this step is performed at the beginning of all the included tests

• Once the test authenticates as the administrator, it calls the createUser function and repeats it for each line in file containing the list of users
• Since this test uses a single HTTP Request to create the user it is the fastest and most scalable way to add new members
  • This test only adds users, it does not perform any other duties such as joining a member to a group, setting the Start page, or selecting add-on licenses
• For convenience, the username is appended to all user-based transactions and requests
  • This assists troubleshooting if a particular iteration of the test could not add a specific user
    • All of the included tests follow this design pattern

This test is similar to the process used on the Example: Add members to the portal resource, command line utility and Add members from a file feature built into Portal for ArcGIS.

Add a New User (portal_users_add2)

The portal_users_add2 Test Plan is an easy way to add new members to Portal for ArcGIS but includes a few options. In addition to creating the user, this test allows the administrator to set additional properties like the Start page (also known as the landing page) and a Portal Group.

• This test expands the user creation process by 3 additional requests per user
  • If creating thousands of users, you may notice this test takes longer to complete than portal_users_add1
    • This is due to the fact that more work is taking place

Note: A new member can actually be added to more than one Portal Group on creation. However, for simplicity, portal_users_add2 only adds the user to one group and the same group is used for all members. The group used is defined from the PortalGroupId User Defined Variable. This GUID Id needs to be manually looked up from your Portal for ArcGIS Site. If you do not wish to add the user to a Group, simply disable the setProperties request in the test.

Add a New User (portal_users_add3)

The portal_users_add3 Test Plan is an automated way to add new members to Portal for ArcGIS with the most options for an administrator. This test allows you to set the Start page and Portal Group but adds the ability to specify Add-on licenses like ArcGIS Pro and Extensions and certain User type extensions.

• Immediately after the administrator authentication, the test makes a call to retrieve GUIDs for the ArcGIS Pro and User type extensions
  • These GUIDs will be used later when assigning the licenses to the users

• The add-on licenses add several more requests to the user creation process
  • Although powerful, these additional requests can add time to the overall task as they are performed for each member that is created
• The test is configured to assign the user:
  • ArcGIS Pro Advanced and all available Extensions (as of 10.9/10.9.1)
  • All User type extensions

Note: There are other Add-on licenses such as Applications and ArcGIS Runtime extensions that were not included in the portal_users_add3 Test Plan. Many of these other licenses would require there own specific HTTP request. Again, while this can be convenient and powerful, it can add time to process of adding each user. There are also some licenses like App bundles which were not included in the test as they are automatically included with user license type (e.g. Creator).

Set the Security Question/Answer for New Users (portal_users_update_profile1)

The portal_users_update_profile1 Test Plan is a little unique. It is the only test in the project which does not log in as a Portal for ArcGIS administrator. Instead, it logs in as each user and assumes it is performing the initial login for each member as it will set their security question and answer.

• Presetting the security question and answer is completely optional
• Your organization may instead prefer to have each user set  these values when they first log in

Disable a User (portal_users_disable1)

The portal_users_disable1 Test Plan is an automated way for taking a list of users and disabling their membership to the portal. Once the account is disabled the user cannot log in. This is a less destructive function than delete. 

• Disabling users is fairly straight-forward and performed with one REST call to disableUsers

Note: For simplicity, the disableUsers request in the portal_users_disable1 Test Plan is only disabling one member at a time. However, for each call to the disableUsers function, the request will accept groups of users for improved efficiency. As of 10.9/10.9.1, disableUsers accepts up to 25 users at a time.

Note: The portal_users_disable1 test can be executed over the same users successfully. From the point of view of ArcGIS Enterprise, it is just disabling the member(s) again.

Enable a User (portal_users_enable1)

The portal_users_enable1 Test Plan is an automated way for taking a list of users and enabling their membership to the portal. Once the account is enabled the user cannot log in.

• Enabling users is fairly straight-forward and performed with one REST call to enableUsers

Note: For simplicity, the enableUsers request in the portal_users_enable1 Test Plan is only enabling one member at a time. However, for each call to the enableUsers function, the request will accept groups of users for improved efficiency. As of 10.9/10.9.1, enableUsers accepts up to 25 users at a time. 

Note: The portal_users_enable1 test can be executed over the same users successfully. From the point of view of ArcGIS Enterprise, it is just enabling the member(s) again.

The Thread Group Configuration

Unlike the previous Apache JMeter Article tests that are time-dominant, the Test Plans in this project are iteration based. In other words, when creating or disabling a specific users, we only need to work on the users of interest from the list once.

• The Thread Group "step load" configuration that is included by default with every Apache JMeter installation includes a very convenient Loop Count setting to specify exactly how many iteration the Test Plan should execute
  • The Loop Count setting should match the number of lines in the "Users File" that contain members to be added/disabled/enabled

Note: All Test Plans in the project are configured with the same Thread Group setting. Additionally, all of the included tests are executed with one concurrent test thread.

Test Execution

Also, unlike the previous Apache JMeter Article tests that are executed from the command-line, you can probably get away with running this administrative automation Test Plans right from the GUI. Of course, this depends on how many users you are planning to create, disable, or enable. If you are working with a few hundred, then the GUI would be fine. However, if you plan to create thousands or tens of thousands of users (or more), you will want to run the Test Plans from the command-line for the best usage efficiency of the test workstation resources.

See the runMe.bat script included with the portal_administration1.zip project for an example on how to run a test as recommended by the Apache JMeter team. This script is configured to run portal_users_add3, but can easily be adjusted to running any of the tests.

• The runMe.bat script contains a jmeterbin variable that will need to be set to the appropriate value for your environment

Note: It is always recommended to coordinate the start time with the appropriate personnel of your organization. This ensures minimal impact to users and other colleagues that may also need to use your on-premise ArcGIS Enterprise Site. 

Validating the Test Plans

If the test is being run from the GUI, there are several listeners that have been added to all of the included Test Plans that offer immediate feedback on the status.

• The View Results Tree element offers a convenient way to quickly examine the status of each transaction (e.g. "Create User Account -- portalpublisher2") and its respective requests (e.g. "/arcgis/portaladmin/security/users/createUser--portalpublisher2")
  • Thanks to the Response Assertion rule elements added to each request, the green check mark is trusted indicator of a successful transaction or request

• The View Results in Table element offers a handy way to see status of each transaction and its response time all from one table

Troubleshooting a Command-line Test Execution

As mentioned earlier, when working with large amounts of users, the recommended approach is to run the Test Plans from the command-line. However, administrators will be very interested to understand which users, if any, encountered errors through the automation. It is here that the JMeter Test Report can offer great insight.

• From the Request Summary pie chart on the initial page of the Test Report, you can quickly see if any errors were encountered

•  If errors were encountered from the test run, the Statistics table (bottom of the first report page) can make the failed user requests easily to find when sorting by the FAIL column

Final Thoughts

There are many frameworks, tools and utilities out there to perform administrative task automation for ArcGIS Enterprise. Most likely, they all have their own strengths. Apache JMeter is handy as it provide a graphical interface for building and adjusting the REST requests need to perform the functions. The HTML/JavaScript reports which can be automatically created at the end of a test report are a nice bonus for understanding if whole job was successful or which particular parts failed.

• To download the Apache JMeter Test Plan used in this Article see: portal_administration1.zip 

A Quick Word on Using Multiple Threads

All of the included tests could be configured to use multiple, concurrent threads for faster execution. This is fine from a technical point of view, but all of these test perform write operations to the internal database for the Portal for ArcGIS component. As with any database, such operations can be resource intensive and can only go so fast. Using too many concurrent threads may actually slow down the performance of these tests.

A Quick Word on Deleting Users

The tests included with the project did not include a delete user operation. Deleting a member from the portal is permanent (without backups being available) and such tools that automate this action should use caution. Additionally, some users may have uploaded a plethora of content to the portal. This content would need to be delete or transferred to another user before removing that member.

Apache JMeter released under the Apache License 2.0. Apache, Apache JMeter, JMeter, the Apache feather, and the Apache JMeter logo are trademarks of the Apache Software Foundation.

portal_administration1.zip