In a previous blog post, my esteemed colleague and board game nemesis Kelly Gerrow-Wilcox discussed the basics of capturing web traffic in a web browser using the built in browser developer tools. But what about when you’re consuming services in ArcGIS Pro, ArcMap or any other non-browser client? Enter: web traffic capturing tools. There are numerous free tools (such as Fiddler, Wireshark, Charles and others) which allow users to capture web traffic from their computers. This blog will focus on capturing HTTP/HTTPS traffic using Fiddler. I've chosen Fiddler because of its relatively simple interface and broad adoption within Esri Technical Support.
Basics: download, configuration and layout
Fiddler can be downloaded here. After installation, the only critical configuration that needs to occur is to enable it to capture traffic over HTTPS.
With Fiddler open go to Tools > Options
In the pane that opens, check Capture HTTPS CONNTECTs and Decrypt HTTPS traffic. (This allows you to capture any requests sent using HTTPS, which is slowly but inevitably replacing HTTP as the protocol for transferring data across the web).
If it’s necessary to capture network traffic from a mobile device, some extra configuration is required. Both the mobile device and the machine where Fiddler is installed will need to be using the same wifi network. The following documents outline the steps to capture mobile traffic:
The Fiddler application itself is split into two main sections; the Web Sessions list and the…other pane (I couldn’t find an official name so for the purposes of this blog we’ll refer to it as the Details pane).
The Web Sessions pane includes a sequential list of every request sent by the client to a web server. Important information contained here includes:
The type of protocol used (HTTP or HTTPS)
Which server the request was sent to, and the full URL
Note: the columns in the Web Sessions pane can be custom configured by right clicking anywhere in the headers and selecting Customize Columns. Some useful fields to turn on can be:
Overall Elapsed (available under Collection: Session Timers. This is the overall time it takes for the request to be sent and returned)
ClientBeginRequest (available under Collection: Session Timers. This is the time your software first began sending the request, using your computer’s time)
X -HostIP (Select Collection: Session Flags and manually enter in the Header Name. This is the IP address of the server destination of the request)
If you click a single web session, that triggers the Details pane to populate with a wide variety of information for that specific request.
Intermediate: What do these details mean? Do they mean things? Let's find out!
There are numerous tabs in the Details pane. The most useful (for our purposes) are Timeline, Statistics, and Inspectors. The others are all advanced functionality outside of the scope of this blog.
The Statistics and Timeline tabs are both helpful when investigating any performance related issue, for example if a service is taking a long time to load in the Map Viewer. The Timeline tab is useful for identifying which request in a multi-request process is acting as a bottleneck. To utilize the Timeline tab, select multiple requests in the Web Sessions list. The timeline will display the requests in a sequential “cascade” format. Any requests taking an unusually long time will clearly stand out with a significantly longer bar in the timeline.
Statistics displays the exact times every step of the request took, from the client initially making a connection to the last step of the client receiving the response. This breakdown is useful to potentially identifying which step in the process of a single request is acting as a bottleneck. For example, if every step is taking a fraction of a second, but there is a multi-second pause between ServerGotRequest and ServerBeginResponse that would indicate that something on the server side is causing a slowdown.
Lastly, the Inspectors tab is the where the bulk of information is displayed and likely where the vast majority of any troubleshooting will be done. Here is where the curtain is drawn back to reveal the nitty gritty of how applications interact with web services. Inspectors is further divided into two main sections; the Request information (everything related to the request sent by the client) and the Response information (everything related to the response returned by the server). Both divisions have a nearly identical set of subdivisions which display the content of the request/response in different formats. Below are the useful tabs for our purposes:
Headers – A list of additional information that is not part of the main request. This may include information like security/authentication information, the data format of the request or response, the type of client making the request, etc. This is a good place to find an ArcGIS Online token, when relevant.
WebForms (request specific) – Depending on the type of request, this will display a breakdown of each request parameter and the value of that parameter. For example, when submitting a search query this section will display the parameters of the query (like keywords, date ranges, etc).
ImageView (response specific) – If the request is for an image, the ImageView will display the image which is returned. Obviously, this is particularly useful for requests involving tiled services.
Raw – This will display the entire request or response in text format.
JSON – If the request/response includes content in JSON format, this tab displays the content in a more human readable format. This is particularly useful for requests/responses to the REST api of ArcGIS Enterprise servers.
XML – If the request/response includes content in XML format, this tab displays the content in a more human readable format. This is particularly useful for requests/responses to OGC services.
Advanced: That’s great Alan. But what am I supposed to actually do with this information?
How you use network traffic information is going to depend on what you’re trying to learn or solve. Checking network traffic can help identify the where and what of a problem but cannot tell you the solution. This is where your knowledge of your app, your web services and if all else fails, some good old fashioned web searching come into play. Here are a few common examples of ways to isolate the problem you’re facing:
Check the HTTP/HTTPS response code in the Web Sessions pane. Anything that isn’t 200 should be investigated (it might not necessarily be a problem, but it’s worth looking at). Again, here’s a description of what these mean. Even a 200 response could contain error messages or other useful information.
A 304 response from a server will trigger the client (web browser, ArcMap, etc) to use the client’s cache and Fiddler is therefore not actually capturing a complete response from the server. If there is a 304 response on a critically important request, try again either in Incognito mode or clear your client’s cache.
A 401 or 403 response typically mean the server requires some sort of authentication. This would help, for example, identify an unshared feature service in a web map which is shared publicly.
A 504 response typically means something timed out. Use this in conjunction with the Timeline, Statistics and Overall Elapsed column mentioned above to troubleshoot performance issues.
If you can’t find the problematic request, open the Raw, JSON or XML tabs of the response and just scroll through the requests looking for one that returns an error.
Raw, JSON and XML contain the exact same information, just formatted differently.
When errors occur, the error listed in the response may be more detailed than the error provided in the user interface of whichever application was being utilized.
Find a way to ignore irrelevant requests!!
One of the most challenging factors in troubleshooting network traffic is the volume of requests that are sent/and received for even minor actions. Below are strategies to help avoid cluttering your log with unnecessary requests.
Turn off Capture (File > uncheck Capture Traffic) when you know Fiddler’s not capturing relevant information.
Close any browser windows or background processes that don’t need to be running.
If Fiddler is capturing traffic you know is not related to what you’re investigating, Filter it out of the Web Sessions by right clicking a session > Filter > select what session parameter you want to filter.
If you’ve captured a number of requests that you know you don’t need, select and delete them.
Target Fiddler to only capture requests from a single application by clicking the ‘Any Process’ button (next to the small bullseye icon), holding and then releasing your mouse over the application you want to capture from. This would be useful, for example, to capture all traffic coming from ArcMap while ignoring everything that occurs with your browsers.
Once you have isolated the request(s) relevant to the issue you’re investigating, the following tips can help determine what the actual problem is.
If you can isolate the problematic request, consider what is the nature of that request in order to help determine any next steps.
Are ALL requests to the service failing? Perhaps the entire server is down or inaccessible.
It’s possible to resend any requests by right clicking a Web Session > Replay > Reissue and Edit.
This is especially helpful for isolating a specific header or request parameter that might be problematic. Modify the information under WebForms or Headers to see if that fixes the problem you’re encountering or reproduces the problem you’re investigating.
If you have a request that’s succeeding and one that’s failing, copy the headers or WebForms parameters one at a time from the request that’s working to the request that’s failing. Once the request works, you’ve successfully isolated the parameter/header in the requests that’s causing the problem.
It’s possible to send repeated requests by right clicking a Web Session > Replay > Reissue Sequentially.
This is helpful for capturing issues which might be intermittent. Send the request 20 or 30 times automatically and see if hit the issue you’re looking for.
Web service query requests can be viewed in the browser with a user-friendly interface. This allows you to easily tweak and resend requests. To view a query request in the browser:
Right click the query session
Copy > Copy the URL
Paste in a browser window
Change the section in the URL “…f=json…” to “…f=html…”
Click enter to browse to the page
Fiddler and other network capture software are not silver bullets to solve all web traffic related GIS issues, but they are useful tools to help. With a bit of practice, utilizing this type of software can help resolve a wide variety of issues when accessing web services in GIS applications.
Got any good Fiddler (or general network traffic logging) tips? Feel free to leave them in the comments!
Hi I created Apps for different sites situated in different locations using Portal for Arcgis .All the Apps seem to be working just fine apart from 1 site were I m getting complaints that some published tile layers are not visible even when turned on in layers tool.
What could be causing this problem? I asked that the proxy parameters be added to the LAN settings but I m told the problem still persists.
We have attempted several times to configure a server on a virtual machine running the following setup. The server was previously running the same DBMS and OS with ArcGIS Server 10.4 successfully; however, we wanted to upgrade and include Portal. Unfortunately, we have had problems with the upgrade.
Microsoft Windows Server 2012 R2 Ver. 6.3.9600
IIS Ver. 6.2.9200
Microsoft SQL Server 2016
ArcGIS Server Enterprise 10.5.1
Web adaptors for Portal and Server
In previous attempts using the Enterprise Builder app, I have opened SSL ports as instructed for 6080, 6443, 7443, and 2443; however, I have since removed bindings in trying to get back to where we were with a simple install of ArcGIS Server.
Currently, IIS has been configured with bindings for ports 80, 6080 and 6443.
For the moment, Windows Firewall is turned off; however, I have set inbound Rules to allow access to the following local ports:
ArcGIS Enterprise Setup
80, 6080, 7080, 443, 2443, 6443, 7443, 9876
1098, 4000-4004, 606, 6080, 6099, 6443
2443, 9876, 29080
I have tried installing using the automated Enterprise Builder application and manually by using the ArcGIS Server for Enterprise installation ISO distribution. In both cases, the software appears to install and configure correctly, but attempts to access the arcgis/manager (or any other related links) through a browser returns 404 errors. IIS is running as indicated by other web services.
In some attempts, it has appeared that the ArcGIS server software is running correctly, because I can see the map REST services in the system services list. Nevertheless, we are unable to access them with the same 404 error results.
I have many years of experience with ESRI products but am not much of a web programmer, so these setup problems are confounding me. There is probably a simple issue which you may have seen previously that is blocking access to the system framework; however, despite many web searches and after many unsuccessful (and some partially successful) attempts, I am at a loss as to what the problem is.
If you need more information about setup, I will be happy to provide it. I appreciate your assistance on this.
Exciting things are coming for GIS users in 2018 - ArcGIS Desktop 10.6 and ArcGIS Pro 2.1 are being released in January, along with some great new tools, features and functionality. Esri Technical Support is excited to work with our customers in these new environments, but what does this mean for some of the older platform versions?
Alas, on January 1, 2018, ArcGIS 10.1 is officially retired!
Here's a quick list of what this entails:
Technical Support will not be available for ArcGIS Desktop 10.1, ArcGIS Server 10.1, or Enterprise GDBs at 10.1
It will not be possible to request a Support Case for ArcGIS Desktop/Server/Enterprise 10.1
We are here to help you with upgrading. If you need technical support for the upgrade process, give us a call.
As this year comes to a close, it's important to plan ahead. Get a jump start on upgrading your applications and geodatabases, and reach out to us if you have questions.
For more information, check out some of our documentation:
This blog post provides the latest updates regarding deprecated features in the recent release of ArcGIS 10.5.1.
With each release, Esri assesses and adjusts the products and functionality supported in the ArcGIS Platform based on customer needs and technological trends. The purpose of the Deprecated Features for ArcGIS document is to provide as much advanced notice as possible regarding these changes.
Here are some of the major changes in ArcGIS 10.5.1:
ArcGIS 10.5.1 is the last release to support Visual Studio 2013 for the ArcObjects SDK.
In the near future, the cluster functionality in the ArcGIS Server component of ArcGIS Enterprise will be deprecated. Instead, it is recommended to create separate ArcGIS Server sites where multiple clusters would have been used previously.
ArcGIS Enterprise 10.5.1 will stop bundling the portalpy module in favor of the ArcGIS API for Python. No further development is planned for this module.
ArcGIS 10.5.1 will be the last release to support the PostgreSQL 9.3.x series of releases, DB2 versions 9.7 and 10.1, and the ST_Raster data type for Oracle, SQL Server, and PostgreSQL.
ArcGIS 10.5.1 is the last release to support anything other than the Data Store product as a data store for a Hosting Server.
Note: The deprecation of cluster functionality does not affect the ability to create multi-machine sites. ArcGIS Server sites with multiple machines continue to be fully supported.Gregory L. - Online Support Resources
Nobody likes to talk about it, but sometimes computers can crash. Yup, the entire thing just fails and nothing at all can be recovered (if you haven’t backed up your data, go do it now!) Or what if your laptop is stolen, or you flipped your kayak and your machine sank to the bottom of Lake Superior? You just don’t have it anymore and there is absolutely nothing you can do to get it back. When these types of things happen, any Esri licenses that were authorized on the machine may be lost, too.
In the past, an authorized maintenance contact had to call Esri Technical Support to submit a license appeal and recover the lost licenses. Now, this functionality is built in to My Esri, empowering your organization with self-service functionality and enabling you to get back up and running quickly.
I wanted to make sure that our customers are aware of this great new functionality and walk through how you’d go about getting your licenses back in the event of a catastrophic failure or loss as described above – though I really hope that never happens.
To perform the following steps, you will either need “Esri Admin” permission or the “Take Licensing Actions” permission. Sensitive information such as machine IDs, license numbers, and other personal information have been replaced with asterisks in the following screenshots.
First, log in to My Esri and click the My Organizations tab.
Please note that I’m demonstrating the steps in a QA environment and that your experience won’t include the green QA…
Click the Licensing tab.
This will bring up the Licensing Overview page and if you have the correct permissions, you should see the Recover Lost Licenses option both in the Licensing panel as well as a card.
Next, click Recover Lost Licenses.
The Recover Lost Licenses screen explains that this is a process to retrieve licenses from a machine that is no longer accessible due to system failure, system loss, or destruction. The License Recovery process requires the signature of the organization’s License Administrator in a Certificate of Destruction. This process is irreversible and should only be used as the absolute last option when all other solutions to rectify the problem have failed.
An example of when you would not use the Recover Lost Licenses option is if you can still access the machine and deauthorize the licenses normally. The instructions provided describe how to perform standard license deauthorization:
Once you’ve determined that it really isn’t feasible to scuba dive to the bottom of Lake Superior to recover your machine (and hence, its licenses), follow the steps outlined below to complete the recovery.
Step 1: Find Your Machine
To proceed with license recovery, select how you would like to find the machine. There is an option to search by products on the machine or use the machine’s UMN IDs if you know those.
Option A: Search for machine by product
Search for the machine by populating the dropdown boxes.
We see that the search for ArcGIS Desktop Advanced Concurrent Use licenses for this organization returns five machines.
Selecting the machine from which the licenses need to be recovered will take you to Step 3.
Option B: Select Machine using the UMN
Enter the UMN for the machine and click Search. Since the UMN by definition is associated with a single machine, you should get only one result in this case, as opposed to searching for a machine by product.
Click Select to take you to Step 3.
Step 3: Review Selected Machine
This step will show you a list of products our records show were activated for the selected machine.
After reviewing the selected machine, you have the option to go back if this is not the correct machine or proceed with the license recovery process.
Step 4: Accept Terms and Conditions
Review and agree to the terms and conditions, and click Next.
Step 5: Summary to process License Return
This step gives you another opportunity to fully review the selected licenses to return. If the selection is correct, click the “Process Return” button near the bottom of the page.
You’ll receive a confirmation screen showing the status of each license return.
And that’s it. You are now able to authorize these licenses on a new, dry machine!
In the event that not all licenses are returned successfully, you will be presented with a summary of which licenses were returned and which were not. These should be exceptions; not the norm. In these cases, please work with Esri Customer Service or your local distributor to finalize the recovery process.Kory K. - Customer Advocacy Lead
If you want to know how much traffic your ArcGIS Server site can handle, you'll have to thoroughly test. And because you're probably not going to convince thousands of people to log on to your apps and services all at once, you need something like JMeter to perform load testing.
My first introduction to performance testing, and really just the JMeter in general, was with this awesome blog post from Randall Williams. I eventually started to use JMeter widely and also suggested users to run a sanity check and load test their GIS environments before moving to production.
In the enterprise world, a domain-based approach is widely used for secure authentication and authorization, where credentials of currently logged in Windows users are seamlessly passed to web applications, allowing single-sign-on. JMeter lets you emulate requests being sent from a real user by constructing relevant headers and passing them along with the request. This post focuses on configuring JMeter to perform load testing when the services are secured with Integrated Windows Authentication (IWA).
Once you have modified ArcGIS Server for Windows Authentication, you can forge ahead with adjusting JMeter to handle the authentication challenge. Here are the steps:
1. Navigate to the JMeter bin directory (apache-jmeter-2.7/bin).
2. Open the file jmeter.properties in text editor and set to the following:
3. Save and close the file.
4. Open the file httpclient.parameters and set to the following:
5. Save and close the file.
6. You must use the HTTP Authorization Manager configuration element to construct a relevant authentication header. The Authorization Manager lets you specify one or more user logins for web pages that are restricted using server authentication.
7. Complete the HTTP Authorization Manager as follows:
Username: “User logon name” for Windows domainPassword: Windows domain passwordDomain: [DOMAIN] Other fields like “Base URL” and “Mechanism” can be left as it is.
To accurately simulate the users, you can setup each thread to login with different credentials by placing an HTTP Authorization Manager configuration element in each thread group element.Tip: You can add a 'View Results Tree' listener to debug your test plan. Thus, you can review the request and response data to ensure that your test plan works well.
Below is a sample request with the Authorization Manager disabled:
Below is a properly configured HTTP Authorization Manager:
Here you can see JMeter sending authentication information in an Authorization header: NTLM.
Because the way Microsoft NTLM (also known as Windows Challenge/Response) and IWA work, the first few requests return a 401 response as part of the NTLM handshake scheme. This means that for the first request, you might encounter an unusually high response time.
This should help you understand how your services from ArcGIS Server would perform with Windows Authentication security. You may have a more complex situation if more Active Directory domains are involved (for example, domain trusts, forest trusts, complex nested groups, and so forth), or if there is a performance issue with your domain controller. Head over to your Active Directory Administrator for more information. If there's a performance bottleneck that cannot be eased, you may want to use the other type of security scheme.
Happy load testing!Divyam Gulati - Server Support Analyst
02/06/2017 Update: Patches have been released for Portal for ArcGIS version 10.3 and 10.4, which can be foundhere. We are continuing to work on patches for ArcGIS Server Manager (10.2.x, 10.3, and 10.4) and Portal for ArcGIS (10.2.x). We appreciate your patience as these patches are being developed.11/23/2016 Update: Patches have been released for ArcGIS Server version 10.3.1 and 10.4.1, which can be found here. We are continuing to work on patches for ArcGIS Server Manager (10.2.x, 10.3, and 10.4) and Portal for ArcGIS (10.2.x, 10.3, and 10.4). We appreciate your patience as these patches are being developed.11/10/2016 Update: Patches have been released for Portal for ArcGIS versions 10.3.1 and 10.4.1, which can be found here. We are continuing to develop patches for ArcGIS Server Manager (10.2.x, 10.3.x, and 10.4.x) and Portal for ArcGIS at version 10.2.x, 10.3, and 10.4.
The forthcoming updates to the Firefox, Chrome, and Safari browsers cause errors in some web applications for ArcGIS Server and Portal for ArcGIS when users upload content.The following browser versions are affected:
Firefox 49 (Release Date: September 20, 2016)
Chrome 54 (Release Date: October 18, 2016)
Safari 10 (Release Date: September 20, 2016)
Note: Microsoft Internet Explorer and Microsoft Edge are not affected.The following ArcGIS products are affected:
ArcGIS Server 10.2.x, 10.3.x, and 10.4.x (patch deployed on 11/14/2016 for ArcGIS Server 10.3.1 and 10.4.1)
Portal for ArcGIS 10.2.x, 10.3, and 10.4 (patch deployed on 12/16/2016 for Portal for ArcGIS 10.3, 10.3.1, 10.4 and 10.4.1)
The following workflows are affected:Portal for ArcGIS Home Application
Updating details for any content, such as items within “My Content” or fields within a user profile
Creating new groups via "Groups" > "Create A Group"
Uploading content via "My Content" > "Add Item" > "From my computer"
Uploading content to the Portal for ArcGIS Map Viewer via "Add" > "Add Layer from File"
Adding attachments when editing features in the Portal for ArcGIS Map Viewer
Updating thumbnails on any Item Details page or via "My Organization" > "Edit Settings" > "General"
Updating the banner or background images via "My Organization" > "Edit Settings" > "Home Page"
Below is an example of the error encountered when attempting to upload content in the Portal for ArcGIS home application.
This example shows the error which can be seen when trying to upload content to Portal for ArcGIS
ArcGIS Server Manager
Uploading service definitions (SD) when publishing services
Uploading SDE database connection files when registering a database as a data store
Uploading SOE or SOI files when adding extensions
Uploading KMZ files
Below is an example of the error encountered when attempting to upload an SD in ArcGIS Server Manager.
This example shows the error seen when uploading content to ArcGIS Server Manager
To resolve these errors, Esri is developing software patches for ArcGIS Server and Portal for ArcGIS. We will keep you informed as the patches become available.Thomas E. - User Advocacy
At Esri, we are always looking to provide you with cutting-edge capabilities that help turn GIS ideas into a powerful reality. One of our newer features is the ability to create, serve, and consume vector tiles, and it is exciting to see users taking advantage of this technology. We would like to take this opportunity to address some of the system requirements needed to implement these services.
As per our Vector tile services documentation, in order to share vector tile layers within your own infrastructure, you will need ArcGIS 10.4 for Server or higher - including the full Web GIS stack. This is comprised of Portal for ArcGIS, a hosting server using ArcGIS Server, and ArcGIS Data Store. To create the vector tiles, you will use ArcGIS Pro 1.2 or higher.
You might ask: Why are ArcGIS Pro and Portal for ArcGIS required to accomplish this task?
ArcGIS Pro offers a robust system of authoring capabilities, which allows content creators to build beautiful maps; and it has the tools to store these maps as vector tile packages for sharing with portals. These vector tile packages are then uploaded and shared through your portal, whether that is ArcGIS Online or Portal for ArcGIS.
Vector tile layers can be restyled without needing to rebuild the vector tile package.
Vector tile layers have unique capabilities. For example, they can be easily restyled with a custom look and feel, which happens dynamically without needing to rebuild the vector tile package. This is possible because Portal for ArcGIS takes advantage of the ArcGIS geoinformation model, which allows you to use, create, and share geographic information throughout your organization, the community, and openly on the web. For ArcGIS Server users who are used to sharing map layers with a portal by registering the REST endpoints as an item, or by creating custom applications directly against those REST endpoints, the geoinformation model provides a more flexible framework whereby geographic information is arranged into map and scene layers. These can be combined to build maps and scenes that can also be used in apps, in analysis, shared with groups, and so on.
The ArcGIS geoinformation model is a framework that lets users easily build informative and compelling maps by pulling different layers together.
To recap, create vector tile layers and vector tile packages using ArcGIS Pro. Share the vector tile packages with Portal for ArcGIS. The portal will automatically unpack the package so that other people can use those vector tile layers in their maps and apps.Thomas E. - Server Advocacy Lead
This blog post provides the latest updates regarding deprecated features in ArcGIS 10.4 and in the recent release of ArcGIS 10.4.1.
With each release, the platforms and functionality supported in the ArcGIS platform are assessed and adjusted based on customer needs and technology trends. The purpose of the Deprecated Features for ArcGIS document is to provide as much advanced notice as possible regarding these changes.