nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Urgent ArcServer mapservice issue

BillChappell

Hi,

We have an app that is having issues, and it looks like it's ArcServer but it could be a network issue. And I've scoured the internet for answers but an still stumped. The service draws fast from the endpoint when you tell it to preview or use the JS viewer. However, when querying right from the REST endpoint it's a pig, query all records and it seems to hang and die. Query 200 records and it runs fairly fast but slower then it should. We are using the Esri-Leaflet API and It can be fast locally, but from outside the firewall it's slow and draws in patches before it stops, before it finishes.

Any ideas or clues?

Layer: streams (ID: 1) endpoint

ArcGIS API for JavaScript: WI_PWL2019 built-in JS viewer, from ArcServer Endpoint.

https://pubgis.health.ny.gov/arcgis/rest/services/WI_PWL2019/MapServer/1/query?where=1%3D1&text=&objectIds=&time=&geomet… 200 records.

https://pubgis.health.ny.gov/arcgis/rest/services/WI_PWL2019/MapServer/1/query?where=1%3D1&text=&objectIds=&time=&geomet… All records..

ArcServer 10.6,, MS Server 2016, 16GB ram

Find more posts tagged with

Accepted answers

BillChappell

After a week of frustration, we noticed this worked well in ArcServer 10.3 and not in 10.6.1. We noticed the network packets would increase in size in 10.6 and were larger in 10.6. Moving around after a minute or so would crash the browser as it approached 2 GB. In 10.3, the browser memory did not increase and the packets were smaller. Running 2 servers, same data, save JS code, with the 2 different versions, showed us it was the ArcServer software. To confirm, we swapped software on the servers and it still showed it was the software. Now were hoping 10.7 fixes it.

All comments

BillChappell

Migrated our servers to Arc Server 10.7 and it works great. We now have a long term fix.

JoshuaBixby

I won't be surprised if your issue relates to resultOffset and resultRecordCount parameters and exceededTransferLimit property. The two parameters were introduced in 10.3 for dynamic map services, but the property wasn't introduced until 10.3.1. The JavaScript code will behave different when exceededTransferLimit property exists or doesn't exist, is my guess, so I don't think the code is exactly the same between your two tests.

BillChappell

Against the server. We have a dev, eval, and prod servers, all were tested in 10.6 and 10.3. The 10.6 had the issues on all 3 levels. Servers OS, Processors, and Ram identical, Same project produced data for the service. Same JavaScript code with the same settings in ArcServer's interface. We have been in contact with Esri support, call is under NYS DOH.

RandallWilliams

Hmmm...Are you performing these tests directly against the GIS Server (port 6443), or are you performing this test against the web tier? I ask because it almost sounds like the browser isn't caching the JSAPI like it should be, and pragma/nocache headers are defined at the web tier. Basically, I want to understand if we're actually testing apples to apples, as I haven't seen these issues on the various servers I run in house.

BillChappell

Here is a sample page, I was using to test.

<html>
<head>
<meta charset=utf-8 />
<title>PubGIS Test</title>
<meta name='viewport' content='initial-scale=1,maximum-scale=1,user-scalable=no' />

<link rel="stylesheet" href="https://unpkg.com/leaflet@1.4.0/dist/leaflet.css"
integrity="sha512-puBpdR0798OZvTTbP4A8Ix/l+A4dHDD0DGqYW6RQ+9jxkRFclaxxQb/SJAWZfWAkuyeQUytO7+7N4QKrDh+drA=="
crossorigin=""/>
<script src="https://unpkg.com/leaflet@1.4.0/dist/leaflet.js"
integrity="sha512-QVftwZFqvtRNi0ZyCtsznlKSWOStnDORoefr1enyq5mVL4tmKB3S/EnC3rRJcxCPavG10IcrVGSmPh6Qw5lwrg=="
crossorigin=""></script>

<script src="https://unpkg.com/esri-leaflet@2.2.3/dist/esri-leaflet.js"
integrity="sha512-YZ6b5bXRVwipfqul5krehD9qlbJzc6KOGXYsDjU9HHXW2gK57xmWl2gU6nAegiErAqFXhygKIsWPKbjLPXVb2g=="
crossorigin=""></script>

var str = L.esri.basemapLayer('Streets').addTo(map);

var streams = L.esri.featureLayer({
url:'https://pubgis.health.ny.gov/arcgis/rest/services/WI_PWL2019/MapServer/1',
//url:'https://services8.arcgis.com/k1v79LrnPTwysMSs/ArcGIS/rest/services/WI_PWL2019/FeatureServer/1',
useCors: false,
style: function (feature) {
var c,o = 0.75;
switch (feature.properties.WBCATGRY) {
case 'Impaired':
c = '#FF0000'; //red
break;
default:
c = '#0000FF'; //blue
}
return {color: c, opacity: o, weight: 2};
}
});

streams.bindPopup(function (layer) {
return L.Util.template('<p>PWL ID: {PWL_ID}<br>Name: {NAME}<br>Catgry: {WBCATGRY}</p>', layer.feature.properties);
});

streams.addTo(map);

var usastreams = L.esri.featureLayer({
url: 'https://services.arcgis.com/P3ePLMYs2RVChkJx/arcgis/rest/services/USA_Rivers_and_Streams/FeatureServer/0',
useCors: false,
where:"STATE='NY'",

style: function (feature) {
var o = 0.75;
var c = '#0000FF'; //blue
return {color: c, opacity: o, weight: 2};
}
});

usastreams.bindPopup(function (layer) {
return L.Util.template('<p>Name: {NAME}<br>State: {STATE}</p>', layer.feature.properties);
});

var baseMaps = {
"Streets":str
};

var overlayMaps = {
"pubGIS":streams,
"Esri_NY":usastreams
};
L.control.layers(baseMaps, overlayMaps,{collapsed:false}).addTo(map);
</script>

</body>
</html>

JoshuaBixby

How are you structuring your Leaflet code? I just ginned up a quick example based on Simple DynamicMapLayer | Esri Leaflet but using your service, and the performance isn't much different than the JS Viewer on the REST page.

BillChappell

I'm starting to think it's a combination of four things.

Too much data, I agree with you but the decision makers haven't accepted that yet. Wait till they view it on a phone or old PC. Realistically you can't accurately identify what stream you clicked on until you're zoomed in.
Some network issues, we have dev, eval, and production environments setup, it's 90% working in eval, but it's failing in Production.
I'm also looking at Leaflet w/ Esri-Leaflet plugin, as part of the issue, I made a simple page using the ArcGI JS api, and it flies compared to the Leaflet page. However with the roll out tomorrow, this is only an option for the next version.
ArcServer on eval is 10.3 and production is 10.6, the hardware is the same but we rarely use eval beyond testing, and in this case, it's out preforming the newer version. We're going to install 10.6 on it now to see if it's an issue.

For grins, I exploded the 2363 multi-lines to lines and got 117,289 lines.

BillChappell

I'll look into vector tiles, one issue is I'm using Leaflet instead of ArcGIS JS. I've also been told no scale dependencies, as someone up the food chain wants to see everything on and doesn't care about the issues it causes.

The streams layer is generalized by Leaflet, if you zoom in you can see the lines moving and getting more accurate. I've talked about different services for different zoom levels but was overruled.

JoshuaBixby

The server appears to have been offline for days, but it is up this morning so I took a look.

Regarding this statement:

The service draws fast from the endpoint when you tell it to preview or use the JS viewer.

The reason the JS viewer loads so quickly is that it isn't drawing the service, it is downloading an image snapshot of the service. Open the developer tools in the browser and see what REST calls are being made by the JS viewer code. You will notice the browser is calling Export Map—ArcGIS REST API: Services Directory | ArcGIS for Developers and not Query (Map Service\Layer)—ArcGIS REST API: Services Directory | ArcGIS for Developers.

Regarding this statement:

However, when querying right from the REST endpoint it's a pig, query all records and it seems to hang and die. Query 200 records and it runs fairly fast but slower then it should.

Typically, or should I say nearly all the time, "all records" when querying a service means "all records in batches of 1,000." GIS services have a default maximum record count: Set map service properties—Documentation | ArcGIS Enterprise )

Maximum number of records returned by the server: Clients, such as the ArcGIS Web APIs, can perform query operations to return specific information, or records, from a map service. This property specifies how many records can be returned by the server to a client for any given query operation. Specifying a large number of records to be returned by the server can slow the performance of client applications consuming your map service, such as web browsers, and your GIS server.

Looking at the streams layer properties, MaxRecordCount has been set to 2400. Not only does that tell me the default has been overridden, but the value of 2400 gives the impression the intent behind changing the default is to disable this functionality since the service has 2363 records.

I cannot emphasize enough, disregarding/overriding MaxRecordCount is almost never the answer to whatever issue you think doing so addresses.

Setting the MaxRecordCount matter aside, querying "all" records with returnGeometry false results in all attributes for 2363 records being returned quite quickly, which tells me the issue is related to geometries of the features.

Poking around some of the geometries, I see several thousand vertices are quite common for features. There is no hard, fast definition of "dense" geometry, but 3,000 to 10,000 vertices starts to get into that realm. When working with thousands of vertices per feature, projection-on-the-fly can be a performance killer for GIS services, especially when MaxRecordCount has been overridden. Also, serializing geometries takes times when there are thousands of vertices. Thirdly, returning geometries with thousands of vertices in JSON format isn't space efficient. If you return the first 100 features with geometries, the JSON file is almost 35 MB. If that scales linearly, returning all features to a web client will involve sending 800+ MB over the wire. Sending 800+ MB over a wide area network (WAN) is never fast.

RandallWilliams

There's a few ways I'd address this. These suggestions are not mutually exclusive.

First, vector tiles.

VectorTileLayer | API Reference | ArcGIS API for JavaScript 3.27

If that's not going to work, I'd cache these services so you don't have to query the server with every pan and zoom and pull down all that geometry.

Failing that, I'd consider adding two or three scale dependent layers representing the rivers.

At small scales, the river features can be generalized. As you zoom into larger scales, they can be generalized less. There's no reason to render complex river geometry at large scales - they won't be able to see that anyway.

BillChappell

Ugh, it looks like the networking guys are now having issues on the site. Basically, it's every stream in New York from the USGS topo's So it's multi polylines merged down to 2300 records. It was published from a FileGeoDatabase. During the service creation, analyze shows it drawn in .77 seconds, and from the REST endpoint viewer it takes about a second to draw. But try to consume the service in ArcGIS JS API, Leaflet, ArcMap or ArcPro and it seems to hang and stop. I think it's just too much data but key decision makers want it all on at all scales.

ThomasColson

Need more details. What type of geometry, is it coming from SQL, if so, what flavor? Or, is in the hosted data store? Either way, I'm suspecting attribute or spatial index as the problem here. What does Fiddler - Free Web Debugging Proxy - Telerik show you regarding time between each request/response? Run fiddler on your inside, and outside firewall requests.

BillChappell

2300 records

mvolz47

When you query all records, how many records are we talking about?

Join the Conversation

Signed in members can post, follow updates, and more. New here? Register a free account.

Quick Links

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.