topic Re: Slow performance administering services in ArcCatalog and ArcGIS Server Manager 10.1 in ArcGIS Enterprise Questions

Slow performance administering services in ArcCatalog and ArcGIS Server Manager 10.1

BrettElliot — Thu, 06 Sep 2012 21:54:37 GMT

Hi everyone,

Recently we have been having issues with access the map services in ArcCatalog. After trying to recreate the error on a test machine, we narrowed down part of it to having the Configuration Store set to a network drive as opposed to C:/.

Originally, we had set the configuration store to a network drive because "the location must be accessible to all machines that are registered to the site".

We shared the C:\arcgisserver\config-store across the network so that all machines could access it instead.

Since changing this setting, the response time is much much faster than it was before. It used to take ~30 seconds to pull up the 'right click' menu in ArcCatalog, or view the services directory in ArcGIS Server Manager.

If anyone else is having similar issues, I hope this can help.

Any insight to why this would cause everything to slow down would also be appreciated.

Thanks,

Re: Slow performance administering services in ArcCatalog and ArcGIS Server Manager 1

JustinGreco — Fri, 14 Sep 2012 01:05:31 GMT

Same thing was happening for me, I had switched it from local to a network share in order to add additional servers to the site. I experienced the same issues in Catalog and manager. Switching it back to local sped everything back up like you had said.

Re: Slow performance administering services in ArcCatalog and ArcGIS Server Manager 1

BrettElliot — Fri, 14 Sep 2012 14:31:20 GMT

Good to hear its not just us.

We had it as a UNC path so our caching machine could see it. We are now just trying to figure out if it's a network problem causing this slowdown, or if it is an ArcServer problem.

Cheers,

Brett

Re: Slow performance administering services in ArcCatalog and ArcGIS Server Manager 1

JustinGreco — Fri, 14 Sep 2012 15:13:56 GMT

I just got off the phone with tech support and they did not know what is causing the issue. I am thinking it could be something with the network.

Re: Slow performance administering services in ArcCatalog and ArcGIS Server Manager 1

karldailey — Mon, 17 Sep 2012 17:55:01 GMT

i have found the interface in all forms to be extremely slow, and although I only just begun looking at 10.1, I really dislike it. it takes several minutes just to publish one map. I like to put my maps on one drive and cache filed on another drive. That helped alleviate disk IO bottlenecks, but now it seems to force them onto the same disk. has anyone noticed this? Overall, I will not be upgrading from 10 to 10.1.

Re: Slow performance administering services in ArcCatalog and ArcGIS Server Manager 1

JustinGreco — Wed, 19 Sep 2012 00:48:24 GMT

You can put your cache on a separate drive. Just got to the Site tab, the Directories, and you can edit the cache directory or add another cache directory.

The only slow experience I have had with 10.1 is with what I described below, setting the config-store to a network share.

Re: Slow performance administering services in ArcCatalog and ArcGIS Server Manager 1

RoyceSimpson — Thu, 27 Sep 2012 22:15:26 GMT

In my case the slow performance was Windows Server 2008 related. I upgraded it to "R2" and performance is great now.

Re: Slow performance administering services in ArcCatalog and ArcGIS Server Manager 1

BrettElliot — Thu, 27 Sep 2012 22:19:57 GMT

Good to know Royce,

We are currently working with ESRI Tech support to find out a solution. There is a possibility it is our server / network set-up. I'll make sure to update this post when we find out what is happening.

Re: Slow performance administering services in ArcCatalog and ArcGIS Server Manager 1

JacobRaymond — Mon, 01 Oct 2012 19:37:21 GMT

Just to follow up this is apparently a known issue that will be addressed in SP1 (Bug# NIM085259). It turns out that this is an issue that gets noticeably worse with the more services you have. We had Around 100 services running so it was getting quite unbearable but we will store the config_store folder on the C:\ drive until this issue is resolved.

Re: Slow performance administering services in ArcCatalog and ArcGIS Server Manager 1

JacobRaymond — Wed, 24 Oct 2012 17:35:52 GMT

I have been experiencing the same issue. However it does not look like it will be resolved in Service Pack 1. I looked at the list of things that will be resolved and did not find any mention of Bug# NIM 085259.

Ref: http://downloads.esri.com/support/downloads/other_/189810.1_SP1_Announcement.pdf

To me, if we cannot administer the site properly we cannot move to 10.1. In fact I am concerned how everyone else is accomplishing this with the problems being faced in administering the site when multiple machines are added to a cluster with a shared drive (i.e. the suggested method in the help documents).

Thanks

Neeraj

This is a copy of the bug report that was sent to me by ESRI Canada tech support. It claims that it is fixed at SP1. I guess we will see, if SP1 ever gets released.

---

Synopsis
The time required to get permissions for a service increases as the number of services in the ArcGIS Server site increases on a shared config store.
Problem Description
The following has been tested with the default permissions on the services in the site i.e esriEveryone has been given access to all the services in the site except the System folder.

services//permissions returns in under a second with about 20 services spread in multiple folders.

As you increase the number of services, accessing the permissions resource on a service increases as well. With about 140 services, the accessing of the permissions takes 6 to 7 seconds.

This issue is reproducible only when the config store is on a UNC. If it is a local path, it does not happen.

Status Resolved

SP Fixed 10.1SP1

Re: Slow performance administering services in ArcCatalog and ArcGIS Server Manager 1

StevenMenefee1 — Fri, 02 Nov 2012 15:03:14 GMT

I've also noticed that trying to access the 10.1 services via catalog or the new Server Manager that it is incredibly slow. However, we are going to have a multiple machine setup, which means the config-store must be accessible to our other GIS Server machines...so I don't think making it a local path is an option for us is it??

Re: Slow performance administering services in ArcCatalog and ArcGIS Server Manager 1

RoyceSimpson — Tue, 06 Nov 2012 13:56:42 GMT

I've been struggling with a multi-machine site for months... even with SP1 if I reduce the machines to 1, the services, both in Catalog and web REST endpoints retrieval are very fast. As soon as I add 2+ machines to that site (all VM's of the same specs as machine 1) the performance of the services drops dramatically, to the point of being unusable in any way. I'm attempting to configure a site that has 2 machines in one cluster for exposing map services to the public and one cluster with 5+ machines specifically for caching purposes.

From what I can tell, one of the big selling points of AGS 10.1 is scalability, flexibility. I'd like to be able to pragmatically be able to move machines from one cluster to the other based on load, caching regimens, etc... Can't do that if I have multiple sites.

Anyway, I sure hope 10.1 can iron this out. If not, it seems that 10.1 is the new Windows Vista. Please help prove that prediction wrong esri.

Re: Slow performance administering services in ArcCatalog and ArcGIS Server Manager 1

JacobRaymond — Tue, 06 Nov 2012 19:24:49 GMT

So we upgraded to SP1 and it made things worse. It did speed up access via the ArcGIS Server Manager web interface, but it made accessing the service listing in ArcCatalog unbearably slow, even if the config-store is on a local drive. I would not recommend upgrading to SP1 as this issue is not resolved, in fact it is much worse than before (at least for us it is).

Re: Slow performance administering services in ArcCatalog and ArcGIS Server Manager 1

MuryadiOey — Wed, 12 Dec 2012 15:48:47 GMT

I have similar issue about this slow performance. From my testing, the more I added servers into a cluster, the slower the server performance. I think ESRI need to fix the background process. Here is one of my test, if the cluster only one server, it took at least 15 sec to publish SD from Server Manager and it tooks at least 30 sec from ArcCatalog. Once I added another server (total 2 servers), the time was double or more. To publish SD from Server Manager, it took at least 30 sec and from ArcCatalog, it took at least 180 sec. When I added another server (total 3 servers), i took at least 60 sec to publish from Server Manager and more than 5 minutes from ArcCatalog.
I would like to see if ESRI can fix this issue as soon as possible by hot fix rather than waiting for SP2.

Thank you

Re: Slow performance administering services in ArcCatalog and ArcGIS Server Manager 1

MuryadiOey — Wed, 12 Dec 2012 20:05:54 GMT

Actually our issue was resolved. It was a firewall issue and we fixed that.

Neeraj

Can you share a little bit about causing by firewall?

Thanks!

Re: Slow performance administering services in ArcCatalog and ArcGIS Server Manager 1

VernonWoods — Thu, 13 Dec 2012 14:33:36 GMT

We are also experiencing slowness when trying to right click on AGS services in ArcCatalog 10.1 via an administrative data connection.
We already have AGS SP1 installed which solved other problems but doesn't seem to help with this.
We tried moving our config store back (from a network drive) to a local drive of an AGS server (still referencing via a UNC since we have multiple servers in the site) but this didn't appear to help the slowness.

Our OS is Microsoft Windows Server 2008 R2 Enterprise, 6.1.7601 Service Pack 1 Build 7601

We have no firewall between our Desktop sessions and the AGS servers.

On point to note is that a user-connection via ArcCatalog 10.0 does not produce the slowness. Unfortunately an admin-connection is not possible from ArcCatalog 10.0

Re: Slow performance administering services in ArcCatalog and ArcGIS Server Manager 1

BrandonCales — Mon, 28 Jan 2013 14:32:25 GMT

Anyone have any luck with this. My connection is really slow too. I don't want to mess with the firewall settings if I don't have to and am hoping there is another resolution. Is there any testing to figure out where the bottleneck is?

Re: Slow performance administering services in ArcCatalog and ArcGIS Server Manager 1

GeraldLee__GISP — Mon, 28 Jan 2013 16:07:59 GMT

just my 2 cents worth here. Dont know if this helps or not but my connection speed was absolutley unbearable. To the point that i uninstalled arcgis and reinstalled using local paths only. The speeds impoved to almost immediate connectivity. i tried every which way to use UNC server paths so i could use a multi site setup but my productivity went to zero. And service publishing was extremely long too before i went back to local....afterwards, publishing and connecting was a breeze..

Re: Slow performance administering services in ArcCatalog and ArcGIS Server Manager 1

JonathanQuinn — Mon, 28 Jan 2013 21:25:38 GMT

There are two bugs logged for ArcGIS Server 10.1 in regards to the performance when listing/accessing services:

ArcCatalog: NIM086874
ArcGIS Server Manager: NIM085259

The cause of both bugs is the requests that are made to validate the services. Currently, 10.1 SP1 resolved NIM085259 as there is one new admin tool to validate services. ArcCatalog, however, calls for 4 requests to the service, which causes the delay you're seeing. I don't know the timeline for a resolution, but that's the cause for the performance issue. You can use Fiddler to watch these requests go through:

Fiddler:
http://www.fiddler2.com/fiddler2/version.asp

Re: Slow performance administering services in ArcCatalog and ArcGIS Server Manager 1

GeraldLee__GISP — Tue, 29 Jan 2013 12:12:04 GMT

I appreciate the heads up. Thanks for that information !. Im glad to find out it was not me or my setup-- which is normally the cause.:).