topic Re: New Map Service Cache Size in ArcGIS Enterprise Questions

New Map Service Cache Size

DaveTenney — Fri, 24 Mar 2017 14:53:40 GMT

All,

i was just curious if how the way things are cached in 10.3.1/10.4.1 vs 10.2.2 will cause the cache to be larger in size?

we have a map service that we wanted to cache, nothing has changed other than the version, we upgraded from 10.2.2 to 10.4.1

10.2.2 cache size 96gb

10.4.4 cache size 125gb

we have havent changed a single thing, not one. we havent changed anything in regards to the map itself and its data, nor have we changed anything in the settings of the service when we publish.

thanks

Dave

Re: New Map Service Cache Size

MichaelVolz — Fri, 24 Mar 2017 15:17:06 GMT

Did you try to just create an empty cached service at 10.4.1 and just copy the cache from 10.2.2 to see if that old cache would still work in 10.4.1?

Re: New Map Service Cache Size

RebeccaStrauch__GISP — Mon, 27 Mar 2017 18:49:57 GMT

I have never seen the size of the cache get larger, but I typically do as Michael recommends and copy the 10.2.x cache from an old machine to new. Or, if upgrading in-place, that is on same machine, before updating I move the caches to a temp folder and then move back after the upgrade...actually speeds it up a bit.

i skipped over 10.3/10.4, but have been testing moving caches from 10.2.2 to 10.5.1. My old workflow for this did not work as it did beteween 10.2.x and 10.3.x (on a test server). In the past, I did as Michael suggested by creating the service and a blank cache (for folder structure) then I would copy the cache over and restart the service. All was good.

Theis process did not work with 10.5 and might be the same issue with 10.4. Instead, I copied the cache over first, with the folder name changed, if necessary to match the service name (I needed to changed mine). THEN I created the service with the matching name and it recognized the cache in the properties. However, for the cache to work, I needed to use the upgrade cache tool.

The reason for this is that 10.2.x used the compact bundle/bundlex format. The compact version with those tow files combined into just a .bundle file was introduced in 10.3 but could still read/recognize the 10.2 compact format. If not sure about 10.4 but 10.5 will not recognize the 10.2 bundlex compact version at all...other than in the upgrade tool. But if you set up the blank 10.5 structure and copy files in, it will not.

again, not sure about 10.4, or why the size would be different, but in case it is related to the compact format changes, I thought I would mention it.

Re: New Map Service Cache Size

DaveTenney — Mon, 27 Mar 2017 18:55:28 GMT

Today support confirmed that starting with 10.3.1 the way bundles are created it would cause an increase in size. Support was not able to confirm at the time if the same process was used in 10.4.1 that was in 10.3.1 but would look into it. We also tested this as we still have a vm running 10.2.2 using the same map and same data as our 10.4.1 vm. we published the map and build only 1 layer, see below:

10.2.2
	layer	tiles	size	exploded
	12	560	2.38mb	560 files
10.4.1
	layer	tiles	size	exploded
	12	690	2.72mb	690 files

as you can see there is a difference, at layer 12 not much of a difference but once we get to layer 19, were talking GB's worth of difference. again....same mxd, same source data, same publish settings.

dave

Re: New Map Service Cache Size

RebeccaStrauch__GISP — Mon, 27 Mar 2017 19:04:56 GMT

So, sounds like you original question re would the cache be larger is yes.

i see you are showing exploded caches. Any reason you are not using the compact formats. Much smaller caches overall (whether 10.2 or 10.3+ version) and many fewer files to copy between machines/folders.

only down size I see re the compact versions are the differences between the versions, and that a 10.2.x client could not read the 10.3 compact format directly (i.e., as a raster set, bypassing any service). But if all you clients are 10.3.+, it shouldn't matter, unless running into the issue I mentioned in my first comment.

Re: New Map Service Cache Size

DaveTenney — Tue, 28 Mar 2017 13:54:08 GMT

we exploded the level so we could visually see what area was being covered in 10.2.2 vs 10.4.1.

we only use compact, but for diving into this issue we wanted to see as much as we can, since you can't really see much with the compact other than how man bundles were created. when we exploded the cache we were able to see that there were 5 extra folders created in 10.4.1 vs 10.2.2.

the issue really is we told the client that map service 1 and 2 take an "x" amount of space. those services were built using 10.2.2, now that we want to upgrade to 10.4.1, the services are being built differently. so now "x" has increased, we are waiting on ESRI to see if they know what the expected % increase in size is.

dave

Re: New Map Service Cache Size

MichaelVolz — Tue, 28 Mar 2017 14:09:20 GMT

Dave:

Besides the increase in space of the cache at 10.4.1, have you encountered any other serious issues with the use of the cache in 10.4.1? Have you tried to also create a compact cache with the same map service? I ask because I am still running AGS at 10.3.1, but my organization plans to upgrade to a new version this summer.

Re: New Map Service Cache Size

DaveTenney — Tue, 28 Mar 2017 14:24:55 GMT

Michael,

we only publishing using the compact format, we only exploded one level of a cache for testing reasons to see what the area was that was being covered. we compared the exploded level from 10.2.2 vs 10.4.1, otherwise all of our services are always published using the compact cache.

we have run into a lot of issues, most of these i think are not unique to 10.4.1, i think they are more of just the typical bugs that seem to travel from one version to the next!

1) basic ArcPy functions not performing as expected

2) ArcGIS server manager is shotty at best (im not sure why ESRI hasn't spent more time on this)

my suggestion to you is to never solely rely on the server manager!

3) I would be cautious of ESRIs claim of "in-place" upgrades!

a) this does seem to work ok for desktop

b) be very cautious with your approach to upgrading server!

disclaimer: i am only one drop in the esri bucket, im sure many many others have upgraded with no problems, and everything has worked like a charm. so please take my input for what its worth.

dave

Re: New Map Service Cache Size

MichaelVolz — Tue, 28 Mar 2017 14:41:02 GMT

Dave:

In terms of your issue with AGS Manager, are you using Chrome 54+ as your browser? There are several issues with Chrome and AGS Manager so I would suggest using IE11 - examples include problems registering database connections (BUG-000099629) and problems displaying statistics on the mapservices (BUG-000101518). There might be even more issues with Chrome 54+ and AGS Manager that I have not encountered.

Re: New Map Service Cache Size

DaveTenney — Tue, 28 Mar 2017 17:19:33 GMT

The problems we were noticing were easily recreated in all browsers...

the big problem was the Manager interface was not in sync with the services. we could recreate the issue of a service appearing as stopped in the manager but was actually running and could indeed be accessed. we were also seeing the opposite of this. we were seeing this in 10.2.2, 10.3.1, and now 10.4.1 so its hard for us to rely on the manager interface much for certain things.

dave

Re: New Map Service Cache Size

MichaelVolz — Tue, 28 Mar 2017 18:45:33 GMT

Dave:

Were you stopping mapservices in ArcCatalog and then seeing in AGS Manager that the mapservices were still running?

If that is not what you were seeing, can you explain in more detail so I can see if my 10.3.1 AGS environment is encountering the same issues as you with AGS Manager?

Re: New Map Service Cache Size

DaveTenney — Tue, 28 Mar 2017 19:04:52 GMT

1) we would stop services via python script

a) ArcSOCs would drop from task manager but remain as "started" with an instance in use with in the manager

b) we would try and start the service back up with the same python script

i) ArcSOCs would come back up in task manager, but the server manager was stuck in "starting" phase

2) we would attempt to start and stop service via the manager interface and we would run into the same problem

a) stop service and arcsocs would drop

b) start service and arcsocs would spin up

c) server manager would be stuck at "starting"

this all plays into not really understand what the Server Manager is reaching out to see if the service is running or not. this is why we have a hard time really relying much on the manager interface, it seems too many time it actually doesn't know what is going on real-time.

dave

Re: New Map Service Cache Size

MichaelVolz — Tue, 28 Mar 2017 19:18:17 GMT

Dave:

Do you stop the services to update data used in the service? If so, are you trying to delete a feature class and then recreate it? If yes to that, have you tried a truncate and append approach to updating data so services do not need to be stopped when only data is being updated but not the feature class schema?

Re: New Map Service Cache Size

JeffPace — Fri, 01 Sep 2017 17:33:26 GMT

I am seeing almost an order of magnitude size increase from 10.5 to 10.5.1. Bundles that were ~300mb are now over 2gb! I changed from jpg to png32, so i did expect some increase, but that seems excessive.

Re: New Map Service Cache Size

MichaelVolz — Fri, 01 Sep 2017 17:55:59 GMT

Are you able to do a comparison between 10.5 and a previous version of AGS? I ask because in the upgrade from AGS 10.3.1 to 10.5.1 the cache size has decreased for my cached mapservice. As such, I'm wondering if cache sizes were improved (decreased) from say 10.3.1 to 10.5.0, but then ESRI changed something between 10.5.0 and 10.5.1 to increase the cache size.