topic Re: Caching tips, tricks, secrets, best practices? in ArcGIS Enterprise Questions

Caching tips, tricks, secrets, best practices?

RoyceSimpson — Tue, 09 Oct 2012 15:24:17 GMT

I've been attempting to cache a bunch of map services on 10.1 for a few weeks now. Some complete in one shot, others take several attempts and even then, I'm not sure if they are complete or not. The Caching status dialogue has yet to show results that give any confidence. The "job Status" tab on the Cache Status details dialogue has jobs that are "processing" even though the servers aren't doing anything and I can't seem to cancel them. I've attempted to run the "manage map server cache status" tool with no success (even though the tool reports "success" after completing). After running that tool, I get "0.0% of the tiles are present" even though I can view the service in the JavaScript viewer and see that there tiles present. I can even open the status.gdb fgdb and get all kinds of info. Is it reliable?

I'm very excited about where AGS 10.1 is going but wow, caching in 10.1 is like herding very temperamental cats. You can view/manage caching related activities from the service itself (by right clicking the service in catalog), going to the "server" tools in catalog or interacting (to a certain extent) with the service in the web based manager. There are tools for caching and reporting on caching and there are are services that do the same. You can run the jobs synchronously (same as "wait for job completion"?) or asynchronously (why not just always async?) report status via map service, or not (I haven't figured that one out yet). Each avenue has different UIs, capabilities, inputs, outputs, etc.

Any insight into this tricky caching business would be helpful. I've read through most (I think) of the online documentation but even then, there are enough "notes", "tips", "caution" and "legacy" statements that honestly it's hard to keep track of what the best path is.

Also, it would be great to see what bugs have been reported/accepted but I can't seem to find any esri pages for that. Esri, didn't you open that up a year or so ago?

Re: Caching tips, tricks, secrets, best practices?

MegPeterson — Thu, 11 Oct 2012 15:25:54 GMT

Hi Royce - I appreciate your comments as I'm running into a very buggy environment with caching at 10.1. Sometimes within seconds I'll get a result from the Geoproccesing/Results dialog that my cache has been completed, when clearly it hasn't.

I've found a couple tools to help ease the pain that may be helpful for you.

In Windows Explorer, open your cache directory and look for the status.gdb. Copy this file to another location. Open a new mxd and navigate to the copied status.gdb and bring in the TaskStatus feature class. You will be able to see the scales and success of each tile that has been run.

I then re-run the cache based on a new feature class that covers the error tiles and areas that weren't created.

The other tip I found when the View Cache Status dialog doesn't seem to refresh is the Manage Map Server Cache Status. It took about an hour to run yesterday, but finally spat out the correct percentages on job completion.

I really hope ESRI reads this post. The new caching tools are in theory great (!), but...would be helpful if not so temperamental.

Re: Caching tips, tricks, secrets, best practices?

RoyceSimpson — Thu, 11 Oct 2012 18:19:05 GMT

Thanks Meg for the reply.

Currently, any attempt I make to "manage map server cache status" on any cached service results in "0.0% of the tiles are present" even though the cache folders for that service are chock full of bundles.

I've no clue how to rectify that so that I can get map cache status' to report properly.

I can see that there is a status.gdb geodatabase and there are shapes with stats and that all looks good, but I can't get ArcCatalog or the web Manager to report anything other than "0.0% of the tiles are present".

I've resorted to looking at the size of each of the scale level folders (L1, L2... L15) to determine if they seem complete or not.

-r

Re: Caching tips, tricks, secrets, best practices?

AaronDrake — Mon, 15 Oct 2012 16:48:14 GMT

Wow, am I glad to hear I am not the only one having these same problems. I will definitely try out some of the suggestions that Meg presented. I agree that Esri is moving in the right direction with these tools, and updates to ArcCatalog's UI.

Hopefully they will get it ironed out with a service pack that addresses these issues.

Re: Caching tips, tricks, secrets, best practices?

JackZHANG — Mon, 22 Oct 2012 09:48:55 GMT

I just stared caching our map service in 10.1 and feel the exactly same pain as you guys. Also noticed to re-build the cache status it takes about the same time as build the cache... is it right? Definitely a few things in the caching tools need to be fixed in the next SP.

Re: Caching tips, tricks, secrets, best practices?

JustinHunter — Tue, 23 Oct 2012 14:47:48 GMT

I'm running into these issues as well. Support tries to help, but they're really not. I followed your steps and saw a huge portion of my map that the status.gdb thinks is successful but has 0 tiles... Is there anything I can do to tell it those tiles are empty?

My computer crashed during my initial caching (6 days in) and I have not been able to restart it. I followed support's steps but I've finally given up and deleted it completely. I've just begun re-caching it from the start. This time I'm using exploded instead of compact because I suspect it's compact that's causing my problems.

If it succeeds I'll run the processing job to change it to compact. In the meanwhile- does anyone know how to restart failed jobs? All I've witnessed so far is it creating empty bundles for all those that need to be created, then reporting as a success... when it is not.

Thanks

Re: Caching tips, tricks, secrets, best practices?

RoyceSimpson — Tue, 23 Oct 2012 14:56:16 GMT

I'm running into these issues as well. Support tries to help, but they're really not. I followed your steps and saw a huge portion of my map that the status.gdb thinks is successful but has 0 tiles... Is there anything I can do to tell it those tiles are empty?

My computer crashed during my initial caching (6 days in) and I have not been able to restart it. I followed support's steps but I've finally given up and deleted it completely. I've just begun re-caching it from the start. This time I'm using exploded instead of compact because I suspect it's compact that's causing my problems.

If it succeeds I'll run the processing job to change it to compact. In the meanwhile- does anyone know how to restart failed jobs? All I've witnessed so far is it creating empty bundles for all those that need to be created, then reporting as a success... when it is not.

Thanks

I haven't found anything to get the caching status or job reporting/cancellation stuff to work to any level of confidence. I've run about 10 caches now in 10.1 to the tune of about .5TB of compact bundles and every one of them reports "0% tiles complete". I've given up on all that cache status reporting stuff and just go to the cache folders and check the total size on disk of all the tiles for each cache... if they report somewhere close to what I had in 10.0x, then I don't worry about it. Hopefully SP1 will fully address these issues.

Esri, what is the ETA on SP1 and will it address any of these caching status reporting issues?

Re: Caching tips, tricks, secrets, best practices?

GarimaTiwari — Fri, 26 Oct 2012 21:44:16 GMT

I've been attempting to cache a bunch of map services on 10.1 for a few weeks now. Some complete in one shot, others take several attempts and even then, I'm not sure if they are complete or not.

>>These jobs can have certain cartographic elements that might be crashing the caching process. If you can consistently reproduce the problem, we would recommend contacting ESRI support so that we can better understand your service and server settings.

The Caching status dialogue has yet to show results that give any confidence. The "job Status" tab on the Cache Status details dialogue has jobs that are "processing" even though the servers aren't doing anything and I can't seem to cancel them.

>> It seems that the caching instances are crashing/failing before updating the cache status. Can you check if there are any Error Reports generated on your server by going to �??ArcGIS Server Manager > Logs> Error Reports�?�? if there are some error reports can you share them with us.

I've attempted to run the "manage map server cache status" tool with no success (even though the tool reports "success" after completing). After running that tool, I get "0.0% of the tiles are present" even though I can view the service in the JavaScript viewer and see that there tiles present. I can even open the status.gdb fgdb and get all kinds of info. Is it reliable?

>>Can you share the cache status & job status tab of your cache service? The information stored in the status.gdb is valid

You can run the jobs synchronously (same as "wait for job completion"?) or asynchronously (why not just always async?) report status via map service, or not (I haven't figured that one out yet). Each avenue has different UIs, capabilities, inputs, outputs, etc.

>>Cache jobs are always run asynchronously, disabling the property to �??wait for job completion�?� simply gives the flexibility to use the ArcMap session for other geo-processing tasks or even close it, while the cache job continues to run asynchronously on the server. Please see the following help link for details: http://resources.arcgis.com/en/help/main/10.1/index.html#/Manage_Map_Server_Cache_Tiles/00540000000p000000/

-Garima

Re: Caching tips, tricks, secrets, best practices?

GarimaTiwari — Fri, 26 Oct 2012 21:49:22 GMT

Sometimes within seconds I'll get a result from the Geoproccesing/Results dialog that my cache has been completed, when clearly it hasn't.

Meg, were you running your cache job with �??wait for job completion�?� disabled. If so then this is as designed. When a job is submitted with this property disabled the ArcMap session simply submits the cache job to server and shows successful, instead for waiting for job completion.
If you had not enabled this property then I think you are running into a bug. Can you share your data with us?

Re: Caching tips, tricks, secrets, best practices?

GarimaTiwari — Fri, 26 Oct 2012 21:52:23 GMT

I'm running into these issues as well. I followed your steps and saw a huge portion of my map that the status.gdb thinks is successful but has 0 tiles...
All I've witnessed so far is it creating empty bundles for all those that need to be created, then reporting as a success... when it is not.

>>Justin, it seems you are running into a bug, can you share your data, server logs & error reports on server with ESRI support, so that we can reproduce & resolve this problem.

Re: Caching tips, tricks, secrets, best practices?

JustinHunter — Mon, 29 Oct 2012 12:37:40 GMT

>>Justin, it seems you are running into a bug, can you share your data, server logs & error reports on server with ESRI support, so that we can reproduce & resolve this problem.

I have a ticket open with esri.ca but they've really been unable to help. Unfortunately, they've given me so many suggestions that all that is now gone. My data is still there, but I've re-done my map and the server was wiped out and reinstalled.

I began trying this on exploded mode- we'll see how it goes.

Re: Caching tips, tricks, secrets, best practices?

nicogis — Wed, 31 Oct 2012 06:42:00 GMT

if it can be help, esri posts on New features for planning and monitoring map caches in ArcGIS 10.1

Re: Caching tips, tricks, secrets, best practices?

JustinHunter — Wed, 31 Oct 2012 11:51:54 GMT

I have a ticket open with esri.ca but they've really been unable to help. Unfortunately, they've given me so many suggestions that all that is now gone. My data is still there, but I've re-done my map and the server was wiped out and reinstalled.

I began trying this on exploded mode- we'll see how it goes.

In addition to this- I just got off the phone with Esri support- they suggested I build the cache each province/territory at a time and make a zipped backup each time in case it fails. So far so good. It also gives AGS and my computer, which I still do my day-to-day work on as caching is happening in the background, some time to breathe. I'm also skipping a majority of water tiles since before I was doing AOI Envelope of Canada, which includes tons of water to the north/northeast, now it'll just be empty tiles which will be ignored and my maps background will be water blue.

Also, I don't know if this makes any difference, but I turned on the 'Allow clients to cache tiles locally' and it seems to be using much less resources than before. Again, could be in my mind.

Re: Caching tips, tricks, secrets, best practices?

JackZHANG — Fri, 02 Nov 2012 01:47:02 GMT

The Manage Map Server Cache Status tool also has problem to deal with user specified cache boundary. I have a map service want to re-build the cache status. The mxd's extent is whole Australia, but in the map service property settings - Area of interest to cache I specified a much smaller area as the cache boundary. When running the tool, the Area of Interest Polygon, which is a feature class called ManagedMapServerCacheStatus::area_of_interest in a scratch geodatabse for the tool got nothing in it. I even went into the scratch database and manually appended my cache boundary to the feature class, but the tool still tries to rebuild the cache status for the whole extend of the mxd, rather than the specified cache boundary in the map service properties or in the tool...

Re: Caching tips, tricks, secrets, best practices?

GarimaTiwari — Wed, 14 Nov 2012 21:58:02 GMT

Jack,

The ManageMapServerCacheStatus tool is designed to work using "Full extents" of the map service when running with "REBUILD_CACHE_STATUS" mode. In the upcoming release we plan to provide a new mode which can work with user specified feature.

--Garima

Re: Caching tips, tricks, secrets, best practices?

DerekLaw — Tue, 20 Nov 2012 21:30:35 GMT

Hi Royce et al.,

Any insight into this caching business would be helpful.

FYI please see this blog post, Accelerating map cache creation

Hope this helps,

Re: Caching tips, tricks, secrets, best practices?

LuyangRen — Tue, 27 Aug 2013 15:37:28 GMT

>> It seems that the caching instances are crashing/failing before updating the cache status. Can you check if there are any Error Reports generated on your server by going to �??ArcGIS Server Manager > Logs> Error Reports�?�? if there are some error reports can you share them with us.

Hi Garima,

I have the same problem of "0.0% of the Tiles are present". I think the Caching Tools is crashed because any service with cache cannot be display now. But any service without cache works fine.

So I checked the cache status in ArcCatalog and it says "0.0% of the Tiles are present" and the status is "Tile generation is not in progress". Attached is the screenshot of the cache status in ArcCatalog. And the job status is empty.

[ATTACH=CONFIG]27021[/ATTACH]

I check the logs in ArcGIS Server Manager and I got this record:
The containing process for 'System/CachingTools' job 'j44d79cdb591e47bab6bf33444d2d101e' has crashed.

The story is my colleague created a map service with a huge size cache, which is before the time of the above record. So I believe the Caching Tools is crashed. So any service with cache created after that cannot display.

Could you help me fix the problem?

Thank you
Luyang