Caching tips, tricks, secrets, best practices?

RoyceSimpson · ‎10-09-2012

I've been attempting to cache a bunch of map services on 10.1 for a few weeks now. Some complete in one shot, others take several attempts and even then, I'm not sure if they are complete or not. The Caching status dialogue has yet to show results that give any confidence. The "job Status" tab on the Cache Status details dialogue has jobs that are "processing" even though the servers aren't doing anything and I can't seem to cancel them. I've attempted to run the "manage map server cache status" tool with no success (even though the tool reports "success" after completing). After running that tool, I get "0.0% of the tiles are present" even though I can view the service in the JavaScript viewer and see that there tiles present. I can even open the status.gdb fgdb and get all kinds of info. Is it reliable?

I'm very excited about where AGS 10.1 is going but wow, caching in 10.1 is like herding very temperamental cats. You can view/manage caching related activities from the service itself (by right clicking the service in catalog), going to the "server" tools in catalog or interacting (to a certain extent) with the service in the web based manager. There are tools for caching and reporting on caching and there are are services that do the same. You can run the jobs synchronously (same as "wait for job completion"?) or asynchronously (why not just always async?) report status via map service, or not (I haven't figured that one out yet). Each avenue has different UIs, capabilities, inputs, outputs, etc.

Any insight into this tricky caching business would be helpful. I've read through most (I think) of the online documentation but even then, there are enough "notes", "tips", "caution" and "legacy" statements that honestly it's hard to keep track of what the best path is.

Also, it would be great to see what bugs have been reported/accepted but I can't seem to find any esri pages for that. Esri, didn't you open that up a year or so ago?

MegPeterson · ‎10-11-2012

Hi Royce - I appreciate your comments as I'm running into a very buggy environment with caching at 10.1. Sometimes within seconds I'll get a result from the Geoproccesing/Results dialog that my cache has been completed, when clearly it hasn't.

I've found a couple tools to help ease the pain that may be helpful for you.

In Windows Explorer, open your cache directory and look for the status.gdb. Copy this file to another location. Open a new mxd and navigate to the copied status.gdb and bring in the TaskStatus feature class. You will be able to see the scales and success of each tile that has been run.

I then re-run the cache based on a new feature class that covers the error tiles and areas that weren't created.

The other tip I found when the View Cache Status dialog doesn't seem to refresh is the Manage Map Server Cache Status. It took about an hour to run yesterday, but finally spat out the correct percentages on job completion.

I really hope ESRI reads this post. The new caching tools are in theory great (!), but...would be helpful if not so temperamental.

RoyceSimpson · ‎10-11-2012

Thanks Meg for the reply.

Currently, any attempt I make to "manage map server cache status" on any cached service results in "0.0% of the tiles are present" even though the cache folders for that service are chock full of bundles.

I've no clue how to rectify that so that I can get map cache status' to report properly.

I can see that there is a status.gdb geodatabase and there are shapes with stats and that all looks good, but I can't get ArcCatalog or the web Manager to report anything other than "0.0% of the tiles are present".

I've resorted to looking at the size of each of the scale level folders (L1, L2... L15) to determine if they seem complete or not.

-r

AaronDrake · ‎10-15-2012

Wow, am I glad to hear I am not the only one having these same problems. I will definitely try out some of the suggestions that Meg presented. I agree that Esri is moving in the right direction with these tools, and updates to ArcCatalog's UI.

Hopefully they will get it ironed out with a service pack that addresses these issues.

JackZHANG · ‎10-22-2012

I just stared caching our map service in 10.1 and feel the exactly same pain as you guys. Also noticed to re-build the cache status it takes about the same time as build the cache... is it right? Definitely a few things in the caching tools need to be fixed in the next SP.

JustinHunter · ‎10-23-2012

I'm running into these issues as well. Support tries to help, but they're really not. I followed your steps and saw a huge portion of my map that the status.gdb thinks is successful but has 0 tiles... Is there anything I can do to tell it those tiles are empty?

My computer crashed during my initial caching (6 days in) and I have not been able to restart it. I followed support's steps but I've finally given up and deleted it completely. I've just begun re-caching it from the start. This time I'm using exploded instead of compact because I suspect it's compact that's causing my problems.

If it succeeds I'll run the processing job to change it to compact. In the meanwhile- does anyone know how to restart failed jobs? All I've witnessed so far is it creating empty bundles for all those that need to be created, then reporting as a success... when it is not.

Thanks

RoyceSimpson · ‎10-23-2012

I'm running into these issues as well. Support tries to help, but they're really not. I followed your steps and saw a huge portion of my map that the status.gdb thinks is successful but has 0 tiles... Is there anything I can do to tell it those tiles are empty?

My computer crashed during my initial caching (6 days in) and I have not been able to restart it. I followed support's steps but I've finally given up and deleted it completely. I've just begun re-caching it from the start. This time I'm using exploded instead of compact because I suspect it's compact that's causing my problems.

If it succeeds I'll run the processing job to change it to compact. In the meanwhile- does anyone know how to restart failed jobs? All I've witnessed so far is it creating empty bundles for all those that need to be created, then reporting as a success... when it is not.

Thanks

I haven't found anything to get the caching status or job reporting/cancellation stuff to work to any level of confidence. I've run about 10 caches now in 10.1 to the tune of about .5TB of compact bundles and every one of them reports "0% tiles complete". I've given up on all that cache status reporting stuff and just go to the cache folders and check the total size on disk of all the tiles for each cache... if they report somewhere close to what I had in 10.0x, then I don't worry about it. Hopefully SP1 will fully address these issues.

Esri, what is the ETA on SP1 and will it address any of these caching status reporting issues?

GarimaTiwari · ‎10-26-2012

I've been attempting to cache a bunch of map services on 10.1 for a few weeks now. Some complete in one shot, others take several attempts and even then, I'm not sure if they are complete or not.

>>These jobs can have certain cartographic elements that might be crashing the caching process. If you can consistently reproduce the problem, we would recommend contacting ESRI support so that we can better understand your service and server settings.

The Caching status dialogue has yet to show results that give any confidence. The "job Status" tab on the Cache Status details dialogue has jobs that are "processing" even though the servers aren't doing anything and I can't seem to cancel them.

>> It seems that the caching instances are crashing/failing before updating the cache status. Can you check if there are any Error Reports generated on your server by going to �??ArcGIS Server Manager > Logs> Error Reports�?�? if there are some error reports can you share them with us.

I've attempted to run the "manage map server cache status" tool with no success (even though the tool reports "success" after completing). After running that tool, I get "0.0% of the tiles are present" even though I can view the service in the JavaScript viewer and see that there tiles present. I can even open the status.gdb fgdb and get all kinds of info. Is it reliable?

>>Can you share the cache status & job status tab of your cache service? The information stored in the status.gdb is valid

You can run the jobs synchronously (same as "wait for job completion"?) or asynchronously (why not just always async?) report status via map service, or not (I haven't figured that one out yet). Each avenue has different UIs, capabilities, inputs, outputs, etc.

>>Cache jobs are always run asynchronously, disabling the property to �??wait for job completion�?� simply gives the flexibility to use the ArcMap session for other geo-processing tasks or even close it, while the cache job continues to run asynchronously on the server. Please see the following help link for details: http://resources.arcgis.com/en/help/main/10.1/index.html#/Manage_Map_Server_Cache_Tiles/00540000000p...

-Garima

GarimaTiwari · ‎10-26-2012

Sometimes within seconds I'll get a result from the Geoproccesing/Results dialog that my cache has been completed, when clearly it hasn't.

Meg, were you running your cache job with �??wait for job completion�?� disabled. If so then this is as designed. When a job is submitted with this property disabled the ArcMap session simply submits the cache job to server and shows successful, instead for waiting for job completion.
If you had not enabled this property then I think you are running into a bug. Can you share your data with us?

GarimaTiwari · ‎10-26-2012

I'm running into these issues as well. I followed your steps and saw a huge portion of my map that the status.gdb thinks is successful but has 0 tiles...
All I've witnessed so far is it creating empty bundles for all those that need to be created, then reporting as a success... when it is not.

>>Justin, it seems you are running into a bug, can you share your data, server logs & error reports on server with ESRI support, so that we can reproduce & resolve this problem.