Manage Map Cache Tile: Errors at 10.2.2

[DavidColey]

We are experiencing brand new caching issues at 10.2.2. I am unable to either DELETE_TILES or RECREATE_ALL_TILES from an existing cache. The error we keep receiving is:

Line 30Output failure, error string = Error moving bundle Failed to cache extent: -9223211.962860 3087885.815688 -9079003.170544 3271103.291998 at scale 577790.55428899999.

The error occures when running RECREATE_ALL_TILES or DELETE_TILES operations of the Manage Map Cache Tiles (run as a tool or script) when run from a dedicated server gp cluster machine. Our clustered setup consists of a network file server storing our config, security and data-stores, as well as all server and system directories.  Two servers serve as a mapCluster for map and image services, a single server in a gpCluster for caching and user-defined gp services, and a virtualized web server for the WebAdaptor.

This has occurred as of lastweek whether reading from the GIS SDE database OR from a data-store file gdb. All permissions for directory shares and folder and file security have been accounted for.

When the error occurs, the entire cache becomes corrupt, making any other caching operations in-operable (delete cache, delete tiles, etc). During the recreate, the bundlx file is being removed but the bundle file is not, thus not allowing the tool to overwrite thebundle from the admin-defined D:/arcgistemp folder(s) on the GP server.

Thus far, the only solution has been to kill the service, stop the site, and then remove the entire cache directory, restart the site, recreate both the service and the cache.

Has anyone encountered this?

Thanks-
David

[MichaelVolz]

I do not have an answer for your question, but a possible workaround.

Has the mapservice that you have cached been around across multiple AGS versions (e.g. 10.0, 10.1, and 10.2)?

Do you cache the mapservice because it is too slow as a dynamic mapservice?  If so, was the performance of the dynamic mapservice for this service been retested in later versions where performance of dynamic mapservices has been improved?

I recommend this approach  because a dynamic mapservice (if performing at an acceptable level) is much easier to maintain as you do not need to account for updating the cache which is currently creating problems for you.

[DavidColey]

Thanks Michael.  The services have been through several updates, but this is the first time we received this error.  We are able to create entirely new caches (which we have), but then cannot overwrite or delete those cache bundles.  The tool deletes the bundle exchange file (bundlex) leaving in place the locked bundle, and then the tool cannot overwrite or delete the bundle as it has lost itsreference

[MichaelVolz]

Have you tried to use the same service as a dynamic mapservice instead of a cached mapservice?

I ask because my organization used a bunch of cached mapservices at v10.0, but at v10.2 with better performance we have been able to switch most services to dynamic so we no longer need to update the cache and deal with the problems that come with having to manage the update of the cache.

Does the below thread have any relevance to your current problem:

http://forums.arcgis.com/threads/104423-Caching-please-help!?highlight=manage+cache+error

[DavidColey]

Of course.  I've tried dynamic basemaps at each release and whether the data is local or sde, the performance suffers, at least for us.  I would consider a workaround something like what I used to do at 10.0 or less:  that is, caching on a dev o rstage environment and then moving the tiles to produciton via OS commands.  But since 10.1, this is somewhat problematic with the advent of the Status gdb.  Over time, this tends to corrupt the service.

Plus, we shouldn't have to go through these incredible scripting gymnastics just to get tools that are supposed to work; to work.  Each release since 10.0 has introduced some new issue that is addressed in a subsequent release, which in turn introduces some new error . . . .
David

[MichaelVolz]

David:

Did your cached service exist at v10.2.0 (That is where my current production environment sits)?  If so, did this problem exist at this release?

Or did you migrate from an earlier version (e.g. v10.1 sp1) directly to v10.2.2 whereby you bypassed v10.2.0?  I ask because I might be creating cache mapservices soon, so I'm wondering if I will also encounter your problem when I need to update the cache.

[DavidColey]

No, this is brand new at .2; I have never seen this error before in any release.  No, I have not skipped any releases.  We updated to 10.2.2 from 10.2.1 and in general, things have worked very well with this exception thus far.

[ILIMAKENNEDY]

This is discouraging, and I'm sorry that I don't have a solution. I was looking forward to installing 10.2.2 in order to fix the caching issues that I've had since migrating from 10.0 to 10.2, but with your new errors, I'm extremely hesitant to fix one thing only to introduce another.

My issues at 10.2 are with the following bugs (both fixed at 10.2.2) which makes my cached basemap appear to have missing tiles at some scales:
NIM099294  Improve performance of tiles access when the cache directory is on a shared network location (UNC).
NIM099281  A significant decrease in throughput occurs when requesting cached tiles through the ArcGIS Web Adaptor (IIS).

If you find a work around, please post.
Thanks!

[MichaelVolz]

Ilma:

You indicated that the below NIM is a problem you are encountering with v10.2.0 cached mapservices:

NIM099294 Improve performance of tiles access when the cache directory is on a shared network location (UNC).



Did you use this same cache at an earlier AGS version (e.g. v10.0 or v10.1) without this issue?

[ILIMAKENNEDY]

Michael,
Thanks for a quick response. I used the same cache at 10.0 where the tiles were stored on a NAS and accessible via share without issue. I've rebuilt the cache many times because I thought something was corrupted, but I still have a few scales where tiles disappear (even though the number of files on disk is the same each time) or some scales just take forever for the tiles to appear and the end users complain that the apps are too slow.

When I publish a service with a local copy of the cache directory on a different server, the map displays perfectly.

ilima