For several weeks I have been struggling to generate a fairly sizable map cache. The anticipated size of the cache will be about 800GB, with the final level, 14, estimated to be about 530GB.
This level 14 generation has failed on me a few times, and this typically results in an ArcCatalog crash. Which means I do not see errors in ArcCatalog about why it crashed. It's taken several days to generated 365GB of the 530GB level.
I ran again with "RECREATE_ALL_TILES", and several days later, it crashes again without a good error.
Seemingly running the cache creation with "RECREATE_EMPTY_TILES" -- this similarly takes an extremely long time while seemingly not growing the cache size at all.
I only seem to get farther along if I delete Level 14 altogether and start over. Because this takes several days per trial, I don't have a lot of attempts under my belt. I wanted to ask here before I continued with this. Two questions:
2. We have had tremendous lag issues with an On Demand map cache in our previous experience, but I would like to look at this again. Since we have a significant portion of the cache generated already, is it possible to use this partial cache and let time "fill in the blanks" by allowing users to generate new tiles on demand?
Your insights and expertise are much appreciated, thank you in advance.
The first one has been invaluable for me. Like the tool you linked, it creates footprints based on the missing tiles, but it generates them by making requests to your existing cache rather than relying on ArcCatalog's error messages. However, it can be slow because it makes constant HTTP requests through your REST endpoint. The second script appears to do the same thing but by checking the files on disk instead of using HTTP, so it may be much quicker -- I haven't had a chance to try it myself yet.
In your case, I would also try to figure out the cause of your crashes -- in my experience this kind of thing is usually caused by a symbol or label that the renderer can't handle for some reason, so I suspect it will keep crashing until you can track down the cause. You may need to do some trial and error by deactivating layers one by one and running small test caches on the affected area until you figure out which layer is the problem.
2. We have had tremendous lag issues with an On Demand map cache in our previous experience, but I would like to look at this again. Since we have a significant portion of the cache generated already, is it possible to use this partial cache and let time "fill in the blanks" by allowing users to generate new tiles on demand?
Yes, and you should experience less lag since you already have much of the map generated. It should only lag when a user zooms/pans to an area that hasn't been cached yet, so the more cache you can generate ahead of time, the better.
It sounds as if you're using 10.0, correct? Considering the size of the cache, I'd say you should monitor the CPU usage and RAM of the machine running the cache, so the machine with ArcCatalog open. Are there any errors in the server logs? Take a look at this resource, which help in caching large areas.
