On demand tiles are very slow

[NikolasStevenson-Molnar]

I'm trying to publish a large dataset (about 1GB as a shapefile) as a tiled service to a server running 10.2. It needs to be available for all zoom levels (0-19), but that would be too may tiles at the larger levels. So I'd like to pre-render levels 0-12, then cache the rest on demand. The first part, is no problem; I've published the service and successfully created the pre-rendered tiles. However, the on demand tiles are unacceptably slow, taking up to several minutes to return and/or returning empty responses. I've made sure the dataset has a spatial index and dynamic rendering is quite fast at that level.

I've previously had this dataset published on Server 10.0 and it worked fine with on demand tiles. I've also tried upgrading to Server 10.3. This fixed the issue with empty responses, but the response times are still unacceptably long. The pattern of responses is odd, too: some tiles will come back within 10 seconds, others will take up to a minute, and a few will take up to two minutes.

I know that on demand tiles require generating supertiles, but even given that, the on demand tiles are incredibly slow.

Any ideas?

Thanks,

_Nik

[RebeccaStrauch__GISP]

Nik, if you had these previously cached in 10.0, if the data hasn't changed and the tile schema is the same, you may be able to use the same tiles.  If you are using the newer bundled format, it may take a few steps to convert and then manually merge them to the new service, but typically that works just fine.

Another option, is to figure out the locations that will be hit often and scales 13-19 and just pre-cache those areas.

[NikolasStevenson-Molnar]

Hi Rebecca,

Thanks for the suggestions! Unfortunately, it won't work for us to use our existing cache because it's incomplete. It's likely people would hit uncached areas pretty quick. Similarly, we can't effectively predict which areas are most likely to be hit at the 13+ levels (especially as we get close to 19). Whichever areas we pre-cache, the other areas would be used frequently enough that 2+ minute render times are going to be a problem.

To give a bit more context, this is a US-wide dataset (incl. Hawaii and Alaska) which means an estimated 64 billion tiles at level 19.

_Nik

[RebeccaStrauch__GISP]

Big job!

Just to throw this out there, in case you have access to the Image Server extension....I tested it with some new Alaska imagery, including adding a few cache scales (down to source scale), and what took about 8 weeks in the past took about 4 days.  I tested on both 10.2.2 and 10.3.  Scales larger than source took about 5 days by itself, but in the past would have probably taken a couple months....so, I'm sold on the speed.  I'm still in test mode (hoping to get the extension), but you should see if you have access to it already.  I'm assuming if you are doing nation wide, you're in a larger organization. (??)

[NikolasStevenson-Molnar]

Thanks, I'll look into Image Server. We're actually not a very big organization, but it does look like we have access to the Image extension.

[NikolasStevenson-Molnar]

Oops, I spoke too soon. It was just a trial license for the Image extension.

[RebeccaStrauch__GISP]

Not a cheap extension, but in my opinion, worth it because of personnel and computer crunching time.   Do you have an EDN license with ArcGIS Server setup?  You should be able to use that to prove it's usefulness. That's what I used to test.

[NikolasStevenson-Molnar]

Yes, I do have an EDN subscription, so that could work for testing. The other problem, however, is that I'm dealing with a feature dataset and if I understand correctly, the Image extension is for images and rasters only.

[RebeccaStrauch__GISP]

Oops, sorry.  I didn't catch that.  I think you are correct on the image/raster only. 

Not knowing how complex you service is, but have you tested the speed of the service as dynamic and not cached?  It might not be an option with the extent, and/or with complex symbology, but I know many of my vector services I used to cache, I now just have as dynamic.  Speed has really improved in the past couple releases.

[NikolasStevenson-Molnar]

Yes, I have tried it as a dynamic service. The interesting thing there is that the speed is pretty reasonable once you've zoomed in. It's not great rendering the full extent, but as long as it has a spatial index, the dynamic rendering while zoomed into to a specific areas is pretty good.

Which makes it all the more strange that the on-demand cache is so incredibly slow.

Overall, rendering at full extent is slow enough that I do want to use it as a tiled service. If I could automatically switch from tiled to dynamic past a certain threshold, that would work, too but that's not supported (and even if it were, it would need to be supported by the map clients, too). So I'm afraid it's tiles or bust!