Publishing map services containing rasters very slow

[SimonRoss_User]

We're on ArcGIS Server 10.1 and find that publishing any map servcies that contain raster layers is very slow (an increases with the number of ratser layers in the map).  Publishing vector-only map servcie by comparison is extremely rapid.  We're not copying data to the server when publishing or creating caches.  It seems like the publishing should just be a simple process.  Where we use image services we have no comparable issues but this is not always possibel or preferred.

Any ideas on how we can speed up publishing?

[MichaelVolz]

You've analyzed the mapservice during publishing to make sure the raster location is registered with AGS so the rasters do not get copied to the server?

[WilliamCraft]

By "raster layers" are you referring to raster datasets, raster catalogs, mosaic datasets, or individual images in TIFF, IMG, JPG, or some other format?  What type of storage location is being used for the raster layer sources (RDBMS vs. file system)?  Do the layers take the same amount of time to draw in the map document as they do when rendering the service?  For your map service, is it being served dynamically or is it cached?

[SimonRoss_User]

We are definitely not copying data to the server when publishing.  This issue occurs with any raster we've tried including the following: 

file based rasters - TIFF and JPEG
mosaic datasets (file based images)
unmanaged raster catalogs

Just tried doing this with a raster catalog this mornign and after about 3 hours theservcie publishing result (in ArcGIS Desktop) came back as ERROR 001269: Compressing the Service Definition Failed.  Haven't had a chance to look up what this means yet.  Previously when working with raster layers it has always published eventually

[WilliamCraft · Accepted solution]

The error you are reporting indicates the following based on Esri's help: "Make sure you are not accessing files in the folder containing the service definition."  Is it possible that someone had one of the raster images open during the attempt to publish?  If you're running Windows 7, 8, or 2008 R2 the go to the Start menu and type fsmgmt.msc, hit enter, and click on the Open Files node.  If any of your rasters are showing as open, then mass select them and right click them to Close.  It's possible that some of those files are locked by another process that should have been terminated, perhaps leading to issues with publishing. 

Regarding the main issue you reported, can you check the permissions on the directories involved with the raster data on the file system?  It may be possible that the logon account used for the AGS Windows Service requires read permission on the source data in order to properly publish.  That being said however, the very first thing to make sure of is that the folder(s) containing all of your raster imagery is registered with ArcGIS Server as a data store.

[CraigColeman]

Hi William,

I've just come across this post regarding slow publishing for map services with raster layers.

I'm experiencing pitiful performance when I publish my map service that contains 4 layers based up different raster catalogues.  The paths to the rasters in the catalog are to a UNC share which is visible to and registered with the server.  The path to the actual catalog is on the same UNC as the rasters.

The 4 raster catalogs have 56, 815, 2285 and 10951 rasters respectively.

The process is still extremely slow even though analysis confirms that no files need to be copied to the server - the more rasters I have in the catalog the slower this is.  Publishing the service takes several hours (2+ hours).

The network is 10Gbps, server is a virtualised (VM Ware) 2-node configuration in a single arcgis server instance each with 40GB RAM and 4 processors.  The workstation publishing the service has 128GB RAM, 48-processors and masses of free disk space (>1TB).

Any insight you could lend would be useful.

Craig

[DaveBarrett]

Hi Craig,

Have you had any joy with this issue? We are having the same problems with unmanaged raster catalogs with thousands of small tiles taking hours to publish if at all. We are running two arcgis servers (10.1) in a site. Vector data publishes in minutes with no problems. Both servers use a single data store for the rasters and the rasters are access via UNC paths.

I have also tried to publish the data using individual rasters in ArcMap and it takes just as long.

Thanks

Dave

[CraigColeman]

Hi Dave,

I didn't get any response to this posting other than yours.

I've just learned to accept that it is just going to take this long and I work it into my day somehow.  Yesterday in fact, I republished the services after some updates and it took just over an 1.5 hours.  I believe a Mosaic Dataset would be faster if you have AGS Advanced but we don't (yet) so I can't test this.  My catalogues are all unmanaged but I might see what happens if I make them managed?

It would be nice to get a commentary from Esri as to what it is doing when it publishes a raster catalogue.  I believe that it's not just testing for the existence of each raster but opening each raster and testing it for something.

Craig

[DaveBarrett]

Hi Craig,

Had a bit more of alook into this today and used a managed catalog which published within minutes for the same data. I have been in touch with our ESRI rep here and they going to visit us and work out the issue. Any answer I get I'll let you know. Mosaics are ok but at 10.1 you need the image server extension to publish which we don't have.

Dave

[CraigColeman]

Hi Dave,

I think I've discovered the insanity that is causing this to perform so badly.

During the publication process, ArcGIS appears to be copying (yes, copying!) each raster to a subfolder beneath C:\Users\user\AppData\local\ESRI\Desktop10.1\Staging\connection this is cleaned up at the end but it's a complete waste and (depending on your environment) can impede your ability to publish more than one service at once.

Why it is doing this I do not know nor can I fathom!  In my case, this is copying roughly 40GB of data one raster at a time from the shared network file location which is a) registered with the server and b) visible to the server.

If you create a Service Definition without connecting to the server, it still takes as long but there doesn't appear to be any of the copying.

I'd love to know what Esri say about this!

Craig