Increasing Image Service performance when dealing with multiple coordinate systems

[DanJensen]

All,

I have combined NAIP imagery and local imagery in a single mosaic dataset.  The NAIP imagery covers the entire area I want to see, the local imagery with 6" resolution gives details not available in the NAIP imagery.  Local imagery overlaps the NAIP imagery in targeted areas.  NAIP is in UTM and the local imagery is in state plane NAD27.  Most of my vector data is in state plane NAD83. 

I have added both sets of images to a Mosaic Dataset defined as NAD83 to match the vector data.  When I view both the imagery and vector data in my flex application, the performance is slower than I would like.  It can take up to ten seconds to see the image service when the application is first opened. 

How could I increase performance of the image service with the least negative side affects?

-Dan

[EricRice]

Most of the NAIP I come across is MrSID format.  This could easily be where your performance hit is coming from.  Have you built overviews on the mosaic dataset?

[DanJensen]

The NAIP is definitely tiff format, I made sure of that.  I agree with your comment that NAIP is often .sid.

Good question regarding how it came together. 
I first made a mosaic dataset of the NAIP imagery.  I built overviews on these.  I also built a mosaic dataset of the local imagery.  Because these are high resolution photos and I do not want to see them at all scales, I did not make overviews of these.  I then created the combined mosaic dataset to hold everything.  When I added rasters, under raster type I selected �??Table�?? then pointed to the existing mosaic datasets .  The overviews and all the tiffs, and the local imagery came into my new mosaic dataset.  I tried rebuilding the overviews, but got errors.  It was in this combined mosaic dataset that I specified the NAD83 coordinate system.

-Dan

[PeterBecker]

The difference in the coordinate system is not the issue. The coordinate conversion does have an influence on performance, but it generally not so large to warrant converting it. Most likely is the issue is related to pixel size ranges. You had mentioned that you have two mosaic datasets. One of NAIP imagery with overviews and another of Local imagery without overviews. Please check if both of these on their own perform as required. Note that the projection of the Mosaic Datasets is only the default projection. Client applications specify the required projection. Is you are using a Flex app and the base map is in WebMercator then the imagery from the image service will also be requested in WebMeractor and the server will perform the required conversion. Please check the performance of each Mosaic Dataset as a service separately. For the local service no imagery will be displayed till you zoom in. Now when you create the derived Mosaic dataset and load into it the NAIP and Loca lMosaic dataset, ensure that cell sizes are not recomputed. It is possible that they have been recomputed and the display range of the Local imagery has been changed such that they now get displayed at small scale an as a result a lot of separate image need to be loaded slowing down the initial zoom. Not also that the first time you connect to a service is may be slower as the server may suspend the service if it is not used. When starting up again it may take a while to get up to speed. Is the speed issue you are having there after the initial start up image?

[JeffreySwain]

I am also curious about the compression that your tifs use.  Can you please indicate what the compression says under the Properties > Source Tab.  I have seen tifs with YcbCR compression exhibit the same slow performance that some sids produce.  For the test cases that I have seen, the compression is often nearing the 20:1 that you see with SIDs.  You may also consider not using the pyramids with your tifs and generate more overviews.  If you are seeing the slow performance when you are accessing those 6 inch tifs, then perhaps this is the issue.

I would recommend considering recreating the image service with more overviews by ignoring the pyramids with your tifs.  To ignore the pyramids with the tif, open the raster pyramid options and set the maximum levels to 0.  When you generate the overviews for that mosaic dataset you should see more overviews generated.  If the pyramids with the tif are causing the slow performance, then the additional overviews will improve that.

[DanJensen]

In response to Peter's advice.  I checked the mosaic dataset and there were some issues from my previous efforts to build overviews.  I determined to rebuild the combined mosaic dataset.  Before I did, I reviewed and optimized the MaxPS and MinPS values of the source mosaic datasets.  When I recreated the combined mosaic dataset using State Plane NAD83 I checked the Max and Min PS values and indeed they had changed, so I optimized them in the new combined mosaic dataset as well.  I then timed how long it took to see these photos the first time in my flex application.  It took around 12 seconds for the imagery to appear.

In response to jbswain.  I checked the properties of several individual tiffs in Arc Catalog and next to compression it always said 'None', next to pyramids it said 'level: 5, resampling: Bilinear'.  The exception was the overviews where it said 'LZW', but next to pyramids it said 'absent'.  I may still try building more overviews later.

First I experimented with the coordinate system.  I created a new mosaic dataset using the UTM coordinate system of the NAIP imagery and overviews.  I created a new map service in UTM for my vector data too.  I published a copy of my flex application configured to use these UTM service rather than the State Plane services I used earlier.  When I timed how long it took to see the photos the first time it was 4 seconds.  Performance was 3x better on the initial load than when I used the coordinate system of the vector data.

I also got a performance boost when I changed the maximum number of rasters per mosaic in the Image Service from the default of 20 to 5.  Performance was 2x better on the initial load, going from around 12 seconds to around 6 seconds.

However, these performance results sometimes vary and I am guessing this is due to whether or not the images retrieved are in the arcgisserver output directory, network traffic and other impacts to performance.

Other things I am considering experimenting with: reproject NAIP & local imagery to NAD83 State Plane, add LZW compression to the native images if possible, and supressing pyramids in favor of more overviews.

[PeterBecker]

Dan
Its not clear to me what you mean by time for the first image displayed. If the service has not been used for a while, then the very first request may take longer, due to a number of system and software components that may need to get reset. What is the time for subsequent pan/zoom at different scales? If the app in initiated with a zoomed in area vs overview, is the performance different?

I think there are a number of different things going on.

Structure of the original data- You mention that tiling is None. This is an indication that the original data is not tiled. Tiling the imagery can have a significant positive effect on the performance especially on slower disk systems. If the data is not tiled and the width (in columns) is large then the disk need to do a lot of reading to read even a small part of the disk. Tiling the source will often increase the performance

Disk Subsystem - The performance of the disk system can have a substantial effect on total performance. Is the current data running on an external SAN or similar. If possible copy some of the data to a local storage and see if the performance changes.

NoData - Please check on the setting of NoData on the imagery. What type of imagery is being served. Does it need NoData. If no data is set then additional processing is required to check for NoData and find pixels to fill in those areas. With 10.1 there are specific properties of the Mosaic Dataset that define how NoData should be handled. If your data has large borders of NoData then is is also very advantageous to rebuild the footprints to exclude NoData areas.