Statewide Mosaic dataset and tif pyramids

StephanieSnider · ‎08-05-2013

I'm in the process of creating a statewide mosaic dataset in a file geodatabase. I will be using tifs that already have 5 levels of pyramids, bilinear. Statistics have already been built for the individual tifs on all 4 bands. To optimize performace, I have a couple questions about the "add raster to mosaic dataset" process:

1. Should I choose to use all 5 levels of the pyramids from the original tifs? If not, why?

2. In the Advanced options of this tool, there is a "build raster pyramids" option. I'm assuming I don't need that since my original tifs already have pyramids. Is this correct?

3. Is there any performance advantage to creating a mosaic dataset in ArcGIS Desktop 10.2 file geodatabase verses a 10.0 File Geodatabase?

Is there anything else you could recommend to help make this very large mosaic perform fast? Specifically I need the pan and refresh to display quickly. We have so far not been able to reproduce the quick display speed of the statewide raster mosaic with a statewide mosaisked dataset. Any advice is welcome!

Stephanie

JeffreySwain · ‎08-06-2013

Remember the speed of the mosaic dataset comes from having optimized source rasters and the use of the overviews. In terms of your questions, I think that you could try it each way. For instance, when adding the rasters to the mosaic dataset, you can choose in the options to set the max pyramid level to zero. Then generate overviews normally. This will force the mosaic dataset to ignore your pyramids and generate more overviews. In some circumstances, this is preferable. However, I would still create a similar test mosaic dataset with the pyramids added to see how that performs. The hardest part of making recommendations like this is that while there are recommendations that work in some situations, each raster dataset might be slightly different. Finally, I would leave off the option to generate pyramids as you already have them and I would think there is not as much difference with regard to performance between the versions of geodatabase. What will be different is the functionality that is only available at 10.2 will be lost.

Also don't forget to Analyze your mosaic dataset for specific recommendations.

StephanieSnider · ‎08-06-2013

Thanks! I'll give this a try with the two datasets - one with pyramids/overviews and one with only overviews. I have another question about building footprints. When you add rasters to a mosaic datatset, it creates footprints that are the same size as the original raster (*.tif). Is there any benefit to building footprints after the rasters are loaded to redefine the footprint size. For example, I'm using NAIP imagery that has 6328 columns and 7605 rows per raster. If I build footprints to reduce the footprint size to 2000, would this enhance the display performance of the mosaic dataset? I know footprints are used to remove no data areas, but I wonder if a smaller/larger footprint would affect the performance of the overall dataset.

StephanieSnider · ‎08-07-2013

After testing on a subset of the NAIP imagery, it is clear that using all 5 pyramid levels from the original NAIP rasters is twice as fast to display/refresh than a mosaic dataset that uses only overviews. In addition, using the default tile size of 5120 (rows and columns) results in a quicker display than using a smaller tile size of 2000. Still�?�even with the pyramids and the larger tiles, the display/refresh lag can be 1-4 seconds at various scales. Decreasing the footprint size didn't make any difference in the refresh time with the mosaic dataset.

From my experience with mosaic datasets compared to my experience with raster mosaics, I�??m not convinced that a mosaic dataset will give us better performance. A mosaic dataset is most certainly much easier to create, but the end result (quick display and refresh) is not as good as a raster dataset.

JeffreySwain · ‎08-07-2013

I am not sure what to say about your testing. What was the overview factor you chose? What did the Analyze Mosaic Dataset reveal?

I am also not sure what you did with rebuilding the footprints, most of the time the reason to rebuild the footprints has to do with the presence of NoData in the rasters. The footprints will remove that information from being displayed. If the source rasters are beings referenced too much then it would not be an accurate comparison. I would pay attention to the spread in the Min/Max PS with the mosaic dataset. If you have a large range then that may be the issue. Ideally with the overview only raster this should happen really quickly. So the Max PS of the source rasters should be much lower than with the pyramids. In testing here, the mosaic dataset has always performed faster when designed correctly. Also with the more data you add, the more the performance difference will be revealed. The larger the mosaicked raster dataset gets, the more unwieldy it will become. I am including the help documentation on the recommended mosaic dataset building and a discussion of overviews vs pyramids.

Another consideration should also be the time it will take to produce the raster dataset. The mosaic dataset creation process is much faster than creating the raster dataset.

For further performance once you have the image service created, caching will push the image service to even better performance. I would further recommend contacting Support to demonstrate the performance issue that you are seeing.

StephanieSnider · ‎08-07-2013

Rebuilding footprints was just an experiment. I wanted to see if it would make a performance difference, and it did not.

My Overview sampling factor was the default at 3. I'm working ArcGIS Desktop 10.0 which does not have the Analyze Mosaic Dataset option. However, I do have a PC with 10.2 and plan to run the Analyze tool later on.

I have contacted ESRI support on creation of mosaic datasets before. They recommended moving the mosaic dataset out of ArcSDE and into a File Geodatabase on a network fileshare. While this certainly did increase the performance, it is still not as fast as a raster dataset in the same environment.

You're right about using caching to increase preformance for image/map services.

Is there a particular ESRI mosaic dataset expert you would recommend that I contact on this issue? If so, please feel free to message me directly.

StephanieSnider · ‎08-08-2013

I found the solution!!! When I created pyramids on the source rasters, I took the default -1 setting, letting ArcGIS decide how many pyramids were needed. It created 5 levels of pyramids. I did a test on a subset of images, creating 10 levels of pyramids. Then I created the mosaic dataset, using the 10 pyramid levels and overviews after that. WOW!! Totally much much faster. I'm getting instant to 1 second display/refresh times. This is the key! Yeah!