Raster mosaic datasets: Overview Quality control

[NicholasO_Connor]

Dear ESRI Support and GIS Users.

Can anyone help me with how best to get good quality overviews? I know this is a little subjective…my case:

I create the Raster Mosaic Dataset in a fGDB and add the following tifs to it.

200 tifs, each tif 20MB in size, all tifs fit together like squares on a chessboard

Each tif:

Number of bands: 3

Cell size 1.25 x 1.25

Format: TIFF

Source Type: Thematic

Pixel Type: unsigned integer

Pixel Depth: 8 bit

NoData Value 256,256,256

ColourMap: absent

Pyramids: absent

Compression: LWZ

TOOL PARAMETERS

DefineOverviews

Overview Tile Parameters

Pixel size: 1.25

Number of Levels: 12

Rows: 5120

Columns: 5120

Overview Sampling Factor: 2

Overview Image Parameters

Resampling Method: BILINEAR

Compression: LWZ

Calculate item visibility – take Default settings.

RESULTS

Scale 1:8000 –= viewing the original tif, not the overviews – quality good

Scale 1:8000 – 20000 = viewing the overviews – quality poor. I know this is a little subjective – but the image is quite
blurred, hard to read labels…

TESTED:

[1] I have tried changing the Number of Levels and Overview Sampling Factor

[2] I have tried changing the Resampling Method and Compression

Nothing seems to help.

QUESTIONS

Is the problem in the original tifs? For example the source Type or Compression used?

Is the problem in how I define the Overview Generation?

Feedback welcome and appreciated.

[PeterBecker]

Nicholas

The issue related to the Source Type being Thematic.

When data is thematic it is important not to apply bilinear sampling to it as the resulting values have no meaning. In the generation of the overviews ArcGIS is honoring the thematic flag in the source data and forcing the sampling to be Nearest. This results in the poor apparent quality. The thematic flag is not overwritten even if you change the property of the dataset. There are some solutions:

1 - Get the data originally created not to use the thematic flag. As the data is greyscale I would also recommend that the data be changed to 1band (vs 3Band)

2 - Generate Pyramids for the data. Use the Build Pyramids and Statistics tool which takes as input a workspace. You can turn the statistics generation of, but set the pyramid resampling technique to bilinear, the compression can be set to JPEG 95. This will generate pyramids for the complete dataset and here the sampling is forced to Bilinear. Create the mosaic dataset in the same way, but you can now generate the overviews with a base pixel size of 25m. You will find the images are the expected quality. I would typically recommend using pyramids so as to reduce the number of overviews and enables a user to lock the mosaic dataset to a specific raster and see the raster at all scales.i

[NicholasO_Connor]

Hello Peter

First and foremost thank you very much for following-up on this point and yes, “2 - Generate Pyramids for the data” worked. Perfect.

Secondly, to aid my understanding, you wrote as an explanation, “I would typically recommend using pyramids so as to reduce the number of overviews and enables a user to lock the mosaic dataset to a specific raster and see the raster at all scales.” Up until now I have not generated “any” pyramids just “overviews” which has always produced good quality results and good performance. Your answer to generate pyramids gives the “good quality results” part but would it have any impact on the performance?

I am still a little confused when are pyramids being used or when overviews are being used in ArcMap? I now see the Catalog table (name, objectid, minPS, MaxPs...) and when I had just Overviews it was simple to see the MinPS and MaxPS and calculate the item visibility  and thus the scales if tifs or overviews are being used. How do the Pyramids fit into this table?

Finally, once again thank you for your help.

[PeterBecker]

Pyramids can be considered as reduced resolution versions of the primary rasters and overviews as reduced resolution datasets of the mosaic dataset. Raster datasets can have pyramids and they are generally recommended especially when the datasets are larger (>3000 row/cols). How pyramids are stored is dependent on the format. In many cases ArcGIS adds files along with the primary rasters with the extension .OVR. Pyramids enable the datasets to be viewed at smaller scales without needing to read the full resolution data. For example if a user locks to a single raster so that only that raster is displayed,and then zooms out, then if no pyramid exist the system would need to read and resample the whole rasters (which would take time). Overviews can be considered as pyramids for the mosaic dataset, but are stored as individual raster datasets. They are created so that one can zoom the small scales such as the extent of the mosaic dataset and remove the need for the system to open and display potentially thousands of individual rasters and their pyramids, which would take a long time.

The existence of pyramids can be identified by the LoPS and HiPS values in that footprint table. The LoPS defines the Pixel Size of the raster and the HiPS indicates the pixel size of the lowest resolution pyramid level that makes sense to use. This is determined based on if pyramids exist and the number of rows/columns in the raster. At pixel sizes above the HiPS it is better if overviews exist. If you just add rasters and then define overviews the system will look at the MaxPS value for the rasters to determine the pixels size to create the overviews at.

The MinPS and MaxPS values in the footprint table define the pixels sizes at which a raster will be displayed. For most primary rasters the MinPS is 0 indicating that how ever much you zoom in the raster should be shown. The MaxPS value is typically the same as the PixelSize that the overview is generated at. Similarly for the overviews each level of the overview will have a MinPS, MaxPS range. The Calculate Item Visibility tool can be used to determine the appropriate MinPS,MaxPS values and it analyses the overlapping rasters at different scales to determine the values. In some cases it is better to manually set the values.