Mosaic Datasets, Imagery Storage, and Analysis...

JonM32

Hi Everyone,

Looking for advice on imagery management and working with Mosaic Datasets. To start, here is a bit of background: I work for a local government and manage imagery from various vendors (primarily tiled orthoimagery). Formats are dominated by full resolution GeoTIFFs and compressed MrSIDs, and temporally range multiple years.

My management question is:

1. Does using the Windows Zip tool to compress a folder into a .zip file degrade image data in any way when storing it as a backup? From what I understand, zipping file folders uses lossless compression, but I can't say I'm an expert at this and am not sure how to tell if say 50 years from now, someone wants to unzip the data for use, will be the true original GeoTIFF data that was in there from the vendor? [Any insight on how others are storing backup data is welcome...currently I run with having working data for mosaic datasets on one server, and backup data on another, with a third external drive storing another backup copy]

 

Mosaic Dataset (MD) questions:

1. When you zoom in close on a tile in a MD, are you actually viewing the full resolution image tile stored in the image data source directory or is it just the pyramid built for that image?
2. The "Download Rasters" functionally of a MD, does that allow you to download, or make an exact copy (no resampling or moving over pixels), of the original image tile you selected on the MD? Is the image data altered in any way when using this?

 

Analysis question:

My end goal with the imagery I manage is to use it to build data layers for our region, mainly land cover classifications in ArcGIS Pro. I have a good amount of experience doing so in other software packages, with some in ArcGIS Pro, but what I'm not sure on deals with using MD to do the analysis. In the past, I've always just used an actual image tile from a vendor...or exported a custom image for an area of interest. However, this seems excessive or not possible for very large extents. So, my question is:

1. When using a MD referencing GeoTIFFs, say I want to do an image segmentation for a small area within a city. Can I preform this operation on the MD, and will it be performed on the actual full resolution GeoTIFF or on it's pyramids and/or overviews? Additionally, does having the transmission setting under "Allowed Compression Methods" set to anything other than "None" alter the data I would be running my analysis on? (since all my MD are using JPEG on this setting to display a bit faster for basemap purposes). Ideally, I want to work with the true pixel values from the raw imagery the MD are referencing, since those values will guide all of the analysis work and I'd like to take advantage of the data available.

 

Thank you in advance, for any and all insights!

Jon

PeterBecker

There is very little point compressing imagery using ZIP. I recommend against using ZIP to compress a directory of images. If you are looking for lossless compression of imagery then simplest is to use Deflate or LZW. There are some better lossless compression methods, but the differences will be limited. If you have collection of imagery that are not compressed and you want to compress them then its better run them through Optimize Raster to compress. Some datasets such as elevation compress better using compression such as LERC. For more details on how to convert and optimize imagery check out OptimizeRasters.   

Using Mosaic Dataset is the best way to manage the data as this references the source data (full resolution and any pyramids) and enables the definition of additional attributes, functions to process the data on the fly and enables quick access to all the data either directly in ArcGIS Pro or serving as image services through ArcGIS Image Server.  When directly using the mosaic dataset in ArcGIS Pro it acts as a layer defining functions applied to the appropriate imagery. Yes you are accessing the full resolution source data when zooming in. The compression for transmission that you set is the compression used to transmit the data. It is the client application that can set it and the compression quality at any time. There is no reason to only allow ‘None’.  As you mention setting this to say JPEG  can speed up transmission of the data.

There are Export and download options. Export will typically resample the data and apply any defined functions on the data. The application defines the resampling compression etc. Download downloads the original pixels were possible. I say were possible as if you use say a MrSID then the data is compressed and the data must first be decompressed before writing it to a new file so the data structure will change but not the pixel values (unless you define another lossy compression).

You can directly use Mosaic datasets as input to your analysis tools. This enables large collections of imagery and rasters to be seen as a single dataset, with control of many properties. If you want to ensure that the pixels are exactly the same as those stored on disk then you should use the environment variables to set the appropriate pixel size, projection and extent to ensure the requested pixels align with the source.

Do check out the ArcGIS Imagery Workflows page, it provides lots of best practices