Managing single-picture raster datasets

[WilliamBaugh]

I am working on reorganizing and managing the imagery data for my company. We are a natural resources group with terabytes of historic orthophotos across the county.  As of now, everything is stored as flat-file-based system with all of our orthophotos stored in folders on our server. I'm wanting to reorganize everything the best way possible according to ESRI. 

After doing research, it seems pretty obvious that making use of mosaic datasets is the preferred method when working with tiled-orthophotos (multiple pictures that make up an entire mosaic). 

My main question is what to do with orthophotos that have already been mosaic'd (usually as a SID file) and the original data is not available. Does is make sense to create a mosaic dataset for a single orthophoto?  Should I keep the SID file as is? Should I put the SID file in a file geodatabase?

Any advice is much appreciated. Thanks!

[PeterBecker]

For each of the projects you have (eg imagery from a specific year) create a mosaic dataset. I'll refer to this as a source mosaic dataset. Typically you would create overview for each of these to enable fast access at small scale. For all the images have already been mosaicked together, create another mosaic dataset. In both mosaic datasets ensure that you add fields that include attributes such as year and project ID. You can set mosaic method rules to define which image are displayed by default and access any of the images. Id recommend you also create a dervived mosaic dataset that combines all the mosaic dataset into one. This will provide access to all your imagery.

See Standard Imagery Management workflow in  ArcGIS Imagery Workflows | ArcGIS 

Typically you should not need to restructure the data. If you are moving to better equipment etc then I would suggest you look into possibly changing the format of the imagery to something like TIF (tiled with overviews). See OptimizeRasters. This should ensure that all your metadata and data structures stay the same (nothing lost), but make access faster especially for some of the older files. The MrSID files can be left as they are.

[WilliamBaugh]

Thanks for replying. Just to make sure that I completely understand, and looking below at an example of the data I'm working with:

1) You are suggesting that I create a source mosaic dataset for every orthophoto/Lidar I have, even if it's a single, mosaic'd SID file?

2) You are suggesting that I create a derived mosaic dataset for (for example) every full county source mosaic dataset, or a derived mosaic dataset for every Johnson River" source mosaic dataset

Is that right? Thanks again for your help. 

An example of the data I have would be something like:

1) Full county ortho from 1937 (separated into 1000 different small tiffs)

2) Full county ortho from 1997 (single SID file, no other data available)

3) Ortho from "Johnson River" in 1956 (100 different small tiffs)

4) Ortho of Johnson River in 1972 (single SID file, no other data available)

5) Ortho of the estuary the Johnson River 2017 (single SID file, no other data available) 

6) Ortho of Smith Island from 1999

[PeterBecker]

Create a source mosaic dataset from each of the collections: 1,3 and create overviews. Add attributes for Year

Create a mosaic dataset of all the single MrSid  2,4,5. Add Attributes for year

Create a derived mosaic dataet and add the above two into it. (The number of records in this should equal the total or all the seperate mosaic datasets)

[WilliamBaugh]

Thank you again! Sorry, but I have one more follow up..

Should I create one single mosaic dataset for all my MrSid files (probably around 100 different files/orthophotos)?  Or should I create one mosaic dataset for each individual MrSid file?

As far as the Derived mosaic dataset is concerned, are you suggesting that (after I get every orthophoto into a mosaic dataset) to combine everything into one single Derived Mosaic Dataset and use that as my main "folder" for serving orthos to the rest of people at my company?

Thanks!

[PeterBecker]

Yes you can add all the MrSID to one mosaic dataset.

Yes combining all into one can enable a single service that provides access to all your imagery. You can set rules as to what is displayed by default. When you get new imagery you can add to the service and all app continue to work but using new imagery.

[CodyBenkelman]

I thought I should comment here just to be sure it's clear.  The "Derived" mosaic dataset is our recommended method when combining mosaics.  The key is to use the TABLE raster type, not the default RASTER DATASET raster type.  See discussion here Derived mosaic datasets—Standard Workflow_Creating Mosaic Datasets | ArcGIS 

[WilliamBaugh]

Thanks again for your quick responses!

I've read about as much as there is to read about mosaic datasets online, scoured everything on the "Standard Workflow" of mosaic datasets, done everything that both Peter and Cody have suggested, and watched an hour long video (among others) hosted by Peter (Esri Mining Webinar Series – Streamlining Image Management and Processing Workflows). 

I'm using Arc Desktop 10.5 advanced and I do NOT have ArcGIS Online or ArcGIS Server. Since I just use Arc Desktop, I'm failing to see the point/benefit of compiling all my source mosaic datasets into a Derived Mosaic Dataset. I don't see any real functionality for it. Can you please tell me if I'm missing something?

What Peter demo'd in that video (11 minutes in) - where he uses the "Image Interpolation" app - looks awesome. But from what I can tell, that's only available to Online/Server users. Like that app does, it would be super useful to be able to 1) zoom in to particular location 2) be able to see every single image file within the derived mosaic dataset that overlaps that location, and 3) be able to easily select the one (or multiple) image files that I want to view for that particular location. Is that possible without Arc Online/Server?

When I do that now, it's only showing the top layer and doesn't let me select any other images. . I don't see how the attribute table, which shows all the different files, is useful. Screenshot attached.

Can you please tell me the benefit of Derived Mosaic Datasets if all I use is Arc Desktop? 

As I reminder, the task I have at hand is organizing all of our of imagery data and serving it in the best way possible to all of our employees. Our GIS data is shared over our work network. 

Thank you!

[WilliamBaugh]

This might be related to my previous question, or it might not. But in addition to the question I asked earlier (which I'd still love to figure out), I'm also curious to know the advantages/disadvantages of putting multiple Derived Mosaic Datasets in a single file geodatabase (as opposed to multiple Source Datasets in a single Derived Mosaic Dataset). 

Here is what I did: 

- I have 20 different orthophotos, in their original form, in separate directories on our server.

- I then created 20 different file geodatabases, and created 20 Source Mosaic Datasets within each of these respective file geodatabases (so each file geodatabase has one single Source Mosaic Dataset in it)

- The workflow (and your suggestions) says to create a single Derived Mosaic Dataset and put those 20 mosaic datasets into it. I did this, the screenshot in my previous post shows that Derived Mosaic Dataset (which I don't really see the advantage of; it also doesn't function very intuitively)

- To me, it seems to make more sense to create a "Derived File Geodatabase" to serve these mosaic datasets to my colleagues. For this, I created a new file geodatabase (called "CompleteOrthos_byYear"), created 20 new Derived Mosaic Datasets in this file geodatabase (I stopped at 11 in the screenshot) 

Can you please tell me why i should not do this? To me, it seems to be a very clean and easy-to-understand way to organize all my orthos and serve them to my colleagues. It also seems a lot more intuitive to have it in this format than to have 20 orthophotos all combined in a single mosaic dataset.

Screen shot attached.

Thanks again for your help!