I decided to write a blog post today, to record the steps I took to create a high performance and useful mosaic dataset of aerial imagery. I'm writing this post because I wasn't able to find a straightforward explanation of which parameters and variables to use to create overviews/pyramids/etc and I'd like to remember this as well as benefit others if possible.
The dataset I'm working with is the size of a small county in rural, so its not huge, but also not something we want to load in one single image, especially since this imagery is shared between AutoCAD users and GIS.
This imagery has all ready been split up into tiles the size of a township-range section, so we have about 300 tiles each about 10MB in size. Each image is a plain jpg, with its own spatial reference file (the jpgs are georeferenced). The imagery files are located in a network drive all in one folder with names like img_T109N R20W_03_NE.jpg. My tiles are about 10,000x10,000 pixels. The images do not have pyramids, and in my case I didn't use them due to the extra file size. But you might so keep that in mind in later steps. Lets get started!
Using ArcGIS Pro appears to have a performance benefit as well, so I'll be using that.
1. Create a geodatabase: The first thing to do, is create a new empty geodatabase on our local C drive. Yes, we can move it to a network drive later, but it turns out creating mosaics is much faster on a local hard drive because of all the file read/write activity.
2. Create a new dataset: Create a new mosaic dataset inside this geodatabase. Make sure if you use numbers, they don't come first in the filename. Also be sure to set the correct coordinate system.
3. Add the rasters to the dataset: Right click on the mosaic, and click Add Rasters... This will open the geoprocessing tool.
- Change the dropdown to Folder
- Browse to the folder where your existing rasters are stored and add it to the list. This should be the UNC path if the files are on the network (\\servername\path\to\2016_imagery_tiles)
- Check or uncheck the Build pyramids option depending on your use case
- Check the boxes to generate statistics for the mosaic and for the images. This helps with performance down the road.
- Optional: Under Environments we can set the parallel processing to use 4, or the number of CPUs on our pc
- Click Run to add the files to your mosaic
4. Define overviews: Here's where it gets tricky. ArcGIS provides an automatic method to skip this, by simply building overviews and letting the software define them automatically, but I haven't gotten it to work correctly.
- Open the Define Overviews gp tool
- Add your mosaic to the parameter
- Clear the extent using the clear button (looks like a refresh button) Then the tool will use the extent of the mosaic by default
- Set the Overview Tile Parameters. You can play with these to get them where you prefer, using the symbology we create in the next step.
- Pixel Size: I set mine to 2. This means 2ft because my projection is in ft. This is the size of the largest scale overview. This will be used later on, so remember this value.
- Number of Levels: This is I think the number of overview levels it will try to create. But it probably uses the next value too
- Overview Sampling factor: I set mine to 3 (default)
- Number of rows/columns. I left this at the default
5. Symbolize mosaic: This isn't necessary but can be very helpful to visualize where your overviews will appear before you spend a lot of time generating them.
- Symbolize mosaic footprint layer to show the types Overview, Unprocessed Overview, and Primary.
- Take a look at the Unprocessed Overview lines. If you don't like where they're at, open the attributes table, select the lines with category = Unprocessed Overview, and delete. Save Edits, and repeat step 4.
5. Build overviews: This step is relatively simple, but the most time consuming since it will be processing the mosaic into overviews
- Open the Build Overviews gp tool
- Uncheck the box "Define Missing Overviews". We already did this.
- Optional: Under Environments we can set the parallel processing to use 4, or the number of CPUs on our pc.
- Run the tool and wait.
6. Review Min/Max Pixel Size values: This final steps just ensures that our overviews will be utilized correctly. I'm not sure if it is always necessary, but it seems that using this method, the mosaic automatically sets the primary (high resolution) images to display at smaller scales, which can greatly hamper performance. So what I do is set these primary images to have a max pixel value of the value that we set in the Pixel size of the level 1 overview. That way, the overviews will be displayed until we reach the value that we set.
I have set the MaxPS and HighPS values above to 3, because that is what I set my first level overview to a minimum pixel size.
Hopefully, you have a high performance mosaic with overviews created! Feedback appreciated, leave me a note.