Can ArcGIS Desktop load folder of tile images

[JoeMcCann]

I have a directory of images which make up a single raster "library". The naming convention contains the information on tile "positioning", as seen in this screen grab:

Is there a way that I can load this image into ArcGIS Desktop 10.5, or ArcGIS Pro?

My preference would be to have the application only request those PNG files from disk that it requires as I pan/zoom the image, rather than loading all the files at once. Similarily, I'd like to avoid combining these files into a single "large" file, if possible.

If such a thing is possible, I'll be a) impressed, and b) happy.

Thank you for the help.

[DanPatterson_Retired]

mosaic datasets seem to be in vogue at present  related links although I think the required reading needed upfront will not satisfy your second requirement

[JoeMcCann]

Thank you Dan. I definitely considered creating a mosaic dataset.

My admittedly non-standard application is this...

We are GSuite cusstomers and we use the desktop "File Stream" application to store and share files. Rather than simply maintain a local copy of all data, that Google "File Stream" application only fetches files that are requested, as they are requested. My mind imagines that to be a rough analog of the way a tile server works, so I had hoped to be able to pre-create a library of tiles, and then have a desktop application (Arc*) load those layers, requesting only those tiles it happens to require at the time.

That's a lot of text, sorry. Hope it makes sense.

My understanding of a mosaic dataset is that it is a library of images, but that it does not generate a tile image for each zoom level, so really, entire images within the mosiac would need to be fetched "at-a-time", rather than just a specific zoom level.

[DanPatterson_Retired]

Joe, that is out of my realm and I suspect most people that frequent the Imagery and Remote Sensing place.

You might want to try a 'Share' (top right) with another location within the GeoNet Structure that deals with data logistics... I can't recommend one, but hopefully someone else can.

[CodyBenkelman]

Joe

ArcGIS Desktop (ArcMap or Pro) can do what you're asking in your first post - simply

• create mosaic dataset (MD)
• right click "add rasters" and point to your directory of images
• then you can load the MD into the map and it will only access the required images as you pan & zoom.  it does not load all images into memory.
• You may need to right click "create overviews" to create reduced resolution views for when you zoom out (otherwise if arcgis is trying to resample ~100 images to create a low res view the response can be slow

However I can't comment on GSuite and I don't know how loading images into ArcGIS can help you load them into GSuite.  

If you have ArcGIS Server or ArcGIS Online, you could use Desktop to create Raster TIle Cache (which is the same tiled format that Google and Bing use) and serve that.  If you're on a local network, it is POSSIBLE that GSuite can read our raster tile cache format but I have never heard of anyone trying it.

Cody B. 

[CodyBenkelman]

My post above was a bit rushed - I have some more time - if you can tell us more detail about your configuration and versions (what version of ArcGIS?) etc. we may be able to advise further.  

I just now realized you have PNG files, but is there no georeferencing metadata for ArcGIS to read (other than the filename)?  I assumed these files were georeferenced.  If not, then my advice above will not be successful, so my apologies for making it sound simple.  There's no built-in method for assigning georeferencing information based on the filename, so you'd have to build a script to parse the filenames and create georeferencing metadata that ArcGIS will understand. 

If you can tell us what coordinate system you are using, and how to identify the georeferencing from the file names, there may be people reading this that can help with advice on scripting.  Or if there is any chance you could go back to the source and get the original data in a georeferenced format, the mosaic dataset could work with it directly.

Cody B

[JoeMcCann]

Hi, and thank you.

I did some more testing to see if there was something more ESRI-centric that could accomplish what I am looking for, which is a way to load a folder of tile images into Arc* in a way that only requests those tiles that are required, factoring-in zoom level and display extents. I don't think my previous posts were at all clear in that regard. For what follows, you can disregard the original set of files I had been inquiring about in my original post.

An ESRI mosaic dataset seems to only request data from disk in a per-file way, but makes no allowance for zoom levels. I could build overviews, but that overview file is actually quite large as well, so a single request for any portion of the mosaic's overview would invoke a request for the entire ovr file.

From the original data source (a single TIF-format raster dataset, projected in epsg:3578), I created a new tile scheme (the "Generate Tile Cache Tiling Scheme" geoprocessing tool) specifying a Storage Format of "Exploded". I then created the tile files using the "Manage Tile Cache" tool, which dutifully output a folder structure containing an image for for every "tile". Promising, I thought.

When I load that resultant "raster dataset" into ArcGIS Pro or ArcMap 10.5 or ArcCatalog 10.5, the application is actually still requesting all tile image files at each zoom level, disregarding the display extent. Panning is flawless, of course, once that entire zoom-level's data are loaded, but zooming-in invokes that "whole-level" file-system request 😞

So, probably it's the case that to have Arc* request individual tiles, you actually have to be serving those tiles from a tile server rather than a folder on disk somewhere. Makes sense, too bad for me.

Unless I've missed something.

Thanks!

[CodyBenkelman]

Joe

I'm not sure where you are getting this information, but it's incorrect

I could build overviews, but that overview file is actually quite large as well, so a single request for any portion of the mosaic's overview would invoke a request for the entire ovr file.

The *.ovr file should not be very large - if your input mosaic covers a large extent, the *.ovr is automatically broken up into numerous sub-files.  The OVR file is a tiled TIFF file (internal tiles) with extension ".ovr".  If you copy an overview file and name it *.tif, you'll find it loads into ArcGIS.  It should inherently consider zoom level, and only access the required portion of the file, although I am not absolutely certain of this - but the multiple *.ovr files should avoid any issue of reading excessive data volumes to build a view of any given area.

However, for what you're seeking, creating raster tile cache is probably better.  If you're going to convert to cache, you don't need to build the OVR files - the lower resolution tiles are built from the higher resolution below.  Then re: your cache, you said this:

When I load that resultant "raster dataset" into ArcGIS Pro or ArcMap 10.5 or ArcCatalog 10.5, the application is actually still requesting all tile image files at each zoom level, disregarding the display extent.

For this statement, how are you monitoring what ArcGIS is reading?  You drag and drop what appears to be a single file into the map, but the software knows it's a collection of many tiles.  If you have a large tile cache, I promise it is NOT reading all image files that comprise a single zoom level, it's only reading the tiles needed to cover the current extent.  

Cody B

[RebeccaStrauch__GISP]

From the original data source (a single TIF-format raster dataset, projected in epsg:3578), I created a new tile scheme (the "Generate Tile Cache Tiling Scheme" geoprocessing tool) specifying a Storage Format of "Exploded". I then created the tile files using the "Manage Tile Cache" tool, which dutifully output a folder structure containing an image for for every "tile". Promising, I thought.

I may be reading this incorrectly, but it sounds like you are creating a cache, and then reading in the cache files?  If so, this is not the correct process.

You will want to read your "raw" georeferenced files into the mosaic, and let the mosaic do most of the work with the "ovr" files.  

As Cody is mentioning (and who knows much more than I do), the .ovr files should be relatively small, and Desktop should only grab the tiles it needs based on the extent/scale you are viewing.  Desktop can interpolate between scales, if needed....but something like a web app may stick with scales in a cache (if created) and maybe (?) the .ovr files, if not.  Again, I'm a bit fuzzy on that since I'm still learning all this too.

Some good resources, if you haven't found them yet:

• ArcGIS Imagery Workflows | ArcGIS 
• Imagery Workflows—Standard Workflow_Overview | ArcGIS   (there are also more specialize workflows)
•  What is a mosaic dataset?—Help | ArcGIS Desktop 
• What are the functions used by a raster or mosaic dataset?—Help | ArcGIS for Desktop   some fancy, on-the-fly things mosaic datasets can do

[surveyyb]

Hello, I have encountered the same problem as you. How did you finally solve it