AnsweredAssumed Answered

MRF - S3, Mosaics, Caches and Optimization

Question asked by dandrews@globalraymac.ca on Apr 10, 2018
Latest reply on Apr 16, 2018 by pbecker-esristaff

Hi - we regularly use the Optimize Rasters tools to push a lot of raster; orthos and DEM from UAV and LIDAR collection, as MRF files to AWS S3 buckets and mosaic datasets hosted on SDE on an EC2 server, that are then published out as image services on to a number of our web maps.

 

Everything is scripted with Python and runs (mostly smoothly), we have a watcher script sitting on the EC2 server monitoring new uploads to the S3 bucket, that then downloads a local version of the mrf proxy files. I have read recently about other options that may allow us to get away from this and load the mrf directly to the mosaic as a different raster type

 

MRF load to mosaic as table type

@Peter Becker, in one of your user conference presentations :-

http://proceedings.esri.com/library/userconf/proc17/tech-workshops/tw_630-625.pdf 

you mention in the slice "Embedding Raster Proxies into Mosaic Datasets" that there is a script that will convert the raster proxies (I assume this is the .mrf file) into a table to allow adding to the mosaic as a raster data type. Just wondering if this is a publicly available script and i f so where I might find it?

You also mention the ability to embed the proxy into the mosaic, if not found any other references to this, just wondering if this is a supported procedure now? and if you have any further details?

 

I've also been checking out the git hub on raster types in python to see if I can use that python raster type to load the mrf. I've checked this page but can't find much more info as to whether this is possible.

GitHub - Esri/raster-types: A set of Raster Types for ArcGIS developed using the Python language. 

 

Cacheless MRF with S3?

Also - is it possible to get away from having to download a cache of the mrf local to the EC2 with image server and have the app/map load directly from the s3 bucket? (still using a mosaic, not a tiled cache)

 

Cache locks

Fairly regularly we are getting some of the cache tiles locked by the OS and then the image service either blank or black tiles. We then need to remove all the locks on the cache files (stop the services and a python.exe that we think is our synching script) then remove all the cache tiles and restart the services. 

I know this can happen if trying to access the mrf mosaic via desktop and proxy but are is there anything else that could cause this?

 

MRF optimization

Finally any with any general tips for image service optimisation using MRFs and dynamic moasics (NOT tile caching) as we need to render, change mosaic orders, filter, apply functions, etc., to all our services, that would be greatly appreciated. Anything right from mosaic creation, MRF tile and index sizes, DB tuning, etc would be appreciated.

 

Thanks,

Dave

Outcomes