Image Service Using MRF Not Displaying Complete Images

[LorneDmitruk2]

Hi,

I've been testing image services using MRF files stored on Azure accessed via a file geodatabase and raster proxy files. When accessing the service a scale large enough to require accessing the original imagery not all the tiles are being returned, as shown below.   

Starting at 1:3000

1:2250

1:2000

1:1250

So depending on the scale and what area one is looking at, the missing tiles come and go. Also, the cache files seem to updating OK.

Here's another interesting effect, when the image return type is requested as JPG, some of the tiles are returning incomplete band information. 

So any thoughts about why this happening would be greatly appreciated.

Cheers

[AbhijitDoshi]

Hi,

can you set the system level environmental variables 

GDAL_HTTP_MAX_RETRY = 5

GDAL_HTTP_RETRY_DELAY = 10

once these are set, restart the ArcGIS server and delete the cache, 

Regards 

Abhijit

[LorneDmitruk2]

Hi Abhijit,

With the new environment settings I'm getting the same results as before.

Cheers

[NicholasEverdell]

Hi,

We are having the exact same issue using AWS (S3, EC2, Raster Proxy & MRF Cache on EBS)

Gordon Sumerling‌
Peter Becker

Vinay Viswambharan‌

Any ideas on how to resolve this issue?

Thanks,

Nic

[GordonSumerling]

Hello Nic,

In the Mosaic dataset on the defaults tab there is a setting "Maximum number of Rasters Per Mosaic" by default this is set to 20. Recommend increasing this to 200. You should then see all of your rasters

Gordon

[PeterBecker]

Looks to me like on one of the following:  (most probably 2)

1- Looks like the source data may consist of a large number of small tiles. If this is the case check the level at which overviews are created. The smaller the tiles the larger the resolution the overviews need to be. If you dont have sufficient overviews then system will need to open many files which can degrade performance and lead to issue if 'Max number of rasters per mosaic' is not large enough. If the source is many tiles consider mosaicking them together into much larger tiles (same projection/pixel alignment).

2- If the issues is with raster proxies then check Appedix D in the Optimize Rasters documentation. The issue is often related to running desktop and server on the same machine that causes locks on the cached files. This may have caused corruption of the tile cache indices. Delete the cache. Also ensure the cache is on a local drive (preferably a fast ephemeral drive). Do not point the cache to a network drive or some location shared with multiple machines.

[NicholasEverdell]

Thanks All for the responses, much appreciated.

We have what Peter mentioned and the MRF cache is on a shared folder as we have a multi-machine ArcGIS Server site in AWS, these are in an auto-scaling group and use local service accounts. This meant that when multiple users are using the service at once multiple of these service accounts are trying to Read/Write to the MRF cache at the same time which is causing the read errors.

We moved the service over to a single machine ArcGIS Enterprise build and had no dramas at all.

We also followed Gordon's advice and increased the maximum number of rasters to ensure we don't have any issues in the future either.

[harsh9as]

Hello

I am encounerting a similar issue where I used raster proxies for TIFFs stored on cloud and loaded these raster proxies in a mosaic dataset > build overviews >build pyramids >calculated statistics. Ensured the changes @GordonSumerling suggested. 

Once published as a image service, I am unable to view the rasters beyond 2nd zoom level. The imagery disappears. I assume that first 2 levels of imagery that I see are basically the overviews. I have my source dataset (raster proxy location) registered with ArcGIS server. Pro 2.8 and ArcGIS server 10.8.1 are on different machines. I tried the same process for multiple TIFFs, creating multiple mosaic datasets, without avail. 

The mosaic dataset created in Pro works as expected - renders on all given scales. Any guidance would be appriciated. Thank you.

@PeterBecker 

@AbhijitDoshi 

[bspivey]

"If the source is many tiles consider mosaicking them together into much larger tiles.."

Could you provide any general recommendations on how large the tiles should/could be for optimum performance? We have 10000x10000 RGB-IR COG tiles statewide (~56k). We were hoping for better performance than what we're experiencing (even if the output is entirely comprised of data already contained in the raster proxies) and wondering if creating fewer, but larger tiles would help the performance by reducing the disk I/O when mosaicking on each server node. We're using the S3, raster proxy, image service configuration. 

https://services.gis.nc.gov/secure/rest/services/Imagery/Orthoimagery_Latest/ImageServer

@AbhijitDoshi

Gordon Sumerling‌
Peter Becker

Vinay Viswambharan

[bspivey]

Quite dated I know, but the last image example (yellow tile) can also be caused by the raster proxy cache directory becoming full.