topic Re: Image Service Using MRF Not Displaying Complete Images in ArcGIS Image Server Questions

Image Service Using MRF Not Displaying Complete Images

LorneDmitruk2 — Thu, 15 Aug 2019 15:42:44 GMT

Hi,

I've been testing image services using MRF files stored on Azure accessed via a file geodatabase and raster proxy files. When accessing the service a scale large enough to require accessing the original imagery not all the tiles are being returned, as shown below.

Starting at 1:3000

1:2250

1:2000

1:1250

So depending on the scale and what area one is looking at, the missing tiles come and go. Also, the cache files seem to updating OK.

Here's another interesting effect, when the image return type is requested as JPG, some of the tiles are returning incomplete band information.

So any thoughts about why this happening would be greatly appreciated.

Cheers

Re: Image Service Using MRF Not Displaying Complete Images

AbhijitDoshi — Mon, 19 Aug 2019 11:51:36 GMT

Hi,

can you set the system level environmental variables

GDAL_HTTP_MAX_RETRY = 5

GDAL_HTTP_RETRY_DELAY = 10

once these are set, restart the ArcGIS server and delete the cache,

Regards

Abhijit

Re: Image Service Using MRF Not Displaying Complete Images

LorneDmitruk2 — Mon, 19 Aug 2019 15:01:23 GMT

Hi Abhijit,

With the new environment settings I'm getting the same results as before.

Cheers

Re: Image Service Using MRF Not Displaying Complete Images

NicholasEverdell — Fri, 30 Oct 2020 03:00:27 GMT

Hi,

We are having the exact same issue using AWS (S3, EC2, Raster Proxy & MRF Cache on EBS)

Gordon Sumerling‌
Peter Becker

Vinay Viswambharan‌

Any ideas on how to resolve this issue?

Thanks,

Nic

Re: Image Service Using MRF Not Displaying Complete Images

GordonSumerling — Fri, 30 Oct 2020 03:30:26 GMT

Hello Nic,

In the Mosaic dataset on the defaults tab there is a setting "Maximum number of Rasters Per Mosaic" by default this is set to 20. Recommend increasing this to 200. You should then see all of your rasters

Gordon

Re: Image Service Using MRF Not Displaying Complete Images

PeterBecker — Fri, 30 Oct 2020 04:20:36 GMT

Looks to me like on one of the following: (most probably 2)

1- Looks like the source data may consist of a large number of small tiles. If this is the case check the level at which overviews are created. The smaller the tiles the larger the resolution the overviews need to be. If you dont have sufficient overviews then system will need to open many files which can degrade performance and lead to issue if 'Max number of rasters per mosaic' is not large enough. If the source is many tiles consider mosaicking them together into much larger tiles (same projection/pixel alignment).

2- If the issues is with raster proxies then check Appedix D in the Optimize Rasters documentation. The issue is often related to running desktop and server on the same machine that causes locks on the cached files. This may have caused corruption of the tile cache indices. Delete the cache. Also ensure the cache is on a local drive (preferably a fast ephemeral drive). Do not point the cache to a network drive or some location shared with multiple machines.

Re: Image Service Using MRF Not Displaying Complete Images

NicholasEverdell — Thu, 19 Nov 2020 22:44:31 GMT

Thanks All for the responses, much appreciated.

We have what Peter mentioned and the MRF cache is on a shared folder as we have a multi-machine ArcGIS Server site in AWS, these are in an auto-scaling group and use local service accounts. This meant that when multiple users are using the service at once multiple of these service accounts are trying to Read/Write to the MRF cache at the same time which is causing the read errors.

We moved the service over to a single machine ArcGIS Enterprise build and had no dramas at all.

We also followed Gordon's advice and increased the maximum number of rasters to ensure we don't have any issues in the future either.

Re: Image Service Using MRF Not Displaying Complete Images

harsh9as — Thu, 10 Jun 2021 15:32:47 GMT

Hello

I am encounerting a similar issue where I used raster proxies for TIFFs stored on cloud and loaded these raster proxies in a mosaic dataset > build overviews >build pyramids >calculated statistics. Ensured the changes @GordonSumerling suggested.

Once published as a image service, I am unable to view the rasters beyond 2nd zoom level. The imagery disappears. I assume that first 2 levels of imagery that I see are basically the overviews. I have my source dataset (raster proxy location) registered with ArcGIS server. Pro 2.8 and ArcGIS server 10.8.1 are on different machines. I tried the same process for multiple TIFFs, creating multiple mosaic datasets, without avail.

The mosaic dataset created in Pro works as expected - renders on all given scales. Any guidance would be appriciated. Thank you.

@PeterBecker

@AbhijitDoshi

Re: Image Service Using MRF Not Displaying Complete Images

bspivey — Thu, 27 Oct 2022 11:52:10 GMT

"If the source is many tiles consider mosaicking them together into much larger tiles.."

Could you provide any general recommendations on how large the tiles should/could be for optimum performance? We have 10000x10000 RGB-IR COG tiles statewide (~56k). We were hoping for better performance than what we're experiencing (even if the output is entirely comprised of data already contained in the raster proxies) and wondering if creating fewer, but larger tiles would help the performance by reducing the disk I/O when mosaicking on each server node. We're using the S3, raster proxy, image service configuration.

https://services.gis.nc.gov/secure/rest/services/Imagery/Orthoimagery_Latest/ImageServer

@AbhijitDoshi

Gordon Sumerling‌
Peter Becker

Vinay Viswambharan

Re: Image Service Using MRF Not Displaying Complete Images

bspivey — Thu, 27 Oct 2022 11:55:39 GMT

Quite dated I know, but the last image example (yellow tile) can also be caused by the raster proxy cache directory becoming full.