Issue dedicating more hardware resources using image server raster analytics.

[AdamHogg_MNR]

Hello ESRI Community,

We've installed Image Server (10.9.1) with a raster analytics configuration on a Windows VM in Azure (24-core with GPU and 250 GB of RAM). The Image Server has been federated with Portal on another VM. It also has an Azure SQL enterprise geodatabase registered as a data store. Using ArcPro (2.9) we get confirmation that we've submitted our raster function process to portal for distributed processing, so the configuration is working. That said, when we submit a large process using an image service (e.g., roughly 131 billion pixels) the job takes almost 24 hours to complete while our raster processing VM never exceeds 10% CPU or RAM usage. Is there anyway to increase the amount of VM resources used by a job to decrease processing time?

[BillFox]

elevation service cache?

[AdamHogg_MNR]

Thanks for responding so quickly Bill. We're currently using dynamically cached services.

[BillFox]

I think you are here:

Caching tools and server resources for caching

https://enterprise.arcgis.com/en/server/latest/publish-services/windows/arcgis-enterprise-cache-gene...

[BillFox]

not sure what you trying to do

can you explain a bit more?

[AdamHogg_MNR]

Sorry for the delayed response Bill.

We submit a raster function or geoprocessing tool to our Image Server, which is configured for raster analytics with a high powered dedicated processing VM. We set the parallel processing parameter to 90% prior to submitting. When we submit the job ArcPro(or Map Viewer) verifies processing has been distributed by portal but when we check the usage stats on the VM  dedicated for processing CPU and RAM resource use are minimal (never going above 10%). While jobs do finish processing, large jobs (e.g., roughly 150 trillion pixels) take 24 hours or more. Wondering if we can allocate more resources to processing to speed this time up.

[BillFox]

can you confirm you are hitting SSD high speed disks for everything and high speed/bandwidth network for your off premise setup?

[AdamHogg_MNR]

All network speed/bandwidth are governed by Azure: Cloud/VM's. VM network speeds are ~1700 to 1900 mb/s for both download and upload as per internet speed test. VM isn't using close to that after job is submitted. VM temp file's were routed to a high speed SSD on the VM but are now sent directly to cloud. Final crf's are also written to cloud.

[AdamHogg_MNR]

Modifying where we write to doesn't seem to alter utilization much, though there does appear to be an observable read/write lags at times while processing either to the cloud or directly to SSD. We've also tinkered with shared instance pooling, with a slight improvement in utilization by a few %'s.

[BillFox]

Neeraj Rajasekar @NeerajRajasekar 

has some tips in here that might help

https://www.esri.com/arcgis-blog/products/analytics/analytics/multiprocessing-with-arcgis-raster-ana...