Optimal Path As Line - issues with memory leak and Python environment

[ChristaRohrbach]

I am having issues with the OptimalPathAsLine tool in ArcPy as well as in ArcGIS Pro. My ArcPy workflow, which I run in a cloned environment in PyCharm Community Edition, takes point features and a cost surface as input and runs DistanceAccumulation and OptimalPathAsLine iteratively for the annual timesteps 2008 - 2023.

Issue 1: [SOLVED by bugfix in ArcGIS Pro v3.2.2]

It all worked well, until I decided to transform the scale of the cost surface input from a range of ~33.8 - ~930.2 to 1 - 100 using the Rescale by Function tool (linear function) (EDIT: Raster Calculator also tested with same result). After I had changed the cost surface input to my script, OptimalPathAsLine stopped working for two specific timesteps (two specific combinations of input rasters and destination features): It threw no error, but the calculation would never end and memory consumption kept increasing until I stopped the script.

The behaviour is reproducible by manually running the tool with the same inputs in ArcGIS Pro (EDIT: see screenshots). I did not find any difference between the old and new accumulated cost rasters (both showing 32 bit float as pixel type).

Issue 2: [SOLVED by avoiding int64 field type in the input feature class]

Then I updated ArcGIS Pro from 3.1 (or any 3.x, not sure) to 3.2 in an attempt to fix issue 1. Since then, OptimalPathAsLine in my script does not work anymore. It always throws the following error:

arcgisscripting.ExecuteError: Failed to execute. Parameters are not valid.
ERROR 010778: Big Integer field type is currently not supported.
Failed to execute (OptimalPathAsLine).

I have no idea why I get this error message and to which data and field it refers to.

EDIT: I found out that issue 2 has nothing to do with the rasters, but is caused by the destination feature class: The destination field is a Big Integer. My script imports this feature class from a GeoPackage into a FileGDB using CopyFeatures_management. All integers in the GeoPackage are saved as int64. Since the update to ArcGIS Pro 3.2, these fields are, not surprisingly, imported as Big Integer. I am not sure why the script worked earlier - if CopyFeatures converted int64 to something supported by ArcGIS, or if the GeoPackage changed. Anyway, issue 2 is solved by saving int32 to the GeoPackage.

[DavidPike]

Could it just be a field type change from when you rescaled by function?  Have you checked the field types and precision/scale of your inputs?

[ChristaRohrbach]

I compared the pixel type in the properties, all inputs (cost surfaces and accumulated cost surfaces) have 32bit float, and I ran Get Raster Properties to check "cell value type", which also returned "floating point" for all inputs. Is there anything else I can check?

[DanPatterson]

Rescale by Function (Spatial Analyst)—ArcGIS Pro | Documentation 

has a number of warnings regarding raster statistics and potential environment settings that should/could be set.  I would examine those regarding the reclass by function, which may also lead to the invalid field type

[ChristaRohrbach]

Thank you for the link. I read the raster statistics can be calculated ad-hoc by pressing the button in the tool (which is what I did), but I do not understand potential issues caused by this tool. To exclude it as a potential root cause, I tested re-scaling the values using Raster Calculator and recreating an affected accumulated cost raster and back direction raster from it (manually in ArcGIS Pro).

But still, specific combinations of inputs lead to the endless cell tracing and memory leak. For example, acc.cost surface 2013 + back dir. raster 2013 + destination points 2014. If I swap the back dir. raster with the one from 2016 (but e.g. 2011 will not work neither) or swap the destination points with the ones from 2015, the tool runs successfully.

[DanPatterson]

It is these comments in the link

You may not have valid statistics for the following conditions:

The environment processing extent, cell size, or snap raster is set prior to launching the tool dialog box.
The input raster does not have statistics.

It would be prudent to ensure that you have these set in the Environments tab for the tool.

I would expect that the extent, cell size would be the same for the inputs and that the snap raster would be common about them to avoid the unnecessary work of aligning cells.

Various topics related to these concerns can be found on the left of the help pane beginning with

Cell Alignment (Environment setting)—ArcGIS Pro | Documentation

[ElizabethGraham]

Hello,

Thank you for posting this issue.  I am a Product Engineer on the Spatial Analyst team and would like to help investigate the problem. Please email me directly, hopefully, you can share a reproducible case and at least a subset of your data.  

Thanks, 

Liz Graham (egraham@esri.com)

[ChristaRohrbach]

Hi Liz,

I just sent you an e-mail with some data, thanks!

Christa

[EugenioArima]

I am also experiencing computational inefficiencies with both the new "Distance Accumulation" function and "Optimal path". One distance accumulation calculation on a 3300x3000 raster took 12 1/2 hours to complete using one single-point origin. The optimal path function is soaking 29 GB of my 32 RAM memory and is also taking forever to compute. Something is not right with these new functions. A request: could you convey to your team to NOT DEPRECATE the former cost accumulation function based on the regular Dijkstra algorithm? We should have the option to choose which accumulation algorithm to use.

[EugenioArima]

Update: the optimal path failed to compute. Returned an error "Unable to allocate memory" It seems that the function is trying to trace paths from all cells to the origin instead of focusing only on the destination point back to the origin.