Tools with parallel processing taking forever

1236
7
11-16-2021 04:54 AM
egordobatovk
New Contributor

Hi all, I've recently encountered some strange problem with parallel geoprocessing in ArcGIS Pro 2.8 & 2.9. I was using ArcHydro Tools for processing DEMs, these tools in turn use standard Spatial Analyst Tools, Hydrology tab especially. Some of these tools, as Sink and Flow direction can use parallel processing for speeding them up.

However, after some point, these tools cannot finish their job and just stuck saying "running", however they are not using any processing power anymore. What is even more interesting, it is that tools are not always stuck at the same point, sometimes (according to messages of the tool) it can succeed some steps at which it was stuck last time, sometimes it stucks at the very first step. And after being stuck you cannot cancel this tool, you have to manually stop the process through the task manager.

Funny thing is that in scratch folder/gdb after the tool being stuck you can see, that the step it was trying to finish was completed sucsessfully, however for some reason tool couldn't proseed further. Btw in ArcMap parallel processing with the same tools and data works just fine.

Reinstalling ArcGIS, cleaning temp folders, choosing shorter paths (like c:\data) does nothing. The only solution that worked for now is just turning off the parallel processing in enviroments, or to process this in ArcMap))) The problem is that from 10 minutes of processing the part of workflow it now takes 30 minutes.

If someone has ideas for potential fixes, or encountered the same problem as myself, please let me know.

7 Replies
DanPatterson
MVP Esteemed Contributor

You have gone through?

Parallel processing with Spatial Analyst—ArcGIS Pro | Documentation

Parallel processing with multiple CPUs—ArcGIS Pro | Documentation

Parallel Processing Factor (Environment setting)—ArcGIS Pro | Documentation

and nothing stands out?  I am assuming your machine isn't short on resources in terms of cpu/gpu


... sort of retired...
0 Kudos
egordobatovk
New Contributor

I have gone through all of this, sure.

The thing is, the same machine (i5 8th gen + GTX1050 + 8gb ram) was ok whith this tools on the same data (on Pro 2.8, 2 months ago), yet something happend during this two months. I wonder it can be something about messy registry, but if it is, it's easier to just reinstall windows.

DanPatterson
MVP Esteemed Contributor

From ArcGIS Pro 2.9 system requirements—ArcGIS Pro | Documentation

run

Check your computer's ability to run ArcGIS Pro 2.9.

I hope you meant 8gb video ram and not just cpu

 


... sort of retired...
0 Kudos
Hemlock_Grove
New Contributor III

I am having this same issue, exactly as you describe. Wondering if you ever found a fix?

DylanMcDougall
New Contributor II

I have had the same experience starting around November, including the part where the same tools worked months earlier with the same machine and Arc Pro installation. The system requirements tool confirms that my system exceeds the requirements for 10.9. Are there any log files to check? Is there something we should send to Support?

0 Kudos
tlopezcantu
Esri Contributor

Hello,

To better understand the source of the problem, could you let us know:

- Which Windows OS version is installed in your machine?

- If you manually set the Parallel Processing Factor (in Environment settings) to 0, does the tool successfully executes? 

Thank you very much,

Tania Lopez-Cantu

Tania Lopez-Cantu
0 Kudos
Hemlock_Grove
New Contributor III

Hi,

I have Windows 11.

If I set Parallel Processing to 0, the tool does indeed work.

0 Kudos