Near Tool Extremely Slow

[FlightDeck]

Hey all,

I am trying to process a file containing ~2.1 million points using the near tool. However, it is only processing around 20 records per second. After running for 12 hours it slows to around 3 records per second, and after running for near 24 hours it has only processed 300k of the 2.1 million records. Has anyone else experienced this? I've run this tool in the past and seen upwards of 200 records per second (which is still slow), but is this normal? Is there a better way to accomplish the same task?

I've tried clearing the cache with no improvement. I'm running ArcPro 3.2.2 all drivers are up to date, System has 32GB of ram, Intel i7-13700H 5.0GHz, Nvidia GeForce RTX 4050 etc. The system seems more than capable of running this at a much better speed than I am seeing especially when the GPU is seeing 5% utilization. The CPU is only at 22% and really only seems to be utilizing 4 of the 14 cores/20 threads. 

What if anything am I missing?

[RobertCao]

Hi @FlightDeck ,

Where is your data stored? (local file geodatabase or Enterprise sde) And is your project stored on your local drive or shared network drive? Are you working with VM in this case? Any VPN connection or remote desktop involved? Do you experience the same performance issue while running the Near tool against smaller dataset? 

Thanks,

Rob

[FlightDeck]

Hey Robert,

All of the data is stored on, and is processed on, my local machine. The Drive is an NVMe drive that is dedicated to running and storing Arc Pro and its project files.  No VPN or RDP in this instance. I can try a smaller data set once this finishes. 

In the mean time are you aware of any settings to check? It seems like Arc is grossly under-utilizing available resources...

[RobertCao]

Hi @FlightDeck ,

Basically we would like to see if this performance issue is specific to Near tool or this particular large dataset. To test that, you can try another gp tool against this dataset and also test Near tool against another smaller dataset. 

 This is a good resource for troubleshooting performance issue in ArcGIS Pro. 

To improve rendering quality, you can try configuring the Graphic performance preference as shown below:

[FlightDeck]

Thanks Robert,

This setting along with the Nvidia Control Panel settings are and have been in place for a few weeks now. I will say they do seem to help with rendering images but I have not seen any marked improvement when using the analytics tools. 

The data set is point data. Do you have a preference which GP tool I run against it for testing?

After stopping and starting the tool a number of times it appears to be processing around 100 per sec.  and in 3 hours it has processed ~1.5M records. Once complete I can run the PAT tool. 

[FlightDeck]

Per the attached stats the Avg Points analyzed per second for this run was 145. 

[FlightDeck]

I have run the PAT tool is there anything in particular I should be looking for in the logs?