Low performance of Background Geoprocessing (64-bit) - ArcMap version 10.5

[MarkMindlin]

We have an issue with a performance after installing the product.

The usage of CPU-s is very low in comparison with doing the same task without the product.

The task: Manage Tile Cache geoprocessing.

CPU cores: 16

ArcGIS version: 10.5 ArcMap (I've noticed the same on 10.3.1)

Parallel Processing Factor: 100%

Before the install it showed 100% CPU usage, but after – only 5 -10%.

I checked the “Geoprocessing Options” -> Background-> Enabled 

Are there other settings to use?

[DanPatterson_Retired]

are you assessing things in the windows task manager in the processes tab as referenced in this help topic?

[MarkMindlin]

Yes, I do

Background processing actually starts two processes. You can observe the background processes through Windows Task Manager. On the Processes tab, you'll see the two RuntimeLocalServer.exe processes that make up background processing. These processes should never be ended through the task manager as this could produce unexpected results.

The issue is I see the CPU usage is very low (5-10%). I expect to have 90-100 of CPU and Memory usage increased.

[DanPatterson_Retired]

Ok... a bit confused then.  Without background 64 bit geoprocessing, ArcMap is restricted to whatever is left over of 4Gigs of ram.  With 64 bit enabled you can utilize more... so relative to the task's previous requirements and current allocation is your usage relative to the smaller limit or your installed base?

[MarkMindlin]

It is smaller to installed base.

I have installed 32Gigs of ram - all the usage when I run the GP only 5-6Gigs.

But my main question is how to make all cores working?

[curtvprice]

Without background 64 bit geoprocessing, ArcMap is restricted to whatever is left over of 4Gigs of ram. With 64 bit enabled you can utilize more

ArcMap can use up to 4G (really 3.2 I think). Standard ArcMap background can use its OWN 4G in additional the foreground 4G. If you install BG, it can use basically unlimited RAM (2^64) in theory.  BUT in addition to that: tools that support parallel processing can use additional RAM allocations (4G each subprocess in x32, 2^64 each subprocesses in x64). 

I think the issue Mark has run into is that this particular tool doesn't use multiprocessing very efficiently when run in x64.

Mark, you could ask tech support about this, or just run the tool in the foreground. This sounds like a tool you may want to run in batch with a python script, if you did that you'd probably want to do it using x32 arcpy (C:\Python27\Desktop10.5\python.exe) based on your experience.

[curtvprice]

Whether more than one processor gets used has to do with the way the tool is coded, and has nothing to with background processing. The real question is whether you get any performance benefit with BG processing turned on. If not, disable it.

Background processing does not affect multiprocessing, it just offloads the tool run into the background so the ArcMap interface is not locked up. It also frees up the process to run in x64 which allows it to access more RAM, which may increase performance (or stability) depending on the tool. BG processing is not a way to speed things up using more processes.

Whether background processing gives you any performance increase is entirely tool (and sometimes data) dependent. I usually keep it turned off unless I have a good reason to access > 4G RAM, for example a large vector overlay. If it isn't needed, it actually makes things slower because of the overhead of cranking up BG processing (which is really a 'baby ArcGIS Server' process). BG processing also has many limitations that can limit its usefulness.

Background Geoprocessing (64-bit)—Help | ArcGIS Desktop 

Very few tools take advantage of multiple cores, but more are added as time goes on. These are noted in the help, these are the tools that are listed as observing the parallelProcessingFactor environment (for example Zonal Statistics). If you really want to take advantage of all your cores, you should use ArcGIS Pro, which designed from the ground up as a native multi-threaded app (which ArcMap and BG processing are decidedly not).

FAQ: Do all geoprocessing tools support Parallel Processing? 

[MarkMindlin]

Thank you for your response Curtis,

The task is Manage Tile Cache geoprocessing. I think it is memory consuming.

As I see from the test of the tool (64-bit Background processing)  - it DOES affect multiprocessing. And affect negatively.

So, the only thing is uninstall?

[curtvprice]

Or disable background processing -- no need to uninstall it, you may want it available for another workflow.  Remember the important thing isn't system metrics but actual throughput by the clock, or in your workflow.

[DanPatterson_Retired]

there are comments in the help regarding the support for parallel processing and the number of cores it will use if not set

http://desktop.arcgis.com/en/arcmap/latest/tools/data-management-toolbox/manage-tile-cache.htm