Select to view content in your preferred language

Parallel Processing Factor not helping with Manage Tile Cache processing

1650
5
Jump to solution
08-29-2022 02:29 PM
Labels (1)
MikeSchonlau
Frequent Contributor

I'm running ArcGIS Pro 3.0.1 on an AWS EC2 Windows instance. I'm running the Manage Tile Cache GP tool. All inputs and outputs are being accessed from and written to iops (fast) EBS disks. I'm creating down to Level 22 (141 scale) using the ArcGIS Online tiling scheme. My Parallel Processing Factor is set to "100%". BTW - I've also tried "200%" and "16" which both yielded the same results.

 

Here is the problem - I'm not getting much more than 5-10% CPU utilization. Any thoughts on why?

Thanks, Mike

0 Kudos
1 Solution

Accepted Solutions
MikeSchonlau
Frequent Contributor

Strangely enough, moving to a smaller instance type actually produced much better CPU utilization. Solved

View solution in original post

0 Kudos
5 Replies
MikeSchonlau
Frequent Contributor

Strangely enough, moving to a smaller instance type actually produced much better CPU utilization. Solved

0 Kudos
JasonHrubizna1
Emerging Contributor

Hi Mike, I'm having the same issue. Could you please elaborate on what you mean by "instance type"?

0 Kudos
MikeSchonlau
Frequent Contributor

Hi Jason

 

I was using AWS EC2 Windows "instances" to run tile caches. I don't recall now what specific instance type I ended up using back then, but I recently used a m6i.2xlarge successfully for building caches. I was able to get nearly 100% CPU utilization so it was an effective instance type.

https://aws.amazon.com/ec2/instance-types/

0 Kudos
JasonHrubizna1
Emerging Contributor

Ahh, okay thanks Mike.  I'm just using a regular PC. I experimented with the parallel processing factor and found that setting it to 1 core provided the best performance for building the tile cache. My test job with 10 parallel instances took 3.5 minutes. The same job with 1 instance took 2 minutes.

0 Kudos
PeterKnoop
MVP Regular Contributor

@JasonHrubizna1 If you set the Parallel Processing Factor to a number of processes greater than or equal to the number of system cores on your machine, then it will result in worse performance. Under those circumstances, your are forcing the swapping of a greater number of process across a lesser number of cores, so the overhead slows things done. Hence, when you choose 100% (and greater), you were slowing things down with that overhead (assuming other factors, such as I/O or memory, were not bottlenecks.)

To achieve the maximum performance boost from parallel processing, you should choose a number (or percentage) that results in at least one core being left available for running processes other than the tiling job. Depending on what else you are doing on the computer while the tile job is running, you may even need to leave additional cores available to get maximum performance out of the cores being used for the tiling job.

0 Kudos