nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Fastest tile creation - CPU usage near 100% during the tile creation

MarkMindlin

Hi,

We use the tool "Manage Tile Cache" in ArcMap 10.5, for ArcGIS Server 10.3.

Parameter "area_of_interest" is the feature class created by CacheWorx "Coverage To Feature" Level 19

The help says: "For the fastest tile creation, your CPU should be working near 100% during the tile creation"

Having 16 cores CPU, and using ParallelProcessingFactor 100% or 16

the CPU usage is very low - between 3 to 8 percents

Any help will be greatly appreciated.

Encontre mais publicações marcadas com

Accepted answers

MarkMindlin

We figured out the one reason: area of interest was the feature class with number of polygons more than 1.

After we have dissolved the feature class it seems better: now, 20-30% of CPU usage (still not 90-100 )

Anthony, thank you for your response. We use 10.5, and 16 ArcSOCP processes for 16 core CPU are present.

Another indication of the "proper" use of the tool is number of temporary *.lock" files in the cache folder.

All comments

AnthonyMikelson

I just upgraded from 10.4.1 to 10.5 and it appears the Manage Tile Cache geoprocessing tool is not respecting the multithreading like it is supposed to. In 10.4.1 if you had parallel processing set to 100% then the amount of ArcSOCP should match the number of cores.

Since upgrading to 10.5 I do not see any ArcSOCP processes at all and the caching appears to be much slower.

MarkMindlin

We figured out the one reason: area of interest was the feature class with number of polygons more than 1.

After we have dissolved the feature class it seems better: now, 20-30% of CPU usage (still not 90-100 )

Anthony, thank you for your response. We use 10.5, and 16 ArcSOCP processes for 16 core CPU are present.

Another indication of the "proper" use of the tool is number of temporary *.lock" files in the cache folder.

AnthonyMikelson

After further research I had to eventually uninstall the Desktop Background Geoprocessing. Not sure why yet, but it is working now.

MarkMindlin

We also uninstalled Desktop Background Geoprocessing.

MarkMindlin

In order to find a workaround I tried to use a script with multiprocessing.

Unfortunately, it has an error at the run time. Please help to make it working.

The script

#run “Manage Tile Cache” tool in parallel processing

import arcpy

import multiprocessing

import os

import glob

import sys

import traceback

from multiprocessing import Process, Queue, Pool, \

cpu_count, current_process, Manager

arcpy.env.overwriteOutput = True

arcpy.env.scratchWorkspace = "in_memory"

AAA_Imagery = "D:/gisdata/imagery/public_AAA_Imagery/AAA_Imagery"

Manage_Mode = "RECREATE_ALL_TILES"

Scales__Pixel_Size___Estimated_Disk_Space_ = "564.248588"

Best_WebM = "D:/gisdata/imagery/BestMosaic_2017test.gdb/Best_WebM"

Level_20_coverage = "D:/gisdata/imagery/Levels_coverage.gdb/Level_20_coverage"

def execute_task(bundleAOI):

try:

result = arcpy.ManageTileCache_management(AAA_Imagery, Manage_Mode, "", Best_WebM, "ARCGISONLINE_SCHEME", "", Scales__Pixel_Size___Estimated_Disk_Space_, bundleAOI, "", "591657527.591555", "282.124294")

print(arcpy.GetMessages())

print "result: " + result

except Exception, e:

e.traceback = traceback.format_exc()

raise

if __name__ == '__main__':

#get individual bundles, add it to a dictonary

bundles = {}

count = 1

for bundle in arcpy.da.SearchCursor(Level_20_coverage,["*"]):

bundles[count] = bundle

count += 1

# create a process pool and pass dictonary of extent to execute task

pool = Pool(processes=cpu_count())

pool.map(execute_task, bundles.items())

pool.close()

pool.join()

The error (see the image)

Participe da Conversa

Membros conectados podem postar, seguir atualizações e mais. Novo aqui? Registre uma conta gratuita.

Links Rápidos

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.