Terminating a multiprocessing pool gracefully in a toolbox tool

JonathonHurley — Mon, 21 Oct 2019 16:08:42 GMT

ArcCatalog 10.6

I made a toolbox script that uses multiprocessing. The tool runs fine if left to complete but if the cancelled button is pressed I can't get it terminate the pool. I am fairly new to python and arcpy, but I did find the env.autoCancelling and env.isCancelled and have tried that but not very successfully.

simplified example (not actual code)

import multiprocessing

import arcpy

arcpy.env.autoCancelling = False

NUM_FINISHED = 0

FINISHED = False

def on_complete(results)

global NUM_FINISHED

global FINISHED

if results:

NUM_FINISHED += 1

if NUM_FINISHED == 10

FINISHED = True

def check_processes(pool)

global FINISHED

while True:

time.sleep(10)

if arcpy.env.isCancelled:

pool.terminate()

if FINISHED:

return

def processes_stuff()

''' do stuff here'''

def main(peram_one, peram_two)

pool = multiprocessing.pool(10)

for i in range(10):

pool.apply_async(process_stuff, args=(peram_one, peram_two), callback=on_complete)

pool.close()

check_processes(pool)

...

if __name__=='__main__':

in_feature = arcpy.GetPerameterAsText(0)

to_buffer = arcpy.GetPerameter(1)

main(in_feature, to_buffer)

arcpy.SetPerameterasText(2, in_feature)

When the cancel button is pressed, the processes are sill running and never cancel and the cpu stays at 100% even though they become completely detached from ArcCatalog and the GUI from ArcCatalog shows that it was cancelled. What am I missing?

topic Terminating a multiprocessing pool gracefully in a toolbox tool in Python Questions

Terminating a multiprocessing pool gracefully in a toolbox tool