Terminating a multiprocessing pool gracefully in a toolbox tool

207
0
10-21-2019 09:08 AM
JonathonHurley
New Contributor

ArcCatalog 10.6

I made a toolbox script that uses multiprocessing. The tool runs fine if left to complete but if the cancelled button is pressed I can't get it terminate the pool. I am fairly new to python and arcpy, but I did find the env.autoCancelling and env.isCancelled and have tried that but not very successfully.

simplified example (not actual code)

import multiprocessing

import arcpy

arcpy.env.autoCancelling = False

NUM_FINISHED = 0

FINISHED = False

def on_complete(results)

   global NUM_FINISHED

   global FINISHED

   if results:

      NUM_FINISHED += 1

      if NUM_FINISHED == 10

         FINISHED = True

def check_processes(pool)

   global FINISHED

    while True:

         time.sleep(10)

         if arcpy.env.isCancelled:

               pool.terminate()

         if FINISHED:

               return

def processes_stuff()

   ''' do stuff here'''

def main(peram_one, peram_two)

   pool = multiprocessing.pool(10)

   for i in range(10):

      pool.apply_async(process_stuff, args=(peram_one, peram_two), callback=on_complete)

      pool.close()

      check_processes(pool)

      ...

if __name__=='__main__':

   in_feature = arcpy.GetPerameterAsText(0)

   to_buffer = arcpy.GetPerameter(1)

   main(in_feature, to_buffer)

   arcpy.SetPerameterasText(2, in_feature)

When the cancel button is pressed, the processes are sill running and never cancel and the cpu stays at 100% even though they become completely detached from ArcCatalog and the GUI from ArcCatalog shows that it was cancelled. What am I missing?

0 Kudos
0 Replies