I have a number of Python/arcpy applications that use multiprocessing. Not all arcpy commands like to be multiprocessed but some do work. For example, I can create multiple processes that all "append" into the same SQLServer table from different sources simultaneously. With ArcGIS Pro v3.3.2 I am finding Python /arcpy applications that multiprocess that used work no longer do. The applications worked with v3.2.3.
Is anybody else running into such issues?
Test code below to demonstrate issue. I have installed ArcGIS Pro v3.3.2. The results can vary.
- Run as is. The results should produce "<-- did something ..." statement 5 times. This should work.
- Uncomment line #2 (from arcpy import Exists). Run app. I start to see issues here in that I no longer get 5 statements. Other errors can appear.
- Uncomment line #3 (time.sleep(15)). Run app. Maybe this line helps sometimes\?
- Uncomment line #9 (print(Exists("c:\dev\test-upgrade")). Run app. Errors.
- Uncomment line #24 (# time.sleep(10)). Run app. In the past, I have used delays to get the new processes to spawn properly.
---------- ---------- ---------- ----------
import time
# from arcpy import Exists
# time.sleep(15)
from multiprocessing import Process
import random
def do_something(title):
print(" ---> do something", title)
# print(Exists("c:\work\a-file-name"))
time.sleep(random.randint(5, 15))
print(" <--- did something", title)
def main():
to_process = [ "Albany", "Boston", "Chicago", "Detroit", "Emporia" ]
procs = []
for item in to_process:
print(" ===> process spawn: ", item)
proc = Process(target = do_something, args = (item, ))
proc.start()
procs.append(proc)
# time.sleep(10)
for proc in procs:
proc.join()
if __name__ == '__main__':
main()
---------- ---------- ---------- ----------
