does the in_memory workspace be mutually influenced in every subprocess in multiprocess?

pyfans

I have write a py script with multiprocess, like this:

import arcpy

from multiprocessing import Pool

def worker(fc):

      arcpy.featureclasstofeatureclass(fc,r"in_memory","tempfc")

      ## do something for tempfc

      arcpy.delete_management("in_memory")

if __name__=="__main__":

      p=Pool(3)

      fcs=..... ## get some featureclasses

      p.map(worker,fcs)

      p.close()

      p.join()

 

I want to know if the created "tempfc" and deleted  in_memory workspace will be mutually influenced in  different subprocess?

JoshuaBixby

Processes within a pool don't share memory unless objects are passed as specific multiprocessing data types, e.g., multiprocessing.Array .  Since you are not doing that in your code (I am not even sure any ArcPy objects can be passed that way), the in-memory workspace in each worker process is independent of the others.

An example:

from multiprocessing import Pool
import arcpy
import time

i = 0
def f(x):
    global i
    i += 1
    
    arcpy.CreateTable_management("in_memory", "xyz{}".format(i))
    time.sleep(1)
    
    arcpy.env.workspace = "in_memory"
    return arcpy.ListTables()


if __name__ == '__main__':
    with Pool(4) as p:
        print(p.map(f, range(6)))

Yields:

[['xx1'], ['xx1'], ['xx1'], ['xx1'], ['xx1', 'xx2'], ['xx1', 'xx2']]
>>> 

 

As you can see, each of the four worker processes is able to create the same-named table in its in-memory workspace, and each arcpy.ListTables() only sees the tables in its own in-memory workspace.