Multiprocessing won't create feature classes

[Jay_Gregory]

I'm trying to implement a fairly straightforward multiprocessing script that takes every polygon in a shapefile, tessellates into a grid, takes the center points of each grid square, and then clips the points by the original feature (since the tessellation generates a minimum bounding rectangle).  The tessellation feature class is never created so next step (FeatureToPoint) fails.  I don't understand - can someone help?

Even if I comment out all the arcpy commands except GenerateTessellation, the output is never actually created.  

import multiprocessing
import os
import time
import arcpy

startTime = time.time()

basedir = r"C:\Users\Jay\Documents\ArcGIS\Projects\DataPrep"
scratchGDB = 'C:\\Jay\\data\data.gdb'
output = os.path.join(basedir, "grids.gdb")
features = os.path.join(basedir,"features.shp")

def multifunction(ft):
    import arcpy
    name = ft[0]
    extent = ft[1].extent
    print('Working on {}'.format(name))
    fl = arcpy.management.MakeFeatureLayer(features, "{}fl".format(name), where_clause=f"Name='{name}'")
    gdb = os.path.join(basedir, "{}.gdb".format(name))
    arcpy.management.CreateFileGDB(basedir, "{}.gdb".format(name))
    output=os.path.join(gdb, "{}grid".format(name))
    grid = arcpy.management.GenerateTessellation(output, Extent=extent, Shape_Type="SQUARE", Size="900 SquareMeters")
    print('tellessation')
    pointgrid = arcpy.management.FeatureToPoint(grid, os.path.join(scratchGDB, "{}pointgrid".format(apt)))
    arcpy.analysis.Clip(pointgrid, fl, os.path.join(output, f"{apt}Points"))
    arcpy.management.Delete(fl)
    arcpy.management.Delete(grid)
    arcpy.management.Delete(pointgrid)

def main():
    processList = [feature for feature in arcpy.da.SearchCursor(features, ['Name', 'SHAPE@'])]
    pool = multiprocessing.Pool(1)
    pool.map(multifunction, processList[0:10])
    pool.close()
    pool.join()

if __name__ == "__main__":
    print("Running")
    main()
    executionTime = (time.time() - startTime)
    print('Execution time in seconds: ' + str(executionTime))

[Jay_Gregory]

Thanks!  I wonder if they'd take on this bug.  The line executes, just not when implemented with multiprocessing.  Might be out of scope for them.  

[Anonymous User]

Give it a shot.  The worst they can say is no and maybe update the documentation to let folks know that it won't work in a multiprocessing process.

[JoshuaBixby]

@Jay_Gregory, I ran your code as-is (except changing output location), and it runs fine on my machine with Pro 2.9.2.  The result is an 11MB or so shape file.

[Jay_Gregory]

@JoshuaBixby So the multiprocessing code worked for you?  Would you mind posting the exact code that you ran (there is lots of code I posted in this thread so curious what exactly you ran).  I'm curious what would be different that would have the multiprocessing work for GenerateTessellation.  

Thank you!

[JoshuaBixby]

Here is your code I am just copying and pasting back
--------------------------------------------------------------------------------

import multiprocessing
from processRun import multifunction

def main():
    pool = multiprocessing.Pool(processes=4)
    pool.map(multifunction, range(0,1))
    pool.close()
    pool.join()

if __name__ == "__main__":
    main()

My processRun file looks like:

import arcpy
def multifunction(num):
    arcpy.management.GenerateTessellation("C:\\Users\\jay\\Documents\\test.shp", "-86.809667396 33.5218054870001 -86.7009855469999 33.6051117490001", "SQUARE", "900 SquareMeters", 'GEOGCS["GCS_WGS_1984",DATUM["D_WGS_1984",SPHEROID["WGS_1984",6378137.0,298.257223563]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]];-400 -400 1000000000;-100000 10000;-100000 10000;8.98315284119521E-09;0.001;0.001;IsHighPrecision')

:

[Jay_Gregory]

Okay, well works for you, doesn't work for myself or Jeff.  I wonder what the differences between our environments are that might be causing this.  

I'm on arcpy 2.9 build py37_arcgispro_32704.  Python 3.7.11  Windows 10.  Conda env arcgispro-py3.  I'm guess though it's probably not worth the effort to really figure out what's happening here and why it's working for some folks and not others.