CalculateGeometryAttributes produces None values in multiprocessing

HieuTran12

I was trying to perform a geo analysis and the function CalculateGeometryAttributes produced None values when I wrapped the script in multiprocessing, but it worked correctly in a for loop.

def process(fc):
        arcpy.env.workspace = "G:/usgrids2020/test1/"
        arcpy.env.overwriteOutput=True
        state = fc[34:-7]
        print(f"Processing for {state.upper()}")
        readTime = datetime.datetime.now()
        # arcpy.env.workspace = os.path.join(gdb_path, f"usa{state}.gdb")
        #Import the feature for boundary and water
        # fc = f"usa{state[3:5]}_joined"
        fcInMem = arcpy.management.CopyFeatures(fc, f"in_memory/usa{state}_fcInMem")
        water_fc = f"{fc[:-7]}_water_layer"
        waterInMem = arcpy.management.CopyFeatures(water_fc, f"in_memory/usa{state}_waterInMem")
        #Create the temporary id for joining fields later
        arcpy.AddField_management(fcInMem,'tempid','LONG')
        arcpy.CalculateField_management(fcInMem,"tempid",'!OBJECTID!','PYTHON')
        #Calculate the area(land+water) by GEODESIC method => Kytt recommended
        arcpy.management.AddField(fcInMem, "GEODESIC_AREA", "FLOAT")
        arcpy.management.CalculateGeometryAttributes(fcInMem, "GEODESIC_AREA AREA_GEODESIC", "", "SQUARE_METERS")

This is what it generated:

(None, 'USA_100030117003015')
(None, 'USA_100030127001011')
(None, 'USA_100030151002006')
(None, 'USA_100030121001003')
(None, 'USA_100030112051009')
(None, 'USA_100030148102015')
(None, 'USA_100030112062022')
(None, 'USA_100030168063009')
(None, 'USA_100030166141010')
(None, 'USA_100030164041013')

DanPatterson

does the newer "memory" workspace behave differently than the "in_memory" workspace with multiprocessing?

I suspect a for loop would be better if using memory workspaces in any event

... sort of retired...

HieuTran12

I tried without the "in_memory" as well but the result remains the same.

JoshuaBixby

Commented at python - ArcPy function produced None values in multiprocessing - GIS SE, it looks to be a defect with CalculateGeometryAttributes. I was able to create a minimum reproducible example and have submitted to Esri Support to log a defect.

HaydenWelch

Since this seems to be a bug as per @JoshuaBixby's reply, you could try using an UpdateCursor to get the area for now:

import arcpy
import datetime

def process(fc):
        arcpy.env.workspace = "G:/usgrids2020/test1/"
        arcpy.env.overwriteOutput=True
        state = fc[34:-7]
        print(f"Processing for {state.upper()}")
        readTime = datetime.datetime.now()
        # arcpy.env.workspace = os.path.join(gdb_path, f"usa{state}.gdb")
        #Import the feature for boundary and water
        # fc = f"usa{state[3:5]}_joined"
        fcInMem = arcpy.management.CopyFeatures(fc, f"in_memory/usa{state}_fcInMem")
        water_fc = f"{fc[:-7]}_water_layer"
        waterInMem = arcpy.management.CopyFeatures(water_fc, f"in_memory/usa{state}_waterInMem")
        #Create the temporary id for joining fields later
        arcpy.AddField_management(fcInMem,'tempid','LONG')
        arcpy.CalculateField_management(fcInMem,"tempid",'!OBJECTID!','PYTHON')
        #Calculate the area(land+water) by GEODESIC method => Kytt recommended
        arcpy.management.AddField(fcInMem, "GEODESIC_AREA", "FLOAT")
        with arcpy.da.UpdateCursor(fcInMem, ["SHAPE@AREA", "GEODESIC_AREA"]) as cursor:
            for row in cursor:
                row = dict(zip(cursor.fields, row))
                row['GEODESIC_AREA'] = row['SHAPE@AREA'] # This is in feature units, so you will likely need the conversion factor from arcpy.LinearUnitConversionFactor
                cursor.updateRow(row.values())
        #arcpy.management.CalculateGeometryAttributes(fcInMem, "GEODESIC_AREA AREA_GEODESIC", "", "SQUARE_METERS")

HieuTran12

I tried your solution, but the results are very far off from the Geodesic method for area calculations.

HaydenWelch

Did you make sure that the feature units are meters? If so you can try this:

import arcpy
import datetime

def process(fc):
        arcpy.env.workspace = "G:/usgrids2020/test1/"
        arcpy.env.overwriteOutput=True
        state = fc[34:-7]
        print(f"Processing for {state.upper()}")
        readTime = datetime.datetime.now()
        # arcpy.env.workspace = os.path.join(gdb_path, f"usa{state}.gdb")
        #Import the feature for boundary and water
        # fc = f"usa{state[3:5]}_joined"
        fcInMem = arcpy.management.CopyFeatures(fc, f"in_memory/usa{state}_fcInMem")
        water_fc = f"{fc[:-7]}_water_layer"
        waterInMem = arcpy.management.CopyFeatures(water_fc, f"in_memory/usa{state}_waterInMem")
        #Create the temporary id for joining fields later
        arcpy.AddField_management(fcInMem,'tempid','LONG')
        arcpy.CalculateField_management(fcInMem,"tempid",'!OBJECTID!','PYTHON')
        #Calculate the area(land+water) by GEODESIC method => Kytt recommended
        arcpy.management.AddField(fcInMem, "GEODESIC_AREA", "FLOAT")
        with arcpy.da.UpdateCursor(fcInMem, ["SHAPE@", "GEODESIC_AREA"]) as cursor:
            for row in cursor:
                row = dict(zip(cursor.fields, row))
                row['GEODESIC_AREA'] = row['SHAPE@'].getArea('GEODESIC', 'SquareMeters')
                cursor.updateRow(row.values())
        #arcpy.management.CalculateGeometryAttributes(fcInMem, "GEODESIC_AREA AREA_GEODESIC", "", "SQUARE_METERS")

Which uses the Polygon.getArea() method. It pulls the whole shape object though so if you're doing this for millions of rows it can slow down a bit, but you'll get an accurate Geodesic area.

HieuTran12

Wow, this could be an alternate method of CalculateGeometryAttributes. Or does it have any tradeoffs?

HaydenWelch

The tradeoff is that you have to use an arcpy cursor object, but I do most of my scripting with cursors. Even wrote some wrapper classes that allow you to use cursor objects safely using python syntax (see my post in Python Ideas).

Cursors are great because they gave you really granular access to the feature data. They are still Python objects though and don't have all the horsepower of the base Arc functions that are implemented in C.

For use cases like this though, and basic CRUD ops on a dataset they're great. Especially if you use the where_clause and spatial_filter parameters properly.