geoprocessing describe geometry object object

[ZacharyUhlmann1]

Hi,

I am developing (slowly) workflows to script geoprocessing tasks using Python and Arcpy (along with the other common suspects - pandas, numpy, etc.).  I want to operate using the 'in_memory' workspace OR (situation-dependent) use `arcpy.da.SearchCursor...` to access rows and return objects, perform geoprocessing tasks like merge, union, append, etc. to avoid creating intermediate data. 

ArcMap 10.6.1

Yesterday I created a function that basically does this:

def extract_cursor(fcs_list, target_field_list, target_val_list, fname):
    for i in range(len(fcs_list)):
        fcs = fcs_list[i]
        target_field = target_field_list[i]
        target_val = target_val_list[i]
        with arcpy.da.SearchCursor(fcs, [target_field, 'SHAPE@']) as cursor:
                for row in cursor:
                    # select rows with matching vals
                    if target_val in row:
                        # initiate new feature
                        if 'shapes_union' not in locals():
                            # get geometry object - Polyline in my case
                            shapes_union = row[1]
                        else:
                            # merge new geometry objects to build one containing 
                            # all features/rows matching selection criteria
                            shapes_union = shapes_union.union(row[1])
    arcpy.CopyFeatures_management(shapes_union, fname)

Then I run the function as such (pretend it's housed here: 'path/to/zachs_arc_scripts/zachs_geo_script.py')

import sys
sys.path.append('path/to/zachs_arc_scripts')
from zachs_geo_script import extract_cursor

fcs_in ['kentucky','missouri','kansas']
target_field = ['rivers','streams','rivers']
target_val = ['rural','rural','seasonal']
fname = 'merged_rivers_ky_mo_ks'
extract_cursor(fcs_in, target_field, target_val, fname)

To be honest I originally had this functioning the exact same nested in a class, and instead of a list I saved variables in a csv.  Then passed the csv and extracted the list of variables via a pandas dataframe.  But I figured this was easier to see for feedback purposes.

PROBLEM!  the feature class merged_rivers_ky_mo_ks will be created, but it contains just one val or is empty.  So to troubleshoot I decomposed the function and ran it directly in the Python Window and it works!  The problem is that when run in the Python Window the geometry object is this:

row[1]

<Polyline object at xxxx>

but within the function run through the Window (as described above) - I get this:

row[1]

>>>geoprocessing describe geometry object object at xxxx>

Furthermore, if I 

return(row[1])

within the function I return the proper Polyline object not the geoprocessing describe geometry...

So the problem is arising from this object type which I can only find sparse info on.

Please help with knowledge on this specific object and issue or offer suggestions for alternate workflows.  I really want to get some decent geoprocessing scripts going, but I keep running into inexplicable issues like this.  Thanks!  Zach

[JoshuaBixby]

I can't even get past your first line of code in extract_cursor function:

>>> extract_cursor(fcs_in, target_field, target_val, fname)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in extract_cursor
TypeError: 'int' object is not iterable

I get that error because of:

for i in len(fcs_list):

I understand you are posting sample code, but it is hard to dive into sample code when it isn't functional code.

[ZacharyUhlmann1]

Whoops!  I always forget the "range" in  what should read: 

range(len(list)):

FYI I spent the last 30 min + checking code, rerunning with different data and then sharing some updates.  Gist is it works but it doesn't.  Needs to be tested on feature classes with more than one matching row.  Using for instance two feature classes as inputs with one row a piece that matches the target_val will work.  With feature classes of multiple rows that match - only one row's geometry will be present in the output feature class containing the single row from the union operation.  Same deal with a <geoprocessing describe geometry object xxxx> as the likely culprit.  Apologies on not sharing tables and code, ESRI forum had problems with my draft post due to "invalid HTML" that I could not solve.  I originally tried sharing an attribute table for each of the examples I just described by inserting tables into this post - didn't work.  I can attach image if that would be useful (?).  Thanks for the help.

[DanPatterson]

If I am reading this correctly...  you are using union and you may get more than one record from each table matching the conditions... if one from each is matched, things work as planned, if more than one from each is matched things dont.

union is a one-by-one incremental process until all geometries are unioned, so it is a

for each in ....:

    for each in ...:

        union stuff here

Probably muddled it more than needed but it helps explain why you get what you are getting.

[JohannesLindner · Accepted solution]

I could reproduce the different object types when manually running the commands in the python window (<Polygon object at ...>) vs running the function (<geoprocessing describe geometry object at ...>).

I could not reproduce your problem (empty output).

You also said one value in the output was an error, but that should be the expected outcome when merging all found features, right?

I edited your code, as I couldn't use yours with my data structure and multiple features. Maybe it works better for you.

# fcs_list: list of paths to the feature classes
# where_clauses: list of where clauses used to select only certain entries from a feature class ("SELECT * FROM feature_class WHERE ...")
# fname: path to the output feature class


def extract_cursor(fcs_list, where_clauses, fname):
    # ensure correct input
##    if len(fcs_list) != len(where_clauses):
##        raise ValueError("fcs_list and where_clauses have to be the same length!")
##    desc_list = [arcpy.da.Describe(fc) for fc in fcs_list]
##    if not all([desc["dataType"] == "FeatureClass" for desc in desc_list]):
##        raise ValueError("All elements of fcs_list have to be feature classes!")
##    if len(set([desc["shapeType"] for desc in desc_list])) > 1:
##        raise ValueError("All elements of fcs_list have to be of the same geometry type!")

    # extract all applicable shapes from all feature classes
    shapes = []
    for fc, where in zip(fcs_list, where_clauses):
        shapes += [row[0] for row in arcpy.da.SearchCursor(fc, ["SHAPE@"], where)]

    # merge the shapes
    merged_shape = shapes[0]
    for s in shapes:
##        print(s)  # <-- this prints <geoprocessing describe geometry object at ...>
        merged_shape = merged_shape.union(s)

    # save the merged shape
    arcpy.management.CopyFeatures(merged_shape, fname)

You can call it like this:

import sys
sys.path.append('path/to/zachs_arc_scripts')
from zachs_geo_script import extract_cursor

fcs_in ['kentucky','missouri','kansas']
where_clauses = ["rivers = 'rural'", "streams = 'rural'", "rivers = 'seasonal'"]
# you can also write more complex queries:
# "rivers IN ('rural', 'seasonal')"
# "TextField = 'text' OR IntField = 5"
# "rivers = 'rural' AND river_length > 9000"
fname = 'merged_rivers_ky_mo_ks'
extract_cursor(fcs_in, where_clauses, fname)

[ZacharyUhlmann1]

Thanks Yohannes, Joshua and Dan.  Apologies on delay - I shelved it for a bit while engaged in another task.  Yohannes your solution worked and this syntax is a neat trick (!) I've never seen.  Honestly didn't realize that adding lists basically merged/dissolved them AND in the context of a list comprehension added another dimension.

shapes += [row[0] for row in arcpy.da.SearchCursor(fc, ["SHAPE@"], where)]

I also tried my code again on different feature classes, and it worked this time.  I'll have to revisit both my version and Yohannes' version once I have actual data to work with.  

Thanks Dan for the background on Union - I was not aware of the nuts an bolts.

Zach