Intersect Intermittently Produces Empty Output (run in Python script)

DarrenConly — Sat, 11 Dec 2021 10:42:31 GMT

Hi All,

I have a function (copied/pasted below) that does the following:

Inputs are a user-drawn line and a polygon layer of community types. The goal is to determine which polygon has the most overlap with the line. E.g., if a 1mi line has 0.3mi in community type A, 0.5mi in community type B, and 0.2mi in community type C, the function would return “community type B” as the value because B has the longest overlap distance with the line.
To do this, the function does an Intersect (highlighted in the code snippet) between the line and the polygons, returning a temporary line feature class in which each feature is the piece of the line in each community type. If the line was only in one community type, this temporary feature class would only have 1 feature.
Run a search cursor on the temp feature class, returning a dictionary whose items are {community type1: overlap distance for community type 1, community type 2: overlap distance for community type2, etc.}
Return the dictionary key (i.e., community type) whose value is the largest among the dict values, i.e., return the name of the community type as a string value for the community type polygon that has the most overlap with the line.

Based on some testing, the Intersect_analysis step intermittently (every 10-20 times the tool is run) returns an empty temporary feature class (doing a GetCount of it returns 0 features). If the temp feature class is empty, then there is no “max” overlap value, so in turn function returns an error.

def get_proj_ctype(in_project_fc, commtypes_fc):
    '''Get project community type, based on which community type has most spatial overlap with project'''
    temp_intersect_fc = os.path.join(arcpy.env.scratchGDB, 'temp_intersect_fc') # don't use "memory", use scratch GDB
    arcpy.Intersect_analysis([in_project_fc, commtypes_fc], temp_intersect_fc, "ALL", 
                             0, "LINE")
    
    item_cnt = arcpy.GetCount_management(temp_intersect_fc)[0]
    arcpy.AddMessage("Project segments after intersecting with comm types: {}".format(item_cnt))
    len_field = 'SHAPE@LENGTH'
    fields = ['OBJECTID', len_field, params.col_ctype]
    ctype_dist_dict = {}
    
    with arcpy.da.SearchCursor(temp_intersect_fc, fields) as cur:
        for row in cur:
            ctype = row[fields.index(params.col_ctype)]
            seg_len = row[fields.index(len_field)]
            
            if ctype_dist_dict.get(ctype) is None:
                ctype_dist_dict[ctype] = seg_len
            else:
                ctype_dist_dict[ctype] += seg_len
    try:
        maxval = max([v for k, v in ctype_dist_dict.items()])
        proj_ctype = [k for k, v in ctype_dist_dict.items() if v == maxval][0]
 
        return proj_ctype
    except:
        raise ValueError("ERROR: No Community Type identified for project.")‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Any thoughts as to what's going on?

Further information:

This is on a web tool hosted on ArcServer 10.7.1 and published by me via ArcGIS Pro interface
The community type polygon feature class is stored in an enterprise geodatabase
The line feature, as mentioned above, is drawn by the tool user as a tool input in a web browser interface.
To repeat, this issue only arises every 10-20 (i.e., 90-95% of the time no problems) runs and I can find no pattern in terms of what causes it to arise.
Tried already
- Searching for other GeoNet posts on this issue, most are for problems that are either in the desktop interface (not in python script, or they are for a very consistent problem. But none for an intermittent problem like mine.

Re: Intersect Intermittently Produces Empty Output (run in Python script)

DuncanHornby — Wed, 08 Apr 2020 10:22:10 GMT

Got to ask the obvious question, when someone draws that line is it possible they have drawn a line that does not intersect any of your polygons? May be they are looking at the data at a zoomed out scale and what they think they are drawing on is actually missing the polygons?

Re: Intersect Intermittently Produces Empty Output (run in Python script)

DarrenConly — Thu, 09 Apr 2020 20:01:13 GMT

Good Question, but no, in every test done, the line has been drawn within at least one of the polygons.

Re: Intersect Intermittently Produces Empty Output (run in Python script)

DuncanHornby — Fri, 10 Apr 2020 10:51:04 GMT

OK one thing your function should be doing but it is not is to test if the input parameters actually have any data to intersect. I assume in_project_fc is the line layer? I would check that actually has data in it. If you find it does not then this would indicate the issue is outside this function and at that point nobody can help you as we are not seeing the full code.

On a side note you should always use the syntax highlighter when adding code as it makes it easier to read and in the case of python helps identify indentation issues. You can ALWAYS amend your original question...

topic Re: Intersect Intermittently Produces Empty Output (run in Python script) in Geoprocessing Questions

Intersect Intermittently Produces Empty Output (run in Python script)

Re: Intersect Intermittently Produces Empty Output (run in Python script)

Re: Intersect Intermittently Produces Empty Output (run in Python script)

Re: Intersect Intermittently Produces Empty Output (run in Python script)