arcpy.Clip_analysis error (can't find the cause)

[MichaelSouffront]

I updated a script to clip wetlands data per county. Some counties work, but for others the Clip analysis fails.

Here is the code:

import arcpy
import time

# Start time
start = time.time()
print('Prepping wetlands county clipped data')

# Enivornment variables
arcpy.env.overwriteOutput = True
arcpy.env.workspace = 'in_memory'
arcpy.env.outputCoordinateSystem = arcpy.SpatialReference('NAD 1983 UTM Zone 12N')

# Input paths
countyPath = r'Counties'
wetlandsPath = r'UT_Wetlands'

# Output path
outFolder = r'output.gdb'

# Create feature dataset
wetlandsFolder = '{}/Wetlands'.format(outFolder)
if not arcpy.Exists(wetlandsFolder):
  arcpy.CreateFeatureDataset_management(outFolder, 'Wetlands')

arcpy.MakeFeatureLayer_management(countyPath, 'countyLayer')

maxCount = len(idList)
count = 1

countyRows = arcpy.SearchCursor(countyPath)
for countyRow in countyRows:

  county = str(countyRow.NAME).replace(' ', '_')
   
  outFile = '{}/Wetlands/{}_Wetlands'.format(outFolder, county)

  query = '"NAME" = \'' + str(countyRow.NAME) + '\''
  arcpy.SelectLayerByAttribute_management('countyLayer', 'NEW_SELECTION', query)

  print 'Clipping ' + str(count) + ' out of ' + str(maxCount)

  arcpy.Clip_analysis(wetlandsPath, 'countyLayer', outFile)

  arcpy.SelectLayerByAttribute_management('countyLayer', 'CLEAR_SELECTION')

  count += 1

  print 'Working on wetlands for ' + county.replace('_', ' ') + ' county...'

  # Here I create variables

  # Here I add new fields

  # Here I populate the new fields with the created variables

del countyRow, countyRows
   
# Finish
end = time.time()
print '\nFinished, run time', time.strftime('%M:%S', time.localtime(end - start))

The script will work until it gets to SANPETE county, which gives this error:

arcgisscripting.ExecuteError: ERROR 999999: Error executing function.

The table was not found.

The table was not found. [SANPETE_Wetlands]

The table was not found.

The table was not found. [SANPETE_Wetlands]

Invalid Topology [Incomplete void poly.]

Failed to execute (Clip).

I've tracked the error to the clip function. At first I thought there was an error in the syntax, then I thought arcpy was getting confused as to where to look for the created clipped feature because I was saving it in memory, then I thought maybe the wetlands data was just too big, or there is something wrong with the attribute data for the SANPETE county, but it's non of those. I tried skipping SANPETE but then it runs for a while and fails in BEAVER county.

I'm running out of ideas as to what it may be. I tried the clip manually in arcMap and it works.

Any suggestions as to what else to try or do you see anything I'm missing in the script?

Thanks,

Michael

[curtvprice]

This sure looks like a path name / workspace problem.

Are the counties that work ones with shorter names? Is there any other way that the county names for the ones that are failing are different?

Another thing to look out for is that all names need to be unique in a geodatabase (that includes objects inside feature datasets; you can't have "mypoly" and "myfds/mypoly" in the same geodatabase).

[RebeccaStrauch__GISP]

Hi Michael,

Have you tried to manual select on SANPETE to see if it can be selected and a proper polygon?  Since it is saying it is getting invalid topology, I'm assuming IF is actually is select, it has an error. 

IS this the first selection in the list, or do some complete successfully?  I'm assuming some do, so..

Suggestions

-
- select the polygon manually and inspect it.  Look at the attribute to make sure these isn't any special (maybe hidden?) character that prevents it from being select properly.

- Make a copy of the FC you are selecting from and run Fix Geometry  on it, and then try using this file as input.  That is the link to 10.2 help.  If you are using 10.3,  Repair Geometry—Help | ArcGIS for Desktop

If this is the first record (that is, this is the first in the list), then it may something else, but I didn't look close because of the topology error.

I hope this helps

[MichaelSouffront]

The first couple counties clip fine.

Also, I ran the clip manually for the Sanpete county in arcmap and the clip works. I assumed that the geometry was okay because of that, but I'll run the repair geometry on Monday just in case.

[curtvprice]

Just FYI, a really good way to make sure a table name is legal is instead of

county = str(countyRow.NAME).replace(' ', '_')

do this (btw, NAME is already a string field, so no reason to str() it)

county = arcpy.ValidateTableName(countyRow.NAME, wetlandsFolder)

This takes care of any other special cases that come up, for example, other special characters that may be lurking in the name.

[MichaelSouffront]

That's a good suggestion, however it doesn't solve the problem.

The .replace() was in the script before I updated it, I added the str() in case unicode strings were the ones causing the problem. but no.

[curtvprice]

One thing that you are doing there that is a little dicey is you are looping on a cursor based on a feature table and then doing processing and selection on a layer that is pointing to that same feature table. You may have better luck if you copy your data so you aren't looping point at the same dataset you are doing selections on.... I know this should work but I have run into trouble in the past.

Another approach is to create a list of county names and then loop on that:

lyrCounty = arcpy.MakeFeatureLayer_management(countyPath, 'countyLayer')
names = []
with arcpy.da.SearchCursor(lyrCounty, "NAME") as rows:
     for row in rows:
        names.append(row[0])
names = list(set(names)) # unique-ize list ( in case of multi-poly counties)

for name in names:
    where = "NAME = '{}'".format(name)
    arcpy.SelectLayerByAttributes(lyrCounty, "", where)
    ...

[MichaelSouffront]

That's how it was before. I would loop through the counties and save them in a list and then loop through the list separately, I changed it hoping that making it more simple would help me find the error.

A coworker is now working on it to see if he can find the problem.

P.S. it's not the geometry either.

[JoshuaBixby]

Where are you executing this code?  As a standalone script?  In an ArcGIS Desktop interactive Python window?  If the latter, are you running Background Processing?  (Not saying any of this is the issue, but it helps to know the specifics of where/how the code is being run).  Also, what version of ArcGIS are you running?

What happens if you programmatically only pass it SANPETE, i.e., add a WHERE clause to your search cursor to only return SANPETE.  Speaking of cursors, I see you are using the older style search cursor.  Have you tried using a ArcPy Data Access Search Cursor?

[curtvprice]

I agree that the arcpy.da cursors are far better for almost all applications. They are cleaner to deal with (no cursor variables left around to delete) and are 10 to 100x faster.