topic Re: arcpy.Clip_analysis error (can't find the cause) in Python Questions

arcpy.Clip_analysis error (can't find the cause)

MichaelSouffront — Sun, 12 Dec 2021 05:41:29 GMT

I updated a script to clip wetlands data per county. Some counties work, but for others the Clip analysis fails.

Here is the code:

import arcpy
import time

# Start time
start = time.time()
print('Prepping wetlands county clipped data')

# Enivornment variables
arcpy.env.overwriteOutput = True
arcpy.env.workspace = 'in_memory'
arcpy.env.outputCoordinateSystem = arcpy.SpatialReference('NAD 1983 UTM Zone 12N')

# Input paths
countyPath = r'Counties'
wetlandsPath = r'UT_Wetlands'

# Output path
outFolder = r'output.gdb'

# Create feature dataset
wetlandsFolder = '{}/Wetlands'.format(outFolder)
if not arcpy.Exists(wetlandsFolder):
  arcpy.CreateFeatureDataset_management(outFolder, 'Wetlands')

arcpy.MakeFeatureLayer_management(countyPath, 'countyLayer')

maxCount = len(idList)
count = 1

countyRows = arcpy.SearchCursor(countyPath)
for countyRow in countyRows:

  county = str(countyRow.NAME).replace(' ', '_')
   
  outFile = '{}/Wetlands/{}_Wetlands'.format(outFolder, county)

  query = '"NAME" = \'' + str(countyRow.NAME) + '\''
  arcpy.SelectLayerByAttribute_management('countyLayer', 'NEW_SELECTION', query)

  print 'Clipping ' + str(count) + ' out of ' + str(maxCount)

  arcpy.Clip_analysis(wetlandsPath, 'countyLayer', outFile)

  arcpy.SelectLayerByAttribute_management('countyLayer', 'CLEAR_SELECTION')

  count += 1

  print 'Working on wetlands for ' + county.replace('_', ' ') + ' county...'

  # Here I create variables

  # Here I add new fields

  # Here I populate the new fields with the created variables

del countyRow, countyRows
   
# Finish
end = time.time()
print '\nFinished, run time', time.strftime('%M:%S', time.localtime(end - start))

The script will work until it gets to SANPETE county, which gives this error:

arcgisscripting.ExecuteError: ERROR 999999: Error executing function.

The table was not found.

The table was not found. [SANPETE_Wetlands]

The table was not found.

The table was not found. [SANPETE_Wetlands]

Invalid Topology [Incomplete void poly.]

Failed to execute (Clip).

I've tracked the error to the clip function. At first I thought there was an error in the syntax, then I thought arcpy was getting confused as to where to look for the created clipped feature because I was saving it in memory, then I thought maybe the wetlands data was just too big, or there is something wrong with the attribute data for the SANPETE county, but it's non of those. I tried skipping SANPETE but then it runs for a while and fails in BEAVER county.

I'm running out of ideas as to what it may be. I tried the clip manually in arcMap and it works.

Any suggestions as to what else to try or do you see anything I'm missing in the script?

Thanks,

Michael

Re: arcpy.Clip_analysis error (can't find the cause)

curtvprice — Sat, 02 May 2015 02:09:44 GMT

This sure looks like a path name / workspace problem.

Are the counties that work ones with shorter names? Is there any other way that the county names for the ones that are failing are different?

Another thing to look out for is that all names need to be unique in a geodatabase (that includes objects inside feature datasets; you can't have "mypoly" and "myfds/mypoly" in the same geodatabase).

Re: arcpy.Clip_analysis error (can't find the cause)

RebeccaStrauch__GISP — Sat, 02 May 2015 03:39:44 GMT

Hi Michael,

Have you tried to manual select on SANPETE to see if it can be selected and a proper polygon? Since it is saying it is getting invalid topology, I'm assuming IF is actually is select, it has an error.

IS this the first selection in the list, or do some complete successfully? I'm assuming some do, so..

Suggestions

-
- select the polygon manually and inspect it. Look at the attribute to make sure these isn't any special (maybe hidden?) character that prevents it from being select properly.

- Make a copy of the FC you are selecting from and run Fix Geometry on it, and then try using this file as input. That is the link to 10.2 help. If you are using 10.3, Repair Geometry—Help | ArcGIS for Desktop

If this is the first record (that is, this is the first in the list), then it may something else, but I didn't look close because of the topology error.

I hope this helps

Re: arcpy.Clip_analysis error (can't find the cause)

MichaelSouffront — Sat, 02 May 2015 23:21:08 GMT

The first couple counties clip fine.

Also, I ran the clip manually for the Sanpete county in arcmap and the clip works. I assumed that the geometry was okay because of that, but I'll run the repair geometry on Monday just in case.

Re: arcpy.Clip_analysis error (can't find the cause)

curtvprice — Sun, 03 May 2015 05:31:47 GMT

Just FYI, a really good way to make sure a table name is legal is instead of

county = str(countyRow.NAME).replace(' ', '_')

do this (btw, NAME is already a string field, so no reason to str() it)

county = arcpy.ValidateTableName(countyRow.NAME, wetlandsFolder)

This takes care of any other special cases that come up, for example, other special characters that may be lurking in the name.

Re: arcpy.Clip_analysis error (can't find the cause)

curtvprice — Sun, 12 Dec 2021 05:41:32 GMT

One thing that you are doing there that is a little dicey is you are looping on a cursor based on a feature table and then doing processing and selection on a layer that is pointing to that same feature table. You may have better luck if you copy your data so you aren't looping point at the same dataset you are doing selections on.... I know this should work but I have run into trouble in the past.

Another approach is to create a list of county names and then loop on that:

lyrCounty = arcpy.MakeFeatureLayer_management(countyPath, 'countyLayer')
names = []
with arcpy.da.SearchCursor(lyrCounty, "NAME") as rows:
     for row in rows:
        names.append(row[0])
names = list(set(names)) # unique-ize list ( in case of multi-poly counties)

for name in names:
    where = "NAME = '{}'".format(name)
    arcpy.SelectLayerByAttributes(lyrCounty, "", where)
    ...

Re: arcpy.Clip_analysis error (can't find the cause)

MichaelSouffront — Tue, 05 May 2015 22:32:00 GMT

That's how it was before. I would loop through the counties and save them in a list and then loop through the list separately, I changed it hoping that making it more simple would help me find the error.

A coworker is now working on it to see if he can find the problem.

P.S. it's not the geometry either.

Re: arcpy.Clip_analysis error (can't find the cause)

MichaelSouffront — Tue, 05 May 2015 22:35:15 GMT

That's a good suggestion, however it doesn't solve the problem.

The .replace() was in the script before I updated it, I added the str() in case unicode strings were the ones causing the problem. but no.

Re: arcpy.Clip_analysis error (can't find the cause)

JoshuaBixby — Wed, 06 May 2015 00:41:38 GMT

Where are you executing this code? As a standalone script? In an ArcGIS Desktop interactive Python window? If the latter, are you running Background Processing? (Not saying any of this is the issue, but it helps to know the specifics of where/how the code is being run). Also, what version of ArcGIS are you running?

What happens if you programmatically only pass it SANPETE, i.e., add a WHERE clause to your search cursor to only return SANPETE. Speaking of cursors, I see you are using the older style search cursor. Have you tried using a ArcPy Data Access Search Cursor?

Re: arcpy.Clip_analysis error (can't find the cause)

curtvprice — Wed, 06 May 2015 00:53:40 GMT

I agree that the arcpy.da cursors are far better for almost all applications. They are cleaner to deal with (no cursor variables left around to delete) and are 10 to 100x faster.

Re: arcpy.Clip_analysis error (can't find the cause)

benberman — Wed, 06 May 2015 20:41:39 GMT

As Curtis pointed out, it's tidier to build a list first and then loop through it. Why is the counter starting at 1, and not at 0 ?

Re: arcpy.Clip_analysis error (can't find the cause)

MichaelSouffront — Thu, 14 May 2015 20:01:01 GMT

Running as standalone from Spyder, also tried Pyscripter, ArcGIS 10.3.

We did tried the .da cursors to see if that was it, but no.

See solution explained below.

Re: arcpy.Clip_analysis error (can't find the cause)

MichaelSouffront — Thu, 14 May 2015 20:30:10 GMT

My coworker spent a couple of days on this and couldn't find the error either. At the end we just rewrote the whole thing differently.

The actual error was never found but at this does what we needed.

One thing we added that may have solved the problem was the simplify polygon function on the wetlands data. We believe that the wetlands data had some sliver in between counties or something like that, but there's still the exception that things ran fine when done manually so we will never know for sure.

Thank you all for your help anyway.

Michael

Re: arcpy.Clip_analysis error (can't find the cause)

ThomasLaxson — Tue, 03 May 2016 23:43:29 GMT

For anyone else who might come across this, an answer on StackExchange worked for me. "this is a resources issue". Doing whatever you can to use fewer resources; e.g.:

- Ensure you're in a 64-bit/background environment)

- Generalize/Simplify your geometries

- Repair geometries

- Use fgdb (my issue was that I was processing data in an .mdb that some dinosaur had sent me)

Best of luck.