problem with clipping

[DanielSchatt]

hi all, I'm using PythonWin and also tried PyScripter to clip one polygon geodatabase feature class with each polygon in another polygon geodatabase feature class.  I've used this code successfully before with other data but suddenly, with this particular dataset, it's giving me problems.  

First I create a list (CatchmentIDLIst) from an item of unique values in the clip feature class, then loop through the polygons by selecting on each item in that list and setting that one as the present clip polygon.  I know the list is being created successfully because it prints out correctly in the interactive window after I create it.

But for some reason, the clip (Clip_analysis) operation isn't working properly.  Code below:

import arcpy
import os

MyWorkspace = "C:\stuff\PrecipitationFlooding/Norfolk"
arcpy.env.workspace = "C:\stuff\PrecipitationFlooding\Norfolk\Norfolk_Data.gdb"
arcpy.env.overwriteOutput = True

arcpy.MakeFeatureLayer_management("Catchments_Polygons_Dissolve", "Norfolk_Catchment_FLayer")

# Create list for all the Catchment ID values
CatchmentIDList = []

# Populate list with all the Catchment ID values
with arcpy.da.SearchCursor("Catchments_Polygons_Dissolve","OBJECTID") as CatchmentRowsCursor:
    for eachCatchmentRow in CatchmentRowsCursor:
        CatchmentIDList.append(eachCatchmentRow[0])

print str(CatchmentIDList).strip('[]')

#############################################################
#############################################################
######### Iterate through each catchment ####################
#############################################################
#############################################################

for EachCatchmentID in CatchmentIDList:
    print "Catchment ID = ", EachCatchmentID
    arcpy.SelectLayerByAttribute_management("Norfolk_Catchment_FLayer", "NEW_SELECTION","OBJECTID = " + str(EachCatchmentID))
    
    NorfolkLandUseByCatchment = "Norfolk_Landuse" + str(EachCatchmentID)
    arcpy.Clip_analysis("Norfolk_LandUse_SPS","Norfolk_Catchment_FLayer",NorfolkLandUseByCatchment)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

The exact error is:

Traceback (most recent call last):
File "C:\stuff\PrecipitationFlooding\flooding.py", line 38, in <module>
arcpy.Clip_analysis("Norfolk_LandUse_SPS","Norfolk_Catchment_FLayer",NorfolkLandUseByCatchment)
File "C:\Program Files (x86)\ArcGIS\Desktop10.4\ArcPy\arcpy\analysis.py", line 56, in Clip
raise e
ExecuteError: ERROR 999999: Error executing function.
The table was not found.
The table was not found. [Norfolk_Landuse1]
The table was not found.
The table was not found. [Norfolk_Landuse1]
Invalid Topology [Topoengine error.]
Failed to execute (Clip).

I don't understand why it's trying to find a table "Norfolk_Landuse1".  It should be creating that feature class "Norfolk_Landuse1",  right?

Thanks much..

Dan

[DarrenWiens2]

Invalid Topology [Topoengine error.]

Try running Check Geometry and see if it returns any errors for your problem feature class.

[DanielSchatt]

thanks Darren, the geometry is definitely valid.   I clipped it successfully when I manually did it in ArcMap.

[DanielSchatt]

Turns out my code is fine but apparently Python scripts can't handle large datasets.  I was working with a land use cover for an average size locality that had about 1.2 million polygons.   I decided to try running a small portion of the original area and it ran smoothly.  So I'll just have to break this up into parts.

[BruceHarold]

You should be able to use 64bit arcpy.

Install 64bit background geoprocessing, this will lay down 64bit arcpy.

Use a master/worker script approach, your existing script is worker.py.

master.py can look like this:

import os, subprocess, time

command = r'C:\Python27\ArcGISx6410.4\pythonw.exe C:\your\path\to\worker.py'

os.environ['PYTHONPATH'] = r'C:\Python27\ArcGISx6410.4\Lib\site-packages'

process = subprocess.Popen(command)

while process.poll() == None:

    time.sleep(5)

It might work better to also first in worker.py make a dictionary of OID:Polygon key/value instead of the selection, and use a geometry as input to the clip rather than a selection on a feature class.

Using geometry objects with geoprocessing tools—Help | ArcGIS for Desktop