Error 99999 using Clip/Intersect on huge polygon datasets

[MartinScheiber]

Hello everybody,

I´m trying to clip/intersect two huge polygon datasets (500.000 and around 3.500.000 Polygons).
Every time the geoprocess ends with the following error message:

ERROR 999999: Error executing function. Invalid Topology [Topoengine error.] Failed to execute (Clip).

I tried "Repair Geometry" and there are no errors anymore, but it still doesn´t work.

I´m using ArcGIS 10.0 on Windows 7, 64 Bit, 4 Gb RAM. Both files are in File Geodatabases.
Any help would be highly appreciated.

[curtvprice]

I have done some tests and am convinced it's a memory issue. Would you be able to go into a little more depth into how one would use the gc module in an arcpy workflow?

I suggest including the above two lines near the top of your script, before you import arcpy, and see if it helps.

The gc module is part of the Python standard library, documented on the python.org website.

[JamieKass]

ken2472 -- I am quite sure it is a memory-related issue, as the process runs smoothly and produces output with no geometry errors when processed outside the script in ArcMap. What would my next steps be? I would like to know if there is a good solution to breaking up input polys into smaller pieces for a Union operation, but as each feature is dependent on every other one for a successful and accurate Union, I can't envision a method for subsetting my data.

curtvprice -- I tried some runs after importing gc and enabling it, but I have the same problems. What still confuses me is that I thought gc was automatically enabled anyway. What does enabling gc do then? It seems redundant, unless I do not fully understand the intricacies of gc.

I am still getting ERROR 999999: Error executing function. Invalid Topology [Topoengine error.], Failed to execute (Erase) and Failed to execute (Union). Again, these errors occur on large polygon datasets of approx. 50,000 features and spring up at the end of a long geoprocessing script.

[curtvprice]

I tried some runs after importing gc and enabling it, but I have the same problems. What still confuses me is that I thought gc was automatically enabled anyway. What does enabling gc do then? It seems redundant, unless I do not fully understand the intricacies of gc.

Now that I re-read the documentation, I think you're absolutely correct.
I thought gc was more than the "generic" garbage collection that Python does, but apparently not.

R:\>C:\Python26\ArcGIS10.0\python.exe
Python 2.6.5 (r265:79096, Mar 19 2010, 21:48:26) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import gc
>>> gc.isenabled()
True

[DimitraBouziou]

Hello,

I´m also having the same problem (ERROR 999999: Error executing function. Invalid Topology [Topoengine error.] Failed to execute (Intersect)) while trying to intersect a 5,000,000-point feature with a 2,000,000-polygon feature. Based on the discussion above I guess the only way is to break the datasets into smaller parts?  I am a new user of ArcGIS and I am not very familiar with the software.

I´m using ArcGIS 10.1 on Windows 7, 64 Bit, 16 Gb RAM. Both files are in a Geodatabase.

[KenHartling]

Did you check the blog link that I provided earlier in this thread?

If everything checks out according to the info in that blog, another option is to use the geometry operators. There is a discussion and code sample here.

If it still does not work for you, and if the workaround is not acceptable (the output will be different than the output from the Intersect tool), please submit a technical support incident and provide the data and exact parameters you are using in the Intersect tool so we can investigate.

Thanks,
Ken

Hello,

I´m also having the same problem (ERROR 999999: Error executing function. Invalid Topology [Topoengine error.] Failed to execute (Intersect)) while trying to intersect a 5,000,000-point feature with a 2,000,000-polygon feature. Based on the discussion above I guess the only way is to break the datasets into smaller parts?  I am a new user of ArcGIS and I am not very familiar with the software.

I´m using ArcGIS 10.1 on Windows 7, 64 Bit, 16 Gb RAM. Both files are in a Geodatabase.