Record limit when running Identity tool from PythonWin?

415
1
07-01-2014 12:51 AM
DaveMiller
New Contributor II
Hi guys,

Experienced GIS user here attempting to use PythonWin to run an identity process.

I seem to be running into a hard limit in the number of records generated in the output dataset.

Here's the background:-
I have a Python script which runs an identity operation between two datasets (both File Geodatabase feature classes inside the same geodatabase). It later performs a frequency, then pivot, on the output attribute table and later peforms some field calculations and field deletions in its final stages. I've tried and tested each section of this workflow in separate python scripts other than the identity command run at the first stage which was previously run inside ArcCatalog using the tool dialog box.

When I attempt to run this identity, the output dataset is partially generated, but appears to fail at, or close to, 220,000 records. (First attempt 220,000, second attempt 219,915).

Is there a hard limit on the number of records, or the file size, that can be generated when running the tool inside PythonWin? What else could be throwing it? Environment Settings? Scratch workspace? Is it the install of PythonWin?

Any suggestions gratefully received.

Running ArcGIS 10.1 for Desktop on a Windows 7 64-bit machine with 16GB of RAM.
Tags (2)
0 Kudos
1 Reply
FilipKrál
Occasional Contributor III
Hi Dave, greetings from CEH.

I haven't used Identify extensively but maybe some of my experience with Dissolve and alike might help:

1) If the geometries you are dealing with are really complicated (meaning they have lots of vertices, some features are very small while other features are very big, etc.), consider using the Dice tool to split it into more manageable chunks. I am not sure how to use this with the Identify tool though (it will probably complicate your work flow a bit). All I am saying it that "godzzilla" geometries may be a problem.
See dicing-godzillas-features-with-too-many-vertices.

2) A trick that helped me in the past was to run the 'memory expensive' tool in a separate subprocess.

3) Do you need to run it from PythonWin? Try to run the script from command line and see if that makes any difference. I haven't experienced this myself, but some posts I read mentioned that some things that didn't work in PythonWin or PyScripter worked when the script was executed from cmd.exe.

Let us know how you get on and I hope someone else can add more tips.
Filip.
0 Kudos