topic Re: Insert cursor multiple rows in Python Questions

Insert cursor multiple rows

DonKatnik — Tue, 07 Feb 2012 15:04:52 GMT

I am using an insert cursor to create many (>5,000) new rows in a feature class. The cursor is drawing shapes from one feature class and a variety of attributes from a series to tables to create a large "flatfile" featureclass. It is taking a very long time to run. In VBA, there was a "rowbuffer" object that was more efficient than inserting single row objects -- is there something similar in Python? Is there a faster way to add rows than an insert cursor on the feature class object (by creating a feature layer or table view first)?

Re: Insert cursor multiple rows

curtvprice — Tue, 07 Feb 2012 18:26:00 GMT

I am using an insert cursor to create many (>5,000) new rows in a feature class. It is taking a very long time to run. In VBA, there was a "rowbuffer" object that was more efficient than inserting single row objects -- is there something similar in Python?

You may want to try buffering using the in_memory workspace: create a feature class in the in_memory workspace, insert rows to it using a cursor, then copy them out to disk, use delete rows tool to clear it, repeat. At the end of your process append your pieces together.

You could still use your VBA method if you used a .NET addin instead of Python (none of that for me, thanks).

Esri is addressing this shortcoming of arcpy (slow cursors) with the new arcpy.da method in Arc 10.1. I haven't tried it yet but have been told it will be dramatically faster.

Re: Insert cursor multiple rows

markdenil — Tue, 07 Feb 2012 18:29:50 GMT

How are you doing this? Not knowing makes it hard to locate the problem.
The insert cursor itself should run pretty quickly, but if you are fetching the input for each row from all the various sources as each row is needed, I can see that being very slow.
Your reference to the source of the row items leads me to think you are doing something like that.

Gathering all the row data first, before inserting it into the feature class, would help.
Try building a list of the geometry object and attributes for each row, and nest the row lists in an enclosing list.
Then use that list to feed the cursor.

Perhaps a dictionary of rows keyed on a unique ID may be appropriate instead.
In any event, the idea is to feed the cursor as fast as it can spit the rows into the table.

Of course, you might be doing something like this already.... hard to tell.
Working on an in-memory feature class would likely help too.