Deleting Features: Does cursor mess with order?

[MatthiasBuehler]

Hi,

Question:

I have read that cursors can only be iterated forward, not in reversed order.

Not if I have e.g. 5 features and the following code structure:

delCursor = arcpy.da.UpdateCursor(fc, field)
for row in delCursor:

   if testIfRowNeedsToBeDeleted(..):

      delCursor.deleteRow() # messes with the number of elements in the loop!!

How can I guarantee that the correct features are deleted when I delete features with 'early' OBJECTIDs out of the list, as after the first deletion, the number of elements in the field is already not the same as the cursor's number.

How is this handled internally?

* * *

In the end, I just need to delete specific rows out of a feature class, depending on specific field values and I want to avoid any issues with the order.

My favourite would be this structure in pseudo code:

to deleteFeatureOBJECTIDs = []

loop over cursor:

   if feature needs to be deleted based on multiple field values:

      deleteFeatureIDs.append(current feature)

arcpy.deleteFeaturesInFC(deleteFeatureOBJECTIDs)

[[ I have also tried code-based creation of the SQL 'where_clause', with a string list, but this type of search crashes ArcGIS PRO quite easily. ]]

Any input welcome!

Thanks!

matt

[MatthiasBuehler]

ps.

if possible, I'd like not to have to operated with temp layers.

[NeilAyres]

Not quite sure what you are saying here about affecting the order.

If you run an update and then delete specific rows based on some criteria, the record will be deleted.

The row order, OID based is fairly arbitrary anyway, based on the order of creation.

So, if you have OIDs 1, 2, 3, 4, 5, then you delete 2 & 3, you will be left with OIDs 1, 4 & 5.

It will not automatically re-order or re number the remaining OIDs.

To do that you would have to do a copy features or something to a new table.

[DanPatterson_Retired]

If you have to use cursors, be positive thinking  and keep those features that you want to keep and save to a new featureclass.

You can put this in cursor-esque

awl = [1, 2, 3, 4, 5]
>>> dump = [2, 4]
>>> keep = [i for i in awl if i not in dump]
>>> keep
[1, 3, 5]‍‍‍‍‍

[MatthiasBuehler]

Hi guys, thanks for the replies - on a sunday!!

Oh - I think the key is in understanding that the OBJECTID in a FC is NOT always enumerated from 1 to n. I was under the impression this is the case. (I'm not a GIS expert)

I have just tried manually deleting a feature and adding an other one manually and the result shows me that the OBJECTID of the newly drawn does not replace the last (missing) spot, but is 1 higher than the highest existing OBJECTID value.

-->

1] Can you guys confirm that arcpy handles this the same as when doing stuff manually?

2] If so, then the whole question would be answered by this very important detail.

Thanks guys!

matt

[DanPatterson_Retired]

why don't you run a comparison on your data, since you will then have a standard to compare to.  Of course, it goes without saying, one never alters an existing dataset, but uses a copy.  Storage is cheap, time is not.

[RebeccaStrauch__GISP]

I would say yes, manual vs script will handle things the same, assuming all else is equal.

you may want to read thru Fundamentals of ObjectID fields—Help | ArcGIS for Desktop  to get a little better understanding or OIDs.

Since we don't really have control over the OID assignment, I suggest adding a unique field to store the value of the order (if that's what you are going for).  If important, you could start out by calculating the start value as the current OID value.  Then, before you delet the on record, grab the value and insert it in the "uniqid"  field for the new record.

but as Dan mentioned, always good to compare output from manual vs script (and on a copy).  Always good to test and review, especially if this is critical and will be used "blindly" (leap of faith) in the future.

sorry not including any code...on my iPad.

[XanderBakker]

Hi Matthias Buehler ,

When deleting elements from a collection you have to be careful not to "cut the branch you sitting on":

Source: http://www.publicdomainpictures.net/pictures/10000/velka/1210-1240955295Hn8Q.jpg 

The cursor however, handles this correctly (as far as I know). Look at the snippet below:

def main():
    import arcpy

    fc = r'C:\GeoNet\DeleteFeaturesPython\data.gdb\points'
    print "before", getCount(fc)

    with arcpy.da.UpdateCursor(fc, ('OID@')) as curs:
        for row in curs:
            oid= row[0]
            if oid % 2 == 0:
                # delete even oid's
                curs.deleteRow()

    print "after", getCount(fc)


def getCount(fc):
    return int(arcpy.GetCount_management(fc).getOutput(0))


if __name__ == '__main__':
    main()‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

It prints in my case:

before 876
after 438‍‍

And looking at the attribute table before and after it has removed correctly the rows with en even ObjectID:

   

      before                                                         after

Might be something specific with the case you are working on. 

BTW: love your CityEngine work on Independence Day: Resurgence 

[MatthiasBuehler]

Hi guys,

Thanks for all the inputs, much appreciated.

And yes, the features I'm deleting are derived from some other features, so if the input features have changed, I have to delete the irrelevant derived features.

Cool pic, btw!

Xander Bakker‌

Thanks for the compliment! I'm good in some things, but not so in others. Well, like all of us.

Did you see this presentation?

Arcpy is a new thing I'm learning for my CityEngine services, so I'm trying to get the things I need in a workflow prototype I'm working on for Esri Germany to work.

Huge thanks!

Matt

[XanderBakker]

Hi Matthias Buehler , 

If you can share a bit more information and perhaps some (dummy) data, maybe I can track what is happening and see if there is something in the Python script you're using.

And yes I attended the live-seminar where you presented your work for Independence day.  

Kind regards, Xander