topic Re: Row objects in the Data Access module in Python Questions

Row objects in the Data Access module

ChrisMathers — Tue, 06 Nov 2012 19:15:50 GMT

Can anyone explain the reasoning behind returning a row object as a tuple when using the Data Access module update cursor? I can understand it with a search cursor, you can dict(zip(cursor.fields,row)) and make a dictionary that lets you call the values by field name. I don't understand how I should be using this without counting to find the field index number in the tuple. The help docs are not very clear on this topic. It seems like returning a dictionary would be easier for the users. I only just was given permission to update to 10.1 because sp1 is now out so I may have missed this discussion at an earlier date. So as not to be entirely contrary, I do like the with-as usage for cursor's. That's pretty neat.

Re: Row objects in the Data Access module

ChrisMathers — Sat, 11 Dec 2021 06:28:15 GMT

Just thought of this, it should work.

with arcpy.da.UpdateCursor(FEATURE CLASS,'*') as cursor:
  rowDict=dict(zip(cursor.fields,xrange(len(cursor.fields)+1)))) # zip fields to range of list length
  for row in cursor:
    row[rowDict[FIELD NAME]] = VALUE # use rowDict to find field list index
  cursor.updateRow(row)

Re: Row objects in the Data Access module

ThomMackey — Thu, 08 Nov 2012 01:28:28 GMT

In the interest of learning new ways to skin cats:

The enumerate() function is a good way of looping over iterables while counting how far along you are. Also, don't forget 10.1 uses Python 2.7, so you can use dictionary comprehensions to get rid of the ubiquitous dict(zip()) calls. So you could reduce it to:

fieldDict = {field: index for index, field in enumerate(cursor.fields)}

Barely a change, really, but a bit cleaner.

Re: Row objects in the Data Access module

ChrisMathers — Sat, 11 Dec 2021 06:28:17 GMT

That is cleaner. Thanks Thom.

EDIT: Interestingly, if you want to make a look up table (which I prefer to arrays if you dont need to do complex math) you can nest dictionary comprehensions.

fields={field : index for index,field in enumerate(cursor.fields)}
LookUp={row[fields['ORIG_FID']] : {field : row[fields[field]] for field in fields} for row in cursor}

Re: Row objects in the Data Access module

ThomMackey — Thu, 08 Nov 2012 22:21:11 GMT

You can... but should you? 😉 Don't forget the old axiom - it's twice as hard to debug code as it is to write it; so if you write code as cleverly as you can, you are by definition not smart enough to debug it... 😉

Just kidding, that's actually a really nice, concise way of dumping a feature class into a dict. Between that and the with... context, arcpy is really starting to feel Pythonic!

Re: Row objects in the Data Access module

MikeHunter — Sat, 11 Dec 2021 06:28:20 GMT

A really slick way that I use to return rows as dicts is:

def rows_as_dicts(cursor):
    colnames = cursor.fields
    uc = hasattr(cursor, 'updateRow')
    for row in cursor:
        row_object = dict(zip(colnames, row))
        yield row_object
        if uc:
            cursor.updateRow([row_object[colname] for colname in colnames])

Use this generator function like so:

with arcpy.da.SearchCursor(fc, ['*']) as sc:
    for row in rows_as_dicts(sc):
        x = row['MAPNAME']
        blah ...

This works with both da.SearchCursors and UpdateCursors. In the later case, it will update each row. I got the idea for this from a posting on Cafe Python and modified it to be able to determine which type of cursor is passed to the generator. Excellent idea from Jason or one other ESRI Python guys.

Mike

Re: Row objects in the Data Access module

ChrisSnyder — Sat, 11 Dec 2021 06:28:23 GMT

Use the .fields property (a property of the .da cursor). It gives you a tuple of the field names returned by the cursor... Then use the .index property of the .fields tuple. For example:

updateRows = arcpy.da.UpdateCursor(myTable,["*"])
for updateRow in updateRows:
   updateRow[updateRows.fields.index("BASIN_NAME")] = "Basin "+ str(updateRow[updateRows.fields.index("BASIN_ID")])

I found that using the .fields property like this is no slower (well maybe a teeny bit) than just using the index to set/retreive the field value... even for millions of records. Like less than a second differance...

In my opinion, the .da cursors are the best thing since slided bread.

Re: Row objects in the Data Access module

ChrisMathers — Tue, 13 Nov 2012 16:25:50 GMT

Speed wise they are glorious. I just dont get why we had attribute access changed to a tuple from a property. Never used the index method of a tuple before though, certainly looks nicer than what I had in place.

Re: Row objects in the Data Access module

ChrisSnyder — Tue, 13 Nov 2012 16:37:33 GMT

Yep - I had the same complaint when I started messing around with the da cursors: What happened to .getValue and .setValue? How are you supposed to reference the field names?

updateRow[updateRows.fields.index("FIELD_NAME")])

was the best replacement I could come up with... Maybe a bit longer code wise, but that's okay. Maybe it'd be better to have a little function fetch the index, but...

Why a tuple? I don't know, but I have found it to be quite nice for a lot of things, such as direct table to dictionary conversion and data conversion. It's nice to have the row values be a an iterable data type.

Took me a while to appreciate it (just like the v10.0 MapAlgebra syntax), but now that I get it, I like it way better.

Re: Row objects in the Data Access module

ChrisMathers — Tue, 13 Nov 2012 16:47:42 GMT

Maybe there will be someone from the Arcpy team at SERUG this year I can bug about it :D. It was obviously a deliberate choice to make such a big change. Ill use your method for now Chris. tuple.index is a good solution.

Re: Row objects in the Data Access module

ChrisSnyder — Tue, 13 Nov 2012 17:39:27 GMT

BTW: I have heard that the .da cursors are similar to the way the ArcObjects cursors behave (with an index).

Re: Row objects in the Data Access module

DavidWynne — Tue, 13 Nov 2012 18:55:37 GMT

Maybe there will be someone from the Arcpy team at SERUG this year I can bug about it :D. It was obviously a deliberate choice to make such a big change. Ill use your method for now Chris. tuple.index is a good solution.

Performance is the big reason. Getting the row back in a tuple that you can pull values by index is a much faster approach than creating a row object and then have to make individual calls to get specific values.

The approach of forming a dictionary and then pulling values out from that is okay, but that is adding overhead just to enhance readability. I would much rather just add code comment(s) if I need to clarify which field value is which.

-Dave