topic Re: How to keep missing fields from crashing my code? in Python Questions

How to keep missing fields from crashing my code?

RichardThurau — Tue, 12 Jun 2012 14:41:10 GMT

Hi,
I have a very helpful code I use to run metrics between land cover and a bunch of assessment boundary feature classes using tabulate area. I put my land cover classes in a list and the field names that are calculated from the classes in another list. Update cursor fills in the rest.

The problem is, if I have an assessment boundary that does not intersect a land cover class, when update cursor goes to calculate the field from the class value, it crashes the code.

I will paste a sample code:

ClassList = ["Forest", "Vegetation"]
FieldList = ["Land_Acres","Forest_Acres", "Forest_Percent", "Veg_Acres", "Veg_Percent"]

m2ac = 0.0002471054

FcList = arcpy.ListFeatureClasses("*", "")   
for fc in FcList:
    areaTab1 = "Metrics_" + os.path.basename(fc)
    for field in FieldList:
        arcpy.AddField_management(fc, field, "DOUBLE")
    TabulateArea(fc, "GID", LC, "Class", areaTab1 , 1) #"UTC_Metrics_" + os.path.basename(fc)
    arcpy.JoinField_management(fc, "GID", areaTab1, "GID", ClassList)#"trees11_tab_" + os.path.basename(fc)
    rows = arcpy.UpdateCursor(fc)
    print str(fc)
    for row in rows:  
        row.setValue(FieldList[0], (row.Shape_Area * m2ac))#Total Acres
        rows.updateRow(row)
        row.setValue(FieldList[1], row.getValue(ClassList[0]) * m2ac) #Forest Acres
        rows.updateRow(row)
        row.setValue(FieldList[2], (row.getValue(FieldList[1]) / row.Land_Acres) * 100) #Forest Percent
        rows.updateRow(row)
        row.setValue(FieldList[3], row.getValue(ClassList[1]) * m2ac) #Veg Acres
        rows.updateRow(row)
        row.setValue(FieldList[4], (row.getValue(FieldList[3]) / row.Land_Acres) * 100) #Veg Percent
        rows.updateRow(row)
    del row
    del rows

So, this code runs great, unless one of the feature classes does not intersect with one of the land cover classes. Then, when the code calls the ClassList for that value, the code will crash in two spots:
1. At the joinfield, where it looking for all classes from the class list to join
2. In the update cursor when it calls the class name to calculate the fields.

Is there a good way to make the code just skip the value rather than crashing the code?

Thanks

Rich
#

Re: How to keep missing fields from crashing my code?

MathewCoyle — Tue, 12 Jun 2012 14:56:54 GMT

Do you want your script to process that feature class at all if it doesn't have the required parameters? You could use a try/except block to exit that iteration and go to the next, or you could use a list field check on your feature class to compare against your list of fields that you want to be there.

Re: How to keep missing fields from crashing my code?

RichardThurau — Tue, 12 Jun 2012 15:14:47 GMT

Mathew Thanks,
I absolutely want the code to be able to just: 1. Join all available fields, and 2. Calculate all available fields. Just keep on trucking through the feature class loop even if one feature class does not intersect with a land cover class.

I thought about a try/except for the update cursor, but I think that would still stop the code from calculating remaining fields. For example, if there was no forest, I'd still want it to calculate vegetation. Which a try/except would have to added to each line in the update cursor, correct?

But, I don't see a way to use try/except for the joinfield, where I will get a failure if my tabulate area table does not contain all fields within the ClassList.

rt

Re: How to keep missing fields from crashing my code?

MathewCoyle — Sat, 11 Dec 2021 19:56:02 GMT

Probably not the best solution, but this should keep your loops running even if an individual iteration fails.

for fc in FcList:
    try:
        areaTab1 = "Metrics_" + os.path.basename(fc)
        for field in FieldList:
            arcpy.AddField_management(fc, field, "DOUBLE")
        TabulateArea(fc, "GID", LC, "Class", areaTab1 , 1) #"UTC_Metrics_" + os.path.basename(fc)
        arcpy.JoinField_management(fc, "GID", areaTab1, "GID", ClassList)#"trees11_tab_" + os.path.basename(fc)
        rows = arcpy.UpdateCursor(fc)
        print str(fc)
        for row in rows:
            try:
                row.setValue(FieldList[0], (row.Shape_Area * m2ac))#Total Acres
                rows.updateRow(row)
                row.setValue(FieldList[1], row.getValue(ClassList[0]) * m2ac) #Forest Acres
                rows.updateRow(row)
                row.setValue(FieldList[2], (row.getValue(FieldList[1]) / row.Land_Acres) * 100) #Forest Percent
                rows.updateRow(row)
                row.setValue(FieldList[3], row.getValue(ClassList[1]) * m2ac) #Veg Acres
                rows.updateRow(row)
                row.setValue(FieldList[4], (row.getValue(FieldList[3]) / row.Land_Acres) * 100) #Veg Percent
                rows.updateRow(row)
            except:
                pass
        del row
        del rows
    except:
        pass

Re: How to keep missing fields from crashing my code?

RichardThurau — Tue, 12 Jun 2012 17:34:40 GMT

Mathew, you are correct. This will keep the code rolling, but doesn't really resolve the issue.

What would be awesome is have a classList based on the land cover. Regardless of what classes intersect, that classList will work. The sample code calculates values for five fields, but the real code has more than 30. The pain is in having to alter the code to match which land cover types intersect (getValue order from the classList).

One thing that would help, can I build the classList from the tabulate area table? Doing that would allow me to create a list that only contained the intersecting land cover types for each feature class. Then the try/except might work in the update cursor. Depends on where pass sends the code. Currently I think it just passes to the next feature class if any of the updateRow functions do not work.

Thanks for your thoughts.

rich

Re: How to keep missing fields from crashing my code?

GregBacon — Thu, 21 Aug 2014 14:04:03 GMT

Richard,

Did you ever find a way to address this? I'm doing the same thing with a tabulation of land cover types within watershed polygons. Not all of my seven possible land cover types are present in each watershed, so the update cursor has problems with fields that may or may not be present.

Greg

Re: How to keep missing fields from crashing my code?

XanderBakker — Thu, 21 Aug 2014 14:51:20 GMT

There are ways to avoid your code from crashing and only update those field that exist. I would probably use some dictionaries for this purpose and switch to the da cursors. If you are willing to drop a small part of your data I can have a look what I can do.

Kind regards, Xander

Re: How to keep missing fields from crashing my code?

GregBacon — Thu, 21 Aug 2014 15:15:25 GMT

I'm posting a simplified table for this example. The table is the result of the Tabulate Area tool and it will be inside a file gdb created by my model. Let's just try to convert the square feet for each cover type to acres. The problem is that you never know which of seven possible cover types you'll have in this table. The path I've started down is this:

import arcpy

arcpy.env.workspace = r'G:\users\gis\gbacon\Watershed_Testing\valleyave\WatershedDelineation.gdb'

fldlist = arcpy.ListFields("TabSummary","","Double")

with arcpy.da.UpdateCursor("TabSummary",fldlist) as cursor:

for row in cursor:

row[0:] = row/43560

cursor.updateRow(row)

The update cursor needs a list of fields, but the fldlist variable can't be used here. The second issue seems to be in my use of row[0:] where I'm trying to say take each value from the first to the last of the possible seven and divide each by 43560. I'm very new to Python and I appreciate your help.

Re: How to keep missing fields from crashing my code?

XanderBakker — Thu, 21 Aug 2014 15:20:42 GMT

Hi Greg,

I'm afraid you added the XML and not the DBF file itself. Can you add it?

Re: How to keep missing fields from crashing my code?

XanderBakker — Thu, 21 Aug 2014 15:23:09 GMT

If I read the DBF into a fgdb I will have a similar situation as you have?

Re: How to keep missing fields from crashing my code?

XanderBakker — Thu, 21 Aug 2014 15:32:38 GMT

some pointers (without having seen the data):

are you sure that all double fields need to be updated?
the fldlist in your case is a list that contains field objects, but the cursor will need a list or tuple of field names. I changed this on line 4
for each row you will need to loop through the fields and calculate the new values (see line 8 - 11)

import arcpy
arcpy.env.workspace = r'G:\users\gis\gbacon\Watershed_Testing\valleyave\WatershedDelineation.gdb'
tbl = "TabSummary"
fldlist = [fld.name for fld in arcpy.ListFields(tbl,"","Double")]

with arcpy.da.UpdateCursor(tbl, fldlist) as cursor:
    for row in cursor:
        i = 0
        for fld in fldlist:
            row = row / 43560
            i += 1
        cursor.updateRow(row)

Haven't tested it though. Since you are changing the input data, test it with a copy of the data. Also don't run the code more than once on the same data...

Kind regards, Xander

Re: How to keep missing fields from crashing my code?

GregBacon — Thu, 21 Aug 2014 15:53:59 GMT

This worked nicely! Thank you. In this case, all “double” fields needed the update. What would change if I had some “double” fields that needed to be left alone?

I just checked the forum and found that my table is in fact in the zip file attachment. I apologize for not having posted the code correctly. It seems like there’s a better way to do it.

Re: How to keep missing fields from crashing my code?

XanderBakker — Thu, 21 Aug 2014 16:12:46 GMT

Hi Greg,

The only file in the ZIP file is an XML file called "TabSummary.dbf.xml" (or at least this is what passed my firewall).

In case you know the names of the double fields you want to maintain untouched, you would have to exclude those from the list of fieldnames. This can be done like this (see code below).

the flds_exclude contains a list of fieldnames that should remain untouched
flds_touse is a new list where all the items from the exclude list are remove from the list of double fields
set() is used to be able to do this type of comparisons with lists, after this the set is converted to a list again

import arcpy
arcpy.env.workspace = r'G:\users\gis\gbacon\Watershed_Testing\valleyave\WatershedDelineation.gdb'
tbl = "TabSummary"
fldlist = [fld.name for fld in arcpy.ListFields(tbl,"","Double")]

flds_exclude = ["myFieldname1", "myFieldname2", "myFieldname3"]
flds_touse = list(set(fldlist) - set(flds_exclude))

with arcpy.da.UpdateCursor(tbl, flds_touse) as cursor:
    for row in cursor:
        i = 0
        for fld in flds_touse:
            row = row / 43560
            i += 1
        cursor.updateRow(row)

Kind regards, Xander

Re: How to keep missing fields from crashing my code?

GregBacon — Thu, 21 Aug 2014 17:18:36 GMT

Thanks again. Set() could be very useful to me.