Batch processing within Python to add, calculate, drop multiple fields

[AlisonMontgomery]

I have been searching the forums and elsewhere online, but have yet to find anything to aid me. 

I am using the Geospatial Modelling Environment toolset, and the tool isectpolyrst, which gives you the raster cell count by type within specified polygons, but not the area, so I am hoping to add new area fields (one per count field), calculate the area field, and delete the old count field. So currently, I have one script that runs GME and appends the count data to the polygons, and then the model (see [ATTACH=CONFIG]26184[/ATTACH]) that adds, calculates and deletes fields. It's the model that I am looking to automate/script.

I know I can run the model part via batch processing and manually entering all the inputs/outputs within ModelBuilder, but ideally, I want a script that can hunt down each count field name, generate a new area field name based on user inputs, calculate this new area field, and then drop the old count field.

Before anyone asks, I am not using Tabulate Area because it keeps crashing when I run it over large shapefiles, so I went with GME.

I know one area of concern is how the original count fields are picked out correctly. I tried listing the fields, but I don't think using the list index will work for this, as the count fields could be anywhere in the attribute table, and there could be different numbers of count fields, which might make the index go out of range or something and error. However, I am not sure if its possible to call fields by a boolean search that only calls up fields whose names match the search input (e.g. is it possible to call up fields named NLCD1, NLCD2, etc by searching with something like "NLCD*"?).

Once I can get script to find the appropriate fields, I know I can just loop through the add field and calculate field, and then drop the fields once that has all run and I will be all set.

If anyone has thoughts (or needs clarification) please let me know! I wish I had real code to show you, but give I am stuck on the first step of finding the fields, I don't have much to show. Thanks in advance!

[StacyRendall1 · Accepted solution]

To do what you want you might have to use a Python script as a script tool (i.e. it ends up being a model builder box with inputs and outputs, just like Add Field or whatever). This is because, as far as I know, there is no easy way to get a list of fields that model builder can iterate over. If there were a way to output a list from a Calculate Value and get model builder to iterate over it you would be able to do as you intended with something like this in a Calculate Value tool; Expression: findFields("%Input Table%"); Code Block:

def findFields(inputFC):   fields = arcpy.ListFields(inputFC)    # list of desired fields   outFields = []   for field in fields:     # if NLCD in the field name, add to the list of fields     if 'NLCD' in field.name:       outFields.append(field.name)   return outFields

Someone may know a way so make this work, so I show it here, but as far as I know you will have to make a script tool. Making a script tool is a bit of work, so read the Arc documentation which explains all this in great detail.

To do what you want you could use list fields to get a list of the FC fields, then search through them to find which fields have the desired names and then iterate over each row and sum the values from the desired fields, adding to the output field all at once. Below is the core of a script to do just that. I have deliberately left out the inputs and outputs of the tool, hopefully you will pick up how to do this from reading the documentation, and thus gain a greater understanding of what is actually going on.

## Get input feature class as inputFC  outputFieldName = 'Count' # add output field arcpy.AddField_management(inputFC, outputFieldName, 'LONG')  # list of fields in input FC inFields = arcpy.ListFields(inputFC)  # list of desired fields - output field is index 0 sumFields = [] for field in inFields:     # if NLCD in the field name, add to the list of fields     if 'NLCD' in field.name:         sumFields.append(field.name)  # use DA update cursor to add sum of values to Count field # will only work with Arc 10.1, there is a slightly different tool for 10.1 with arcpy.da.UpdateCursor(inputFC, [outputFieldName] + sumFields) as cursor:     for row in cursor:         # sum the NLCD fields, no matter how many of them there are (+1 skips the output field)         row[0] = sum([row[i+1] for i in range(len(sumFields))])         cursor.updateRow(row)  ## pass inputFC back as an output

[RhettZufelt]

You need to look at listFields http://resources.arcgis.com/en/help/main/10.1/index.html#//018v00000012000000 using the wildcard to limit it to "NLCD*" fields.

Then you can iterate through the list of fields that match your wildcard and perform what you need to do.

A couple thoughts, if you know the number of grid cells that fall within the polygon (the count) and the grids are a set size per cell, then the count * size per cell is the area.

Also, instead of creating/deleting fields, you can use internal references for your calculator expressions: (of course, you should work on copies of the data until you get it "right")

I.e.,   if you want to take the value from field1 and multiply by something, you don't have to go to a new field, you can clobber the values.

field1 = field1 * value       works as long as the field1 type will support the resultant value.

Also, keep in mind that you can go to the file menu of you model and export to a python script.  This will at least give you the syntax for what you have set up in there already, as a starting point for the scripting.

R_

[AlisonMontgomery]

Thank you both!

Between the two responses, I am well on my way to getting there! I just was unsure about how to use the wild card when pulling out the field names, and your code and ideas helped me figure it out.

Thanks again! If I could mark both as answers, I would 😞