topic Re: Calculate Field loop in Python Questions

Calculate Field loop

ElaineKuo — Thu, 18 Jul 2013 08:16:27 GMT

Hello

System Windows Vista ArcGIS 9.3

Now I have a shape file.
Its attribute table consisted of fields of GridCell ID (GID) and
species ID starting with D.
(Dxxxx (such D8729, D6745, D2765))
The rows are GridCell ID (0-4800).
Each cell in the attribute table has either 1 and 0.

I would like to create a new field called All.
In the field All (Long 9, 9), the cell would should be assigned 1
when the cell of Dxxxx is 1. Otherwise, the cell would be 0.

The code can run the requirement above for one field.
Please kindly advise how to add the loop process for the multiple fields of Dxxxx
Thank you in advance.

the code

##Script Name: calculate sum ##Description: calculate sum of merged range sizes of a taxonomy of migratory birds  ##Created By: Elaine Kuo ##Date: 07/18/2015    #Import standard library modules import arcgisscripting import os  #Create the Geoprocessor object gp = arcgisscripting.create(9.3)  #Set the workspace. gp.Workspace= "H:/temp_D/test_1"   #Set the workspace. List all of the feature classes in the dataset outWorkspace= "H:/temp_D/test"  #Get a list of the featureclasses in the input folder fcs = gp.ListFeatureClasses()   # Loop through every item in the list that was just generated for fc in fcs:      # Break out the name, no path or extension, using the describe object.     desc = gp.describe(fc)     featureName = desc.name      # Add a field to this shapefile, of type LONG     gp.AddField (fc, "All", "Long", 10,10)         # Make temporary featureclasses     gp.MakeFeatureLayer(fc,"lyr")          #   Get a list of the fields in the featureclass     fields = gp.ListFields("lyr", "D*", "Long")          # Loop through every item in the list that was just generated      for field in fields:          gp.toolbox = "Data Management"          # Select records to be copied (C*, i.e. C7658)         query = "\"%s\" = 1" % field.Name         gp.SelectLayerByAttribute("lyr", "ADD_TO_SELECTION", query)                   # copy values in existing fields to the new field          gp.CalculateField("lyr", "All", "1", "PYTHON_9.3")                #Validate the new feature class name for the output workspace.     #OutFeatureClass = outWorkspace + os.sep + gp.ValidateTableName(fc,outWorkspace)      # clear memory of layers     gp.Delete("lyr")  gp.AddMessage(gp.GetMessages()) print gp.GetMessages()

Re: Calculate Field loop

T__WayneWhitley — Thu, 18 Jul 2013 11:17:46 GMT

I think I see what you are doing, although I believe I'd do it with an updatecursor. Am I correct that you ask for the remaining records for the 'ALL' field be coded 0 after you have coded the others 1? If so, to carry on with what you already have, see the below.

I noticed you're looping through a list of fields to form your sel query, so your fields must be varying depending on what fc or table you feed in. Then you're performing a 'select by attributes' based on the formed query using 'add to selection' param -- because of this, you don't need to 'recalculate' the previous selections in successive loop iterations. What I mean is you don't need to indent this line within your 'for' loop:

gp.CalculateField("lyr", "All", "1", "PYTHON_9.3")

Run this outside your loop, then follow that with another 'select by attributes' - I believe it has a 'switch selection' parameter. Then run your field calculation again, as in:

gp.SelectLayerByAttribute_management("lyr", "SWITCH_SELECTION")
gp.CalculateField("lyr", "All", "0", "PYTHON_9.3")

This is untested, so hope that works for you...

Enjoy,
Wayne

PS - See my line of code that included the toolbox alias, "_management". If you call your tools in that manner then you don't need lines such as the one you included: gp.toolbox = "Data Management"

Re: Calculate Field loop

ElaineKuo — Sat, 11 Dec 2021 09:16:14 GMT

Hello Wayne,

I tested my code and it worked well.
Actually I want to have a summary field of
the presence (1) of species ID (column) in the GridCell ID (row).

In other words, the GridCell of the field "All" should be assigned as 1,
as long as the GridCells of D8729, D6745, D2765 is 1.

Thanks for reminding and help, Wayne.

##Script Name: calculate sum
##Description: calculate sum of merged range sizes of a taxonomy of migratory birds 
##Created By: Elaine Kuo
##Date: 07/18/2015
 

#Import standard library modules
import arcgisscripting
import os

#Create the Geoprocessor object
gp = arcgisscripting.create(9.3)

#Set the workspace.
gp.Workspace= "H:/temp_D/test_1" 

#Set the workspace. List all of the feature classes in the dataset
outWorkspace= "H:/temp_D/test"

#Get a list of the featureclasses in the input folder
fcs = gp.ListFeatureClasses() 

# Loop through every item in the list that was just generated
for fc in fcs:

    # Break out the name, no path or extension, using the describe object.
    desc = gp.describe(fc)
    featureName = desc.name

    # Add a field to this shapefile, of type LONG
    gp.AddField (fc, "All", "Long", 10,10)   

    # Make temporary featureclasses
    gp.MakeFeatureLayer(fc,"lyr")
    
    #   Get a list of the fields in the featureclass
    fields = gp.ListFields("lyr", "D*", "Long")
    
    # Loop through every item in the list that was just generated 
    for field in fields:

        gp.toolbox = "Data Management"

        # Select records to be copied (C*, i.e. C7658)
        query = "\"%s\" = 1" % field.Name
        gp.SelectLayerByAttribute("lyr", "ADD_TO_SELECTION", query)

        
        # copy values in existing fields to the new field 
        gp.CalculateField("lyr", "All", "1", "PYTHON_9.3")          

    #Validate the new feature class name for the output workspace.
    #OutFeatureClass = outWorkspace + os.sep + gp.ValidateTableName(fc,outWorkspace)

    # clear memory of layers
    gp.Delete("lyr")

gp.AddMessage(gp.GetMessages())
print gp.GetMessages()

Re: Calculate Field loop

T__WayneWhitley — Thu, 18 Jul 2013 22:23:07 GMT

Soooo, your code does as you wish it to do now?

I am not sure if you have realized yet that you frequently mark your own post as 'the answer'. Basically, you can do either or both of 2 actions: award points to various posts and/or award the best answer (if there is one) to a single post. It is at least unclear to me anything was resolved...

Also, although it isn't always possible to do so, it is considered 'good etiquette' or at least good practice to post your final code or explain if you resolved your problem in a different way. Pretend you have colleagues following up behind you with a similar question researching the thread you started here - did you leave them in the dark?

Keep trying, and...

Enjoy,
Wayne

Re: Calculate Field loop

ElaineKuo — Thu, 18 Jul 2013 22:43:22 GMT

gp.CalculateField("lyr", "All", "1", "PYTHON_9.3")

Run this outside your loop, then follow that with another 'select by attributes' - I believe it has a 'switch selection' parameter. Then run your field calculation again, as in:

=> I think the "gp.CalculateField" is in need in the loop,
because the information of every field of Species ID (Dxxxx) needs to be updated in the field "All."
The field "All" is a summary field of the multiple Species ID (Dxxxx).

Re: Calculate Field loop

T__WayneWhitley — Thu, 18 Jul 2013 23:04:26 GMT

You forget the 2nd parameter in Calculate Field names the field to calculate values for. You have named this "ALL"; so what your inner 'for' loop does is add to an already selected set on your feature layer (creates a new selection on the 1st iteration), then for whatever species ID was found to be populated (1), "ALL" is then calculated as 1.

The subtlety I think you're missing here is your field calculation doesn't do a sum of any kind. It doesn't matter if performing the calculation inside or outside the loop (except it is more efficient to do it outside, after the selection is 'compiled', for lack of a better word).

An example with a single row:

speciesA, speciesB, speciesC, ALL
1, 0, 1, 1

Of course, the value of ALL for this row is 1, correct? (This is not a sum.)
So you can see, if you ran the calculation within your 'for' loop, you'd be calculating the value to 1 twice, once for speciesA and once for speciesC. (which of course is not necessary)

If you actually want the end result to be a sum, i.e., "ALL" in this case should be 2, say so and someone will likely suggest for you a more efficient way to do this, probably with an update cursor.

Enjoy,
Wayne

Re: Calculate Field loop

ElaineKuo — Fri, 19 Jul 2013 03:12:08 GMT

You forget the 2nd parameter in Calculate Field names the field to calculate values for. You have named this "ALL"; so what your inner 'for' loop does is add to an already selected set on your feature layer (creates a new selection on the 1st iteration), then for whatever species ID was found to be populated (1), "ALL" is then calculated as 1.

The subtlety I think you're missing here is your field calculation doesn't do a sum of any kind. It doesn't matter if performing the calculation inside or outside the loop (except it is more efficient to do it outside, after the selection is 'compiled', for lack of a better word).

An example with a single row:

speciesA, speciesB, speciesC, ALL
1, 0, 1, 1

Of course, the value of ALL for this row is 1, correct? (This is not a sum.)
So you can see, if you ran the calculation within your 'for' loop, you'd be calculating the value to 1 twice, once for speciesA and once for speciesC. (which of course is not necessary)

Wayne

Thanks for reminding.
1. Your explanation on putting the gp.CalculateField outside the loop is very clear.
I will modify my code according to your advice.

2. sorry for misusing the word "sum."
Actually I just want to record whether the GridCell is 1, not the total number of "1" summed up.

Elaine

Re: Calculate Field loop

T__WayneWhitley — Fri, 19 Jul 2013 16:35:36 GMT

For clarity, I'll post below your code with modifications we discussed - I didn't test it, that's up to you. Note that this is certainly not the only way to perform your desired task. One more thing - check your Date in the script file header -- I am assuming you intend this to be a 'created' date and to have this completed this year, not 2015?

##Script Name: calculate sum ##Description: calculate sum of merged range sizes of a taxonomy of migratory birds  ##Created By: Elaine Kuo ##Date: 07/18/2015    #Import standard library modules import arcgisscripting import os  #Create the Geoprocessor object gp = arcgisscripting.create(9.3)  #Set the workspace. gp.Workspace= "H:/temp_D/test_1"   #Set the workspace. List all of the feature classes in the dataset outWorkspace= "H:/temp_D/test"  #Get a list of the featureclasses in the input folder fcs = gp.ListFeatureClasses()   # Loop through every item in the list that was just generated for fc in fcs:      # Break out the name, no path or extension, using the describe object.     desc = gp.describe(fc)     featureName = desc.name      # Add a field to this shapefile, of type LONG     gp.AddField (fc, "All", "Long", 10,10)         # Make temporary featureclasses     gp.MakeFeatureLayer(fc,"lyr")          #   Get a list of the fields in the featureclass     fields = gp.ListFields("lyr", "D*", "Long")          # Loop through every item in the list that was just generated      for field in fields:         # Select records to use in a field calculation (C*, i.e. C7658)         query = "\"%s\" = 1" % field.Name         gp.SelectLayerByAttribute_management("lyr", "ADD_TO_SELECTION", query)      # code the 'All' field of the net selected set with a value of 1     gp.CalculateField_management("lyr", "All", "1", "PYTHON_9.3")     gp.SelectLayerByAttribute_management("lyr", "SWITCH_SELECTION")     gp.CalculateField_management("lyr", "All", "0", "PYTHON_9.3")      #Validate the new feature class name for the output workspace.     #OutFeatureClass = outWorkspace + os.sep + gp.ValidateTableName(fc,outWorkspace)      # clear memory of layers     gp.Delete("lyr")  gp.AddMessage(gp.GetMessages()) print gp.GetMessages()

Careful with indention, easy to make an error there!

Enjoy,
Wayne