Calculate Field Tool and Python

[Corey_C_Denninger]

I'd think this one should be easy. 
How do I use my input parameters, to automatically update my Calculate Field expression, within a Python script?  I thought I had it, but I receieve an error.

In my script I would like portions of the expression in the Calculate Field line to change as my input parameters change.  If I force the expression like this, it works:
"( !SUM_POP10! / !SUM_SUM_POP10! )*100", "PYTHON")

But I need both of the fields (either side of the "/" line) in that expression to change based upon one of my input paramters I choose, here:  
popfield = gp.GetParameterAsText(2) #population field

Here is how I have it and where I beleive the Error is coming from; not sure what it doesn't like:

arcpy.CalculateField_management(outputfc, percentfield, '"('+str(popfield)+'/SUM_'+str(popfield)+')*100"', "PYTHON")

Again, as the "popfield =" paramter changes, so does the expression.  So as I choose a new "SUM_POPxx" field, the expression automatically changes with it.

I'd appreciate any assistance.  Thank you.

[StacyRendall1]

oops... That is to do with where the ! is placed - my bad!

Either:

popfield = arcpy.GetParameterAsText(2) #population field

arcpy.CalculateField_management(outputfc, percentfield, '(!'+popfield+'!/!SUM_'+popfield+'!)*100', "PYTHON")

Or:

popfield = arcpy.GetParameterAsText(2) #population field

popfield_new = '!'+popfield+'!'
sum_popfield_new = '!SUM_'+popfield+'!'

arcpy.CalculateField_management(outputfc, percentfield, '('+popfield_new+'/'+sum_popfield_new+')*100', "PYTHON")

I would say that the second one is better - the first kind of muddies what is going on in the calculation, while in the second your field references are absolutely explicit...

Cheers!

[Corey_C_Denninger]

Stacy - Thank you so much for your patience and assistance.  Works great.  I'm sure I do not have the code as efficient as it could be, but it works great and I learned some more about Python along the way.  At this rate, I'll be a decent programmer by 2020. 😉  I'll keep checking the Python, GIS, and stuff blog for more helpful tips. 

Here is how I have the code and it makes sense to me

import arcpy, os
gp = arcpy

arcpy.env.overwriteOutput = True

# Parameters
inputfc = arcpy.GetParameterAsText(0) #Input fc location and name
statstable = r'd:\My Documents\ArcGIS\Default.gdb\STATS' #Stats Table (temporary) output name (don't attempt to put into a feature dataset)
popfield = arcpy.GetParameterAsText(1) #population field
tablename = os.path.basename(statstable) #temporary table
layername = "templayer"
casefield = arcpy.GetParameterAsText(2) #CWM, or BASINNAME, or COUNTYNAME, WUCANAME_n, etc.,
percentfield = arcpy.GetParameterAsText(3) #Percent of Basin, or Percent of CWM, Percent of County, PER_POP_WUCA1_15, etc.
outputfc = arcpy.GetParameterAsText(4) #Output fc location and name

#########################################################################################################################

arcpy.MakeFeatureLayer_management(inputfc, layername)

arcpy.Statistics_analysis(layername, statstable, [[popfield, "SUM"]], casefield)
arcpy.AddJoin_management(layername, casefield, statstable, casefield)
arcpy.CopyFeatures_management(layername, outputfc)
arcpy.RemoveJoin_management(layername, tablename)

arcpy.AddField_management(outputfc, percentfield, "DOUBLE","","","","","")

arcpy.CalculateField_management(outputfc, percentfield, '(!'+popfield+'!/!SUM_'+popfield+'!)*100', "PYTHON")

arcpy.DeleteField_management(outputfc,['SUM_'+popfield,"FREQUENCY","OBJECTID_1",str(casefield)+"_1"])

One final question, what do the '!' do in Python and specifically in this script?

[ChristopherFricke1]

Exclamation marks in the Calculate Field calculation demarcate Fields in the table you are working with.

Ex.  !FIPS! = the value in the fips field

In VB it used to be the square bracket.  This is just one of those annoying things that you have to remember.
Another way to write this that might be easier is: "(!%s!/!SUM_%s!)*100" % (popfield, popfield)

This uses python's string formatting structure.  I find it is a bit easier to read than having all of the plus signs and stuff. 

code example:

arcpy.CalculateField_management(outputfc, percentfield, "(!%s!/!SUM_%s!)*100" % (popfield, popfield), "PYTHON")

[StacyRendall1]

Corey, no worries.

As Christoph said, the exclamation marks let Arc know that it is a field you are referencing within the string (the string telling it what to calculate), not a variable or something. So, it isn't actually anything to do with Python! I guess they made it different from VB, as square brackets might cause confusion with Python lists.

With these Arc codes I tend to flip-flop between using Python string formatting (%s and stuff) and adding strings together, depending on what is most readable in each particular instance...

Your code itself is currently about as efficient as it could be. If you wanted to improve it from here on in (I don't know in this case if there is much improvement possible, it might depend on the size of your datasets as to whether it is worth doing anything), you need to start thinking about what the code is actually doing, and if possible make the procedure more efficient. Some of the things that Arcpy lets you do, like cursors, can make it easier or more efficient to do certain things. First off, you need to start out with a pseudo-code description of what you want to do (preferably before you do anything else, but it never seems to actually happen that way). This is a low-level pseudo-code of your method:

import things

get parameters from Arc

statstable: sum layer popfield for each casefield

layer: join to statstable

layer: copy to output

layer: remove join

output: add field

output: calculate field - percent of sum within each case

output: delete sum field

At a higher level:

import things

get parameters from Arc

copy input features to output

output: add field

output: calculate field - percent of sum within each case

Now we have outlined the procedure, we can think about if it is possible to improve anything. You have to almost remove yourself from thinking about which Arc tool will do the job, and instead find the simplest(/most efficient) way to do something. For example, in one code that I had developed from a Model Builder model I was using a bunch of Select by Attriubute queries, doing a calculation and writing the result back. That was the only way I could think of to do it within Arc/Model Builder. The code took minutes to run on my small test dataset. Then I realised that I didn't have to be constrained by the way it has to be done in Arc; so I got rid of all the Select By Attributes, instead using a Search Cursor to extract the data to a Python dictionary, then perform the calculation on that; afterwards it took about 7 seconds.

However, in this case, I don't think you could get time savings from making any changes, but you could Python-ise it for the sake of learning a bit more... If you are interested, have a go doing something like this (you will need to do some research into Pyhton dictionaries and Cursors...):

import things

get parameters from Arc

copy input features to output

use Search Cursor to iterate over input rows
 Python dictionary: accumulate popfield value, using case field as key
 # i.e try: popDict[inputRow.GetValue(casefield)] += inputRow.GetValue(popfield)
 #     except KeyError: popDict[inputRow.GetValue(casefield)] = inputRow.GetValue(popfield) # first time it comes across each case field, it doesn't exist as a key, so just add it

output: add field

use update curosr to iterate over output rows
 get the pop_casefield_sum sum value from the dictionary
 get the popfield value from the output row
 update the field value for the row to (popfield/pop_casefield_sum)*100

Just yell out if you decide to try this and need any help!


p.s. You can now delete the 'gp = arcpy' on line 2 of your current code...

[Corey_C_Denninger]

Thank you very much Stacy!

[EdwardBriggler]

This was a BIG help. I was stuck on the concatenating part of my script too!

 
# Author: Edward Briggler
# Date: 1/4/2012
# Purpose: To add the 7 phase fields to a feature class and calculate based on selecting by subtype.
# Bring in Arc and Sys. 
import arcgisscripting, sys
# Create the Geoprocessor...
gp = arcgisscripting.create()
gp.Workspace = gp.GetParameterAsText(0)
fc = gp.GetParameterAsText(1)
f = gp.GetParameterAsText(2)
# concatenate the bangs to let the equations know it is a field
new_f = '!'+f+'!'
# only held in memory
tv = "table_view_memory"
# expressions to evaluate the length of Primary Line (in the length of specified field)
## Involves concatenating the *2 and *3 to get the expression right. HAS TO BE A STRING!!!
exp = new_f
exp2 = new_f+'*2'
exp3 = new_f+'*3'
# Turn Feature Class into a layer...can only do the following geoprocesses while a layer or table view.
gp.MakeFeatureLayer_management(fc, tv)
# Add Fields to Layer
gp.addfield_management(tv, "A_PHASE", "FLOAT")
gp.addfield_management(tv, "B_PHASE", "FLOAT")
gp.addfield_management(tv, "C_PHASE", "FLOAT")
gp.addfield_management(tv, "AB_PHASE", "FLOAT")
gp.addfield_management(tv, "AC_PHASE", "FLOAT")
gp.addfield_management(tv, "BC_PHASE", "FLOAT")
gp.addfield_management(tv, "ABC_PHASE", "FLOAT")
# Select by SUBTYPECD and run each expression accordingly
##gp.MakeTableView_management(fc, tv)
gp.SelectLayerByAttribute_management(tv,"NEW_SELECTION", "SUBTYPECD = 1")
gp.CalculateField_management(tv, "A_PHASE",exp , "PYTHON" )
gp.SelectLayerByAttribute_management(tv,"NEW_SELECTION", "SUBTYPECD = 2")
gp.CalculateField_management(tv, "B_PHASE",exp , "PYTHON" )
gp.SelectLayerByAttribute_management(tv,"NEW_SELECTION", "SUBTYPECD = 3")
gp.CalculateField_management(tv, "C_PHASE",exp , "PYTHON" )
gp.SelectLayerByAttribute_management(tv, "NEW_SELECTION", "SUBTYPECD = 4")
gp.CalculateField_management(tv, "AB_PHASE",exp2 , "PYTHON" )
gp.SelectLayerByAttribute_management(tv,"NEW_SELECTION", "SUBTYPECD = 5")
gp.CalculateField_management(tv, "AC_PHASE",exp2 , "PYTHON" )
gp.SelectLayerByAttribute_management(tv,"NEW_SELECTION", "SUBTYPECD = 6")
gp.CalculateField_management(tv, "BC_PHASE",exp2 , "PYTHON" )
gp.SelectLayerByAttribute_management(tv, "NEW_SELECTION", "SUBTYPECD = 7")
gp.CalculateField_management(tv, "ABC_PHASE",exp3 , "PYTHON" )

[AlisonMontgomery]

Hello,

This was such a helpful script to find, and it worked...sort of.

I tried several Calculate Field combinations initially, and they all ran in Python (Python WIN to be exact) just fine, but wouldn't run in ArcMap.

After finding this code, it ran in Python, it ran in ArcCatalog, and then ran in ArcMap. But after one run in ArcMap, it's stopped working. It's fine in ArcCatalog still. I've posted below. Maybe I've missed something in the code.

It gives the error 000539 and says that my CountField is invalid. I know it isn't, as it is an integer, and the fact that it has run perfectly fine everywhere else. I also know it's not the GME software, which closes out prior to arcpy being used, and again, since it has run before.

Any ideas would be helpful.

#Alison Montgomery - Spatial Point Allocation Script
#Created: 06.13.2013
#For use with ArcGIS 10.x, Python Version 2.7.2

import arcpy, sys, os, subprocess
from arcpy import env

#Supply the following arguments prior to running:
#Workspace (full path using backslashes)
#Catchment Polygons (full path, backslashes)
#Point Data (full path using backslahes)
#Count Field for the output: <15 characters to denote the point dataset.
#An output txt file (full path, must be backslashes -> eg. C:\Users\Alison\Desktop\filename.txt)
#Density Field for output: <8 characters
#Area Field from Shapefile
########
#Each argument must be in double quotes, and they must be separated by a space.

#Input fields that will be written into text file for GME processing.
inputPoly = arcpy.GetParameterAsText(0)
inputPoint = arcpy.GetParameterAsText(1)
Field = arcpy.GetParameterAsText(2)

code = 'countpntsinpolys(poly="' + inputPoly + '", pnt="' + inputPoint + '", field="' + Field + '");'

#Input a text file name, write above code to text file.
newFile = arcpy.GetParameterAsText(3)
newFileObj = open(newFile, 'w')
newFileObj.write(code)
newFileObj.close()

#Write textfile name into code line readable by GME call.
lineCall = "run(in=\\\""'' + newFile + ''"\\\")"

#Call GME, execute text file code, "-c" closes GME after processing.
os.system(r'C:\Program Files (x86)\SpatialEcology\GME\SEGME.exe')
subprocess.call(r'C:\Program Files (x86)\SpatialEcology\GME\SEGME.exe -c ' + lineCall + ';');

#Input name for Density field, and input the shapefile's area field.
DensityField = arcpy.GetParameterAsText(4)
CountField = Field
area = arcpy.GetParameterAsText(5)

#Add the new density field to shapefile.
arcpy.AddField_management(inputPoly, DensityField, "FLOAT", "10", "4", "", "", "NULLABLE", "NON_REQUIRED", "")

#Calculate density field using count of points, divided by the area field.
arcpy.CalculateField_management(inputPoly, DensityField, '(!'+CountField+'!/!'+area+'!)', "PYTHON_9.3", "")