topic Re: Calculating double in python results in integer, while AM calc'ing produces decim in Geoprocessing Questions

Calculating double in python results in integer, while AM calc'ing produces decimals

JanicePoehlman — Mon, 26 Jul 2010 21:05:26 GMT

I'm running a calculation that produces decimals while using the AM field calculator but results in integers while using python. Can you help?

The AM field calculator expression:
minutes = [length_mile] *60 / [speed_limit]

The python expression:
gp.calculatefield_management ("highway", "minutes", "!length_mile!*60/!minutes!","PYTHON")

The field to be calculated is added using gp.addfield_management:
gp.addfield_management (filename, "minutes", "double")
After my python code produced numbers rounded to the integer, I double-checked using the field calculator and the result included the decimals. I need the decimals.

Can anyone help with my programming oversight?
Thank you.

Janice Poehlman
Wisconsin DNR

Re: Calculating double in python results in integer, while AM calc'ing produces decim

ChrisSnyder — Tue, 27 Jul 2010 02:56:18 GMT

Python is kinda weird with division, and will return an integer anytime all the inputs are integer (example: 3/2 = 1). The trick is to just force one of the variables to be a decimal (aka float). (example: 3/float(2) = 1.5). Try this:

gp.calculatefield_management ("highway", "minutes", "!length_mile!*float(60)/!minutes!","PYTHON")

Re: Calculating double in python results in integer, while AM calc'ing produces decim

DanPatterson_Retired — Tue, 27 Jul 2010 09:21:36 GMT

or more simply, put 60.0 in the line to enforce floating point division

Re: Calculating double in python results in integer, while AM calc'ing produces decim

JanicePoehlman — Tue, 27 Jul 2010 13:23:21 GMT

Great, thank you Chris and Dan for your replies and solutions.

Janice Poehlman
Wisconsin DNR

P.s. the application is drive-time analysis for recreation land profiles and guidance for funding priorities.