topic Re: Calculating Field with Python problem in Geoprocessing Questions

Calculating Field with Python problem

RaphaelR — Mon, 24 Oct 2011 09:17:40 GMT

Hi,

i just came across something odd:
i tried calculating fields with python and noticed that lots of records don´t get calculated when i try to divide something.
so to test this, nothing was selected, and i just entered 1 / 2. some records correctly showed 0.2 but many still showed 0.
when i tried 1+1 all records show 2.

switching to vbscript also helped.

Re: Calculating Field with Python problem

DanPatterson_Retired — Sat, 11 Dec 2021 21:54:15 GMT

You might have bigger problems ... 1/2 isn't 0.2
But I suspect you did 1/5, in any event, you performed integer division if one or both of your values weren't floating point ie, in Python

>>> print 1/5
0
>>> print 1/5.0
0.2
>>>

Re: Calculating Field with Python problem

RaphaelR — Mon, 24 Oct 2011 10:16:16 GMT

You might have bigger problems ... 1/2 isn't 0.2

lol, yes i meant 1/5.
but still if i add a field (type double) and try to do a field calculation as mentioned above, shouldn´t all the records be calculated in the same way?

Re: Calculating Field with Python problem

DanPatterson_Retired — Mon, 24 Oct 2011 11:07:25 GMT

I would actually click on the Clear Selected Features button to ensure that you have no selected records, don't rely on visuals

Re: Calculating Field with Python problem

RaphaelR — Mon, 24 Oct 2011 11:13:11 GMT

the button is greyed out and the table says 0 records selected.

Re: Calculating Field with Python problem

DanPatterson_Retired — Mon, 24 Oct 2011 16:18:12 GMT

I have never seen that behaviour before, nor have I seen it reported. Does it occur in other files?

Re: Calculating Field with Python problem

ChrisFox3 — Mon, 24 Oct 2011 23:31:07 GMT

I think Dan's original answer still applies here even though you are using Double fields to perform this calculation it is still possible that the 2 values you are dividing are both integers. Try this calculation instead:

float(!field1!)/!field2!

This will ensure that you are always using float values in your calculation.

Re: Calculating Field with Python problem

RaphaelR — Tue, 25 Oct 2011 05:42:45 GMT

thank you both for your suggestions, in the future i´ll try to be a bit more exact with specifiying what want to calculate.

in the meantime i found out that the shapefile i did the calculations with was created with another software (VISUM, a traffic modelling program), maybe that´s the reason. i also couldn´t reproduce this with a couple of other arcgis-created shapefiles.

Re: Calculating Field with Python problem

CharlesNichols1 — Mon, 31 Dec 2012 00:46:23 GMT

I was using the Raster Calculator tool in ArcToolBox. I used the NDVI formula: (NIR - Red) / (NIR + Red) and instead of getting a range of DN values in the resulting raster between -1 and +1, I got only the integer values of -1, 0, 1. In searching ESRI Help, I figured out that the software was interpreting the input raster DN values as integers, so the output would be in integers.

I saw the code that you showed - thank you! It gave me the idea to incorporate the 'Float' function into the formula in ArcToolBox's Raster Calculator tool. It worked!!