Calculating Field with Python problem

592
8
10-24-2011 02:17 AM
RaphaelR
Occasional Contributor II
Hi,

i just came across something odd:
i tried calculating fields with python and noticed that lots of records don´t get calculated when i try to divide something.
so to test this, nothing was selected, and i just entered 1 / 2. some records correctly showed 0.2 but many still showed 0.
when i tried 1+1 all records show 2.

switching to vbscript also helped.
0 Kudos
8 Replies
DanPatterson_Retired
MVP Esteemed Contributor
You might have bigger problems ... 1/2 isn't 0.2
But I suspect you did 1/5, in any event, you performed integer division if one or both of your values weren't floating point ie, in Python


>>> print 1/5
0
>>> print 1/5.0
0.2
>>>
0 Kudos
RaphaelR
Occasional Contributor II
You might have bigger problems ... 1/2 isn't 0.2


lol, yes i meant 1/5.
but still if i add a field (type double) and try to do a field calculation as mentioned above, shouldn´t all the records be calculated in the same way?
0 Kudos
DanPatterson_Retired
MVP Esteemed Contributor
I would actually click on the Clear Selected Features button to ensure that you have no selected records, don't rely on visuals
0 Kudos
RaphaelR
Occasional Contributor II
the button is greyed out and the table says 0 records selected.
0 Kudos
DanPatterson_Retired
MVP Esteemed Contributor
I have never seen that behaviour before, nor have I seen it reported.  Does it occur in other files?
0 Kudos
ChrisFox3
Regular Contributor II
I think Dan's original answer still applies here even though you are using Double fields to perform this calculation it is still possible that the 2 values you are dividing are both integers. Try this calculation instead:

float(!field1!)/!field2!


This will ensure that you are always using float values in your calculation.
0 Kudos
RaphaelR
Occasional Contributor II
thank you both for your suggestions, in the future i´ll try to be a bit more exact with specifiying what want to calculate.

in the meantime i found out that the shapefile i did the calculations with was created with another software (VISUM, a traffic modelling program), maybe that´s the reason. i also couldn´t reproduce this with a couple of other arcgis-created shapefiles.
0 Kudos
CharlesNichols1
New Contributor
I was using the Raster Calculator tool in ArcToolBox.  I used the NDVI formula:   (NIR - Red) / (NIR + Red) and instead of getting a range of DN values in the resulting raster between -1 and +1, I got only the integer values of -1, 0, 1.  In searching ESRI Help, I figured out that the software was interpreting the input raster DN values as integers, so the output would be in integers.

I saw the code that you showed - thank you!  It gave me the idea to incorporate the 'Float' function into the formula in ArcToolBox's Raster Calculator tool.  It worked!!
0 Kudos