Calculating Field with Python problem

[RaphaelR]

Hi,

i just came across something odd:
i tried calculating fields with python and noticed that lots of records don´t get calculated when i try to divide something.
so to test this, nothing was selected, and i just entered 1 / 2. some records correctly showed 0.2 but many still showed 0.
when i tried 1+1 all records show 2.

switching to vbscript also helped.

[DanPatterson_Retired]

You might have bigger problems ... 1/2 isn't 0.2
But I suspect you did 1/5, in any event, you performed integer division if one or both of your values weren't floating point ie, in Python

>>> print 1/5
0
>>> print 1/5.0
0.2
>>> 

[RaphaelR]

You might have bigger problems ... 1/2 isn't 0.2

lol, yes i meant 1/5.
but still if i add a field (type double) and try to do a field calculation as mentioned above, shouldn´t all the records be calculated in the same way?

[DanPatterson_Retired]

I would actually click on the Clear Selected Features button to ensure that you have no selected records, don't rely on visuals

[RaphaelR]

the button is greyed out and the table says 0 records selected.

[DanPatterson_Retired]

I have never seen that behaviour before, nor have I seen it reported.  Does it occur in other files?

[ChrisFox3]

I think Dan's original answer still applies here even though you are using Double fields to perform this calculation it is still possible that the 2 values you are dividing are both integers. Try this calculation instead:

float(!field1!)/!field2!

This will ensure that you are always using float values in your calculation.

[RaphaelR]

thank you both for your suggestions, in the future i´ll try to be a bit more exact with specifiying what want to calculate.

in the meantime i found out that the shapefile i did the calculations with was created with another software (VISUM, a traffic modelling program), maybe that´s the reason. i also couldn´t reproduce this with a couple of other arcgis-created shapefiles.

[CharlesNichols1]

I was using the Raster Calculator tool in ArcToolBox.  I used the NDVI formula:   (NIR - Red) / (NIR + Red) and instead of getting a range of DN values in the resulting raster between -1 and +1, I got only the integer values of -1, 0, 1.  In searching ESRI Help, I figured out that the software was interpreting the input raster DN values as integers, so the output would be in integers.

I saw the code that you showed - thank you!  It gave me the idea to incorporate the 'Float' function into the formula in ArcToolBox's Raster Calculator tool.  It worked!!