nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Calculating Field with Python problem

RaphaelR

Hi,

i just came across something odd:
i tried calculating fields with python and noticed that lots of records don´t get calculated when i try to divide something.
so to test this, nothing was selected, and i just entered 1 / 2. some records correctly showed 0.2 but many still showed 0.
when i tried 1+1 all records show 2.

switching to vbscript also helped.

Find more posts tagged with

Accepted answers

All comments

CharlesNichols1

I was using the Raster Calculator tool in ArcToolBox. I used the NDVI formula: (NIR - Red) / (NIR + Red) and instead of getting a range of DN values in the resulting raster between -1 and +1, I got only the integer values of -1, 0, 1. In searching ESRI Help, I figured out that the software was interpreting the input raster DN values as integers, so the output would be in integers.

I saw the code that you showed - thank you! It gave me the idea to incorporate the 'Float' function into the formula in ArcToolBox's Raster Calculator tool. It worked!!

RaphaelR

thank you both for your suggestions, in the future i´ll try to be a bit more exact with specifiying what want to calculate.

in the meantime i found out that the shapefile i did the calculations with was created with another software (VISUM, a traffic modelling program), maybe that´s the reason. i also couldn´t reproduce this with a couple of other arcgis-created shapefiles.

ChrisFox3

I think Dan's original answer still applies here even though you are using Double fields to perform this calculation it is still possible that the 2 values you are dividing are both integers. Try this calculation instead:

float(!field1!)/!field2!

This will ensure that you are always using float values in your calculation.

DanPatterson_Retired

I have never seen that behaviour before, nor have I seen it reported. Does it occur in other files?

RaphaelR

the button is greyed out and the table says 0 records selected.

DanPatterson_Retired

I would actually click on the Clear Selected Features button to ensure that you have no selected records, don't rely on visuals

RaphaelR

You might have bigger problems ... 1/2 isn't 0.2

lol, yes i meant 1/5.
but still if i add a field (type double) and try to do a field calculation as mentioned above, shouldn´t all the records be calculated in the same way?

DanPatterson_Retired

You might have bigger problems ... 1/2 isn't 0.2
But I suspect you did 1/5, in any event, you performed integer division if one or both of your values weren't floating point ie, in Python

>>> print 1/5
0
>>> print 1/5.0
0.2
>>>

Join the Conversation

Signed in members can post, follow updates, and more. New here? Register a free account.

Quick Links

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.