Error with decimal places

[DamianMilne]

Hi all, I have a table in a geodatabase. Several of the values in one of my numeric fields is 15.099999999999. I'm want to change it to 15.1. I've tried everything but the value refuses to change. I've tried manual editing, rounding with field calculator, changeing between floating and double formats. Nothing works. Is this a glitch in ArcGIS 10?? Please help. Cheers, Damian

[AlexeyTereshenkov]

Hi Damian,

What is the data type of the field? Float? Do you store this table in a file geodatabase?

I'd suggest to try the following. Create a new field with the Double data type and then use Field Calculator to calculate this field with the values of the one you struggle with. This should give you 15.1 written to the Double field.

What regional settings do you have? ArcMap "likes" English regional settings (go to the Region and Language settings of your operating system to see what symbol is used as the Decimal symbol). What is it?

[DamianMilne]

Hi tereshenkov, thank you for getting back to me.
My field datatype is Double. The table is stored in a file geodatabase. I did exactly as you said, created a new field (type Double) and used field caculator to round the field value. Nothing changed. I have attached a print screen so you can see exactly what I did, as well as my field values (original values 'SpeciesID', calculated values 'SpeciesID3). I re-ran the field calculator with another formular to make sure it was working and it does. My Regional settings are English (Australia) and under 'Customize Regional Options' my decimal symbol is "."

Any other thoughts? Cheers, Damian

[AlexeyTereshenkov]

Hey Damian,

I see. Could you please try to do what I've outlined earlier but in the field calculator window, just use the [SpeciesID] field without any additional calculations? If you've created a field with the default settings, then it should round up this for you.

[FabianBlau]

There seems to be no error at all. The problem is the representation of float/double.
Have a closer look to:

http://en.wikipedia.org/wiki/Floating_point

You will learn, that there is not for all numbers a representation. That means the number 753.1 does not exists for your computer.
You cannot avoid this problem.

[DamianMilne]

Hi tereshenkov, i created the new field (double), with the default settings, and did the calculation you suggested. No joy, still the same issue. Fabl seems to have the correct answer at the moment.
Damian

[TiffanyPuett]

What data format are you working with? SHP, GDB, MDB, SDE, ETC?  I ran into a similar issue when calculating records stored in SDE.  If you are try running the calc in a gdb or mdb featureclass. 

Turned out to be a known issue back then; don't know if the latest sp fixes it or not.

[DamianMilne]

I'm using GDB. I'm still waiting for our department to install the latest service pack to see if that fixes the issue or not.

[KimOllivier]

All you have to do is change the display to round to one decimal place. Leave the values as a double type.

You can right-click on the table column to alter the number of decimal places shown, and the values will be nicely rounded.

As others point out floating numbers are always approximately converted from binary to decimal except for some edge cases where they have exact equivalents.

Some databases support a decimal type where the digits are stored as characters and decimal arithmetic is performed. Only needed for traditional money calculations. I think its a bit odd because even those are rounded if you allow division, but I suppose accountants don't do division! They don't even allow negative numbers.

[DamianMilne]

Changing the display doesn't solve my problem. I need to round the values for analysis purposes. Changing the display doesn't change the underlying value when performing analysis (and yes, i've tried.)