Select to view content in your preferred language

polygon to raster data type and pixel value issue

1455
2
05-12-2011 08:03 PM
LucyHardy
Emerging Contributor
I am trying to convert a polygon feature class (stored in file gdb) to a raster (using the polygon to raster conversion tool in ArcToolbox)�?�.the field in the polygon feature class which I am using to assign the pixel value has only 5 unique values (12, 0.2, 2.68, 0.6 and 0.16) �??the field data type is float, precision set to 3.

When I convert to raster the pixel values with decimal places convert from 3 digits to 10 digits and the value alters e.g. 2.68 becomes 2.680000067 �?? I need it to be exactly 2.68�?�. I know that the source feature class value is correct because if I change the field properties to display the value with 10 decimal places 2.68 become 2.6800000000.

I have tried various creative ways to end up with a floating point raster with the correct values (e.g. creating a field as integer type and multiplying the value by 100 to remove the decimal, creating  raster with pixel values as integer and dividing it by a constant floating point raster with value 100.00) but always end up with the 2.680000067.

Any ideas?  Thanks

Lucy.
0 Kudos
2 Replies
Luke_Pinner
MVP Regular Contributor
Have read of this: floating point issues. It's for python, but it's one of the best floating point representation articles I've come across.

Basically don't worry about it.
0 Kudos
LucyHardy
Emerging Contributor
Hi Luke....

I had come to the same conclusion, i.e. it doesnt really matter, but the article addresses why, so thanks for replying...I am enlightened!

Lucy.
0 Kudos