topic Re: Floating Point Rasters and Grid Values in ArcGIS Spatial Analyst Questions https://community.esri.com/t5/arcgis-spatial-analyst-questions/floating-point-rasters-and-grid-values/m-p/684610#M10012 <HTML><HEAD></HEAD><BODY><SPAN>Have a read of the </SPAN><A href="http://docs.python.org/tutorial/floatingpoint.html">floating point tutorial</A><SPAN>. It's for python, but don't worry about that, it's one of the best explanations of the floating point representation issue I've come across.</SPAN><BR /><BR /><SPAN>Basically, don't worry about exact values, what you're getting is as close as can be represented with as base 2 (binary) fractions. If you really must have control over this, keep your data as integer (multiply by 10^n where n is the number of decimal places you wish to keep).</SPAN></BODY></HTML> Wed, 21 Sep 2011 00:38:05 GMT Luke_Pinner 2011-09-21T00:38:05Z Floating Point Rasters and Grid Values https://community.esri.com/t5/arcgis-spatial-analyst-questions/floating-point-rasters-and-grid-values/m-p/684609#M10011 <HTML><HEAD></HEAD><BODY><SPAN>I have a floating point grid/raster and will want to make all values rounded to the nearest tenth.</SPAN><BR /><BR /><SPAN>I have converted to integer (added 0.5 for rounding), but the final floating point output grid is not giving me the exact values I want. For e.g. 4676.0000 should become 467.6, but instead I get 467.60006. </SPAN><BR /><BR /><SPAN>Also 4674.00000 should become 467.4 but instead I get 467.399994.</SPAN><BR /><BR /><SPAN>Any help and suggestions?</SPAN><BR /><SPAN>Thanks</SPAN><BR /><SPAN>Sam</SPAN></BODY></HTML> Wed, 14 Sep 2011 20:19:20 GMT https://community.esri.com/t5/arcgis-spatial-analyst-questions/floating-point-rasters-and-grid-values/m-p/684609#M10011 SAMUELAMOAKO-ATTA 2011-09-14T20:19:20Z Re: Floating Point Rasters and Grid Values https://community.esri.com/t5/arcgis-spatial-analyst-questions/floating-point-rasters-and-grid-values/m-p/684610#M10012 <HTML><HEAD></HEAD><BODY><SPAN>Have a read of the </SPAN><A href="http://docs.python.org/tutorial/floatingpoint.html">floating point tutorial</A><SPAN>. It's for python, but don't worry about that, it's one of the best explanations of the floating point representation issue I've come across.</SPAN><BR /><BR /><SPAN>Basically, don't worry about exact values, what you're getting is as close as can be represented with as base 2 (binary) fractions. If you really must have control over this, keep your data as integer (multiply by 10^n where n is the number of decimal places you wish to keep).</SPAN></BODY></HTML> Wed, 21 Sep 2011 00:38:05 GMT https://community.esri.com/t5/arcgis-spatial-analyst-questions/floating-point-rasters-and-grid-values/m-p/684610#M10012 Luke_Pinner 2011-09-21T00:38:05Z Re: Floating Point Rasters and Grid Values https://community.esri.com/t5/arcgis-spatial-analyst-questions/floating-point-rasters-and-grid-values/m-p/684611#M10013 <HTML><HEAD></HEAD><BODY><SPAN>I'll try that and see. Thanks for the link on floating points.</SPAN></BODY></HTML> Wed, 28 Sep 2011 12:12:50 GMT https://community.esri.com/t5/arcgis-spatial-analyst-questions/floating-point-rasters-and-grid-values/m-p/684611#M10013 SAMUELAMOAKO-ATTA 2011-09-28T12:12:50Z