Rounding occurring in PCS values - improper result when entering exact values

[JoelCusick]

Hello,

I've re-discovered a rounding error when entering parameters in a PCS properties. It is highlighted here on the 2nd to last page of this document from 2008 and I confirmed this phenomena in ArcGIS 10.2.1 today.

This is a place where I would prefer no rounding to occur out to the precision level, and would assume, any decimal after my most significant value would be "zero".

Any help to turn this feature off?  Or report as bug?

http://www.survbase.com/downloads/survbase-pmrail-grs80-coord.pdf

[larryzhang · Accepted solution]

joel,

Have you contacted ESRI via the Remedy (local ESRI support) for investigation?

Through the Remedy, it will help ESRI and you to log, track, even fix?

[JoelCusick]

good idea.  Maybe old school, but with previous forums, someone from ESRI would review, and add a NIMBUS if the error was confirmed.   We have a corporate liscence, so will go that route.

though this has not been answered as an error, you have helped.  Thanks Larry

[MelitaKennedy]

Here's the text from the email I sent you, Joel:

I tried a few different places, and I was seeing the correct, rounded versions in display UI, like in ArcMap, and if I used the definition for a DGN or a point shapefile. If I copy and modify it in ArcMap, the scale factor is the same (not rounded to 1.000004) but the latitude of origin is displaying at 61.0.

We've known about it. The projection engine itself has code to correctly save and copy values like this. The problem comes in from not being able to store 1.00004 properly in binary so displaying the value "correctly" means extra code to look for issues like this. Some ArcGIS UI has it but other don't.

Melita