nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Rounding occurring in PCS values - improper result when entering exact values

JoelCusick

Hello,

I've re-discovered a rounding error when entering parameters in a PCS properties. It is highlighted here on the 2nd to last page of this document from 2008 and I confirmed this phenomena in ArcGIS 10.2.1 today.

This is a place where I would prefer no rounding to occur out to the precision level, and would assume, any decimal after my most significant value would be "zero".

Any help to turn this feature off? Or report as bug?

http://www.survbase.com/downloads/survbase-pmrail-grs80-coord.pdf

AfterApply.jpg

BeforeApply.jpg

Find more posts tagged with

Accepted answers

larryzhang

joel,

Have you contacted ESRI via the Remedy (local ESRI support) for investigation?

Through the Remedy, it will help ESRI and you to log, track, even fix?

All comments

MelitaKennedy

Here's the text from the email I sent you, Joel:

I tried a few different places, and I was seeing the correct, rounded versions in display UI, like in ArcMap, and if I used the definition for a DGN or a point shapefile. If I copy and modify it in ArcMap, the scale factor is the same (not rounded to 1.000004) but the latitude of origin is displaying at 61.0.

We've known about it. The projection engine itself has code to correctly save and copy values like this. The problem comes in from not being able to store 1.00004 properly in binary so displaying the value "correctly" means extra code to look for issues like this. Some ArcGIS UI has it but other don't.

Melita

JoelCusick

good idea. Maybe old school, but with previous forums, someone from ESRI would review, and add a NIMBUS if the error was confirmed. We have a corporate liscence, so will go that route.

though this has not been answered as an error, you have helped. Thanks Larry

larryzhang

joel,

Have you contacted ESRI via the Remedy (local ESRI support) for investigation?

Through the Remedy, it will help ESRI and you to log, track, even fix?

Join the Conversation

Signed in members can post, follow updates, and more. New here? Register a free account.

Quick Links

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.