Select to view content in your preferred language

XY Precision to 15 decimal places

4450
10
01-12-2016 08:45 AM
New Contributor

Hi,

I need to create a grid in a file geodatabase. The coordinates of the grid need to be 15 decimal places.

So say I create a grid with the lower left coordinate set to 73.000000000000000, 28.000000000000000, when I click off the feature an then reselect it, the coordinate has changed to 73.000000000000057, 28.000000000000057.

Can anyone tell me why this happens?

Thanks

Paul

1 Solution

Accepted Solutions
Esri Esteemed Contributor

Fifteen decimal places isn't physically possible with an 8-byte IEEE floating-point value (aka "double precision").  The most you can get is 15 total digits (two left and thirteen right).

What you really need to do is locate where this ridiculous requirement is being generated.  Fifteen places in meters is 1 femtometer (inside the atomic nucleus, and at least a hundred billion times more precise than can be captured by a geographic sensor; not even gamma-ray crystallography needs that kind of resolution).  Fifteen places in angular degrees is roughly equivalent to tenths of nanometers (Angstroms, which are used to measure the distance between atoms in molecules -- only ten million times more precise than any geodata can be accurate).

Esri has an entire whitepaper devoted to how coordinate references operate -- Understanding Coordinate Management in the Geodatabase​.  I think you'll find that anything more than 7 places in geodata is a waste of computing resources.

- V

10 Replies
MVP Emeritus

is the first issue

Secondly, are the numbers absolute or can they be scaled ie 73 - 28 becomes 73,000 - 28,000 or similar idea

Esri Esteemed Contributor

Fifteen decimal places isn't physically possible with an 8-byte IEEE floating-point value (aka "double precision").  The most you can get is 15 total digits (two left and thirteen right).

What you really need to do is locate where this ridiculous requirement is being generated.  Fifteen places in meters is 1 femtometer (inside the atomic nucleus, and at least a hundred billion times more precise than can be captured by a geographic sensor; not even gamma-ray crystallography needs that kind of resolution).  Fifteen places in angular degrees is roughly equivalent to tenths of nanometers (Angstroms, which are used to measure the distance between atoms in molecules -- only ten million times more precise than any geodata can be accurate).

Esri has an entire whitepaper devoted to how coordinate references operate -- Understanding Coordinate Management in the Geodatabase​.  I think you'll find that anything more than 7 places in geodata is a waste of computing resources.

- V

MVP Emeritus

I'm looking around to see if the film crew from Punked is around;  15 decimal places?  Wow....

That should just about do it....
New Contributor

Yeah. It's crazy, but this is what the client wants for their products around the world.

Sent from my Samsung device

Esri Esteemed Contributor

Tell them it will cost an extra US\$2Billion to rewrite the GIS to use "long double" and US\$5Trillion to reinvent geodata collectors to get within six orders of magnitude of the required accuracy.

Most metadata collection isn't accurate to even a single meter (5 places in degrees).

- V

MVP Alum

I dunno Vince, he could run into trouble with that request if his client is not in the U.S.  The usage of Trilllion in some parts of the world is different; that amount might not fit in their database....

Trillion may refer to:

Numbers

• Trillion (short scale) (1,000,000,000,000; one million million; 1012; SI prefix: tera-), the current meaning in both American and British English
• Trillion (long scale) (1,000,000,000,000,000,000; one million million million; 1018; SI prefix: exa-), the former meaning in British English and current usage in some non-English-speaking countries

Chris Donohue, GISP

Esri Esteemed Contributor

Ya, ya, "thousand million" and all that rot.  But I hope the US\$ helped clarify my intent

(having spent some time tracking and/or using AU\$ on occasion, I try to strive for clarity).

- V

Regular Contributor

Ha - this sounds like one of those times where it is your job to step in as the professional in the room and put your client on the right path 🙂

MVP Emeritus

Timothy HalesVince Angelo​ could you mark Vince's answer correct.  There appears to be no active Defense Mods... Better still ...move it..