topic Re: Precision vs. XY Resolution in Geodatabase Questions

Precision vs. XY Resolution

Bud — Fri, 03 Jun 2022 22:19:18 GMT

ArcMap 10.7.1.:

I've created a new point FC from scratch using Catalog (using all the default settings).

The General tab mentions "Storage: High Precision":

Whereas the Domain, Resolution and Tolerance tab mentions "XY Resolution":

What's the difference between "Precision" and "XY Resolution".

Thanks.

Re: Precision vs. XY Resolution

Anonymous User — Fri, 03 Jun 2022 22:37:07 GMT

This article has helped me several times to understand this.

High precision allows you to store coordinates to incredible detail, and as far as I know is the default. In the long ago past, high precision was not available and old datasets had to be migrated to to high precision storage. These days it is just the normal state of affairs - no changes needed.

Re: Precision vs. XY Resolution

DerekLaw — Sat, 04 Jun 2022 02:41:25 GMT

Hi @Bud,

> What's the difference between "Precision" and "XY Resolution"

Back in 2007, I wrote a detailed Esri technical paper that discussed in detail how spatial domains were stored in feature classes in a geodatabase. Please take a look,

Understanding Coordinate Management in the Geodatabase

It might be a little deeper than you wanted to know, but in case you were interested. 🙂

Hope this helps,

Re: Precision vs. XY Resolution

Anonymous User — Mon, 06 Jun 2022 19:45:42 GMT

@DerekLawthank you for this post. The information in your paper is extremely helpful - this is the kind of post that makes the community so valuable.

I see that the paper is dated 2007. Could you please comment what, if any, changes have occurred since then?

Re: Precision vs. XY Resolution

VinceAngelo — Tue, 07 Jun 2022 16:18:25 GMT

No real changes to the geodatabase model infrastructure since then.

Note that "precision" refers to both the vertex equivalence comparison tolerance AND the spatial reference coordinate storage depth (though at this point, *all* spatial references are HIGH precision).

- V

Re: Precision vs. XY Resolution

DerekLaw — Tue, 07 Jun 2022 16:41:29 GMT

Hi @Anonymous User,

> Could you please comment what, if any, changes have occurred since then?

As @VinceAngelo stated, there hasn't been any major changes to the geodatabase data model since the technical paper was published (otherwise, we would have updated it).

Hope this helps,

Re: Precision vs. XY Resolution

JTLancer — Thu, 21 May 2026 22:39:48 GMT

I'm curious about defaults for State Plane cooordinate system. We are standing up a brand new Enterprise GDB and paying close attention to spatial references as our old database from which we are migrating data through various practices over the years have may inconsistencies.

The screenshot below shows the different srid values by using the Create Feature Dataset tool and different results by by using different settings. For xycluster_tol we get differnet values when accepting defualts with no values specified in environments vs providing these values.

My main question is ESRI articles state 0.003281 would be the correct default value to use. If this rounded number is not pvided the default value ends up being 0.00328083333333333. Does this have an effect if we keep the repeating decimals vs rounding to 6 decimal places. I'm leaning toward the rounded numbers as a "master" dataset. The numbers input in the environments affect other things like the xyunits calc.

Also if we leave m and z cluster_tol blank it defaults to .001 in meters rather than feet. In this case we won't be using m and z, but I imagine as a best practice or potential future use of m or z, that we would want to keep the inital spatial reference properties the same. Any thoughts or input would be appreciated.

@DerekLaw @VinceAngelo

Understanding Coordinate Management in the Geodatabase link does not seem to be active any more

@DerekLaw

This article has been a good resource for me as well.

Re: Precision vs. XY Resolution

DerekLaw — Fri, 22 May 2026 00:22:35 GMT

Hi @JTLancer,

> Understanding Coordinate Management in the Geodatabase link does not seem to be active any more

Thanks for bringing the broken link to my attention. I'll try and get it fixed.

Meanwhile, I've attached a copy of the technical paper to this post. I wrote it back in 2007. 😉

Hope this helps,