topic Re: Issues with WGS_1984_(ITRF00)_To_NAD_1983 Datum transformation in Coordinate Reference Systems Questions

Issues with WGS_1984_(ITRF00)_To_NAD_1983 Datum transformation

JohnDeLoretta3 — Mon, 10 Dec 2018 16:30:41 GMT

Hi,

Recently been having issues while working with this datum transformation. My company is working on a project where data comes from Trimble GPS receivers and also survey data. The survey data is both exported from Trimble Business center and also transformed with PostGIS using Proj4 into local state plane coordinates. The GPS data is post-processed with Trimble Pathfinder office and exported as shapefiles into the local state plane coordinate system. The shapefiles are then imported into our SDE which is in WGS84 using the WGS_1984_(ITRF00)_To_NAD_1983 Datum transformation.

When comparing data exported from PostGIS to the data in our SDE they line up fine in ArcMap, until I apply the WGS_1984_(ITRF00)_To_NAD_1983 Datum transformation to my map document (ArcMap 10.5). We see a 1m shift between the datasets when the datum transformation is applied.

It appears that there is no datum transformation applied to the survey data that is transformed using PostGIS/Proj4. Digging deeper I found that between WGS84 and NAD83 there is essentially no difference between the two- at least the original iteration of WGS84 and NAD83. Going even further, I confirmed in Pathfinder Office that there is no transformation between WGS84 and NAD83 (see attachment) . I noticed that the transformation values applied in ArcGIS using WGS84 using the WGS_1984_(ITRF00)_To_NAD_1983 match the transformation values exactly to what Trimble Pathfinder office uses for NAD 1983 (Conus) CORS96 (see attachments).

It is at this point where I am questioning everything I know about datums and coordinate systems.

Does anyone have any input on this workflow? Should Survey be exporting using a transformation in Trimble Business Center/Proj4 or should I ignore the datum transformation?

Any input or clarification would be greatly appreciated, I've run myself into an infinite loop.

Re: Issues with WGS_1984_(ITRF00)_To_NAD_1983 Datum transformation

MelitaKennedy — Mon, 10 Dec 2018 21:24:33 GMT

You're correct that the original NAD83 and WGS84 were considered coincident, but that was 30 years ago and both have moved on. Pun, probably intended.

WGS84 has had several re-adjustments/realizations to keep it in line with the International Terrestrial Reference Frame (ITRF) until the current version is probably within a cm or so. The realizations are identified by the GPS week that they were defined in: G730, G873, G1150, G1674, G1762.

Meanwhile, NAD83 is tied to the North American plate, so it's been drifting farther from WGS84 and ITRF over time.The National Geodetic Survey has also done several re-adjustments/realizations: HARN, CORS96, NSRS2007, 2011.

Esri added the WGS84 realizations only recently for ArcGIS Desktop 10.5.1, so every transformation has been to a "generic" WGS84 version. To compound issues, unless you're US military or contractor or ally, you don't get accurate WGS84 coordinates. You either have general coordinates, or if the data went through any RTK or post-processing, it was aligned with the control points that were used--usually either an ITRF system or some realization of NAD83. At that point, the data is in the same coordinate system as the control point network, but software sometimes doesn't say that.

Melita

Re: Issues with WGS_1984_(ITRF00)_To_NAD_1983 Datum transformation

JohnDeLoretta3 — Mon, 10 Dec 2018 22:22:24 GMT

Thanks Melita! I was really hoping you would see this! So basically, anything in or out of WGS84 is "generic" or only on the GPS receiver end? I should still maintain use of the WGS_1984_(ITRF00)_To_NAD_1983 Datum transformation though, right? Should we just eliminate use of WGS84 in our workflow? I'm worried that we've introduced a 1m shift into our data because GIS adds the transformation when converting to and from WGS84 and it seems all other programs (Trimble Business Center/ PostGIS/Proj4/ AutoCAD) do not.

Re: Issues with WGS_1984_(ITRF00)_To_NAD_1983 Datum transformation

MelitaKennedy — Tue, 11 Dec 2018 00:30:06 GMT

If you have control of the transformation (using Project Tool or on-the-fly in a map, or setting the default transformation in geoprocessing environment), you could use NAD_1983_To_WGS_1984, which is basically a null transformation.

Re: Issues with WGS_1984_(ITRF00)_To_NAD_1983 Datum transformation

JoelCusick — Tue, 11 Dec 2018 01:33:31 GMT

May I make a suggestion for times like these. Much like our tidal flats up here in Alaska, the more you struggle the deeper you get sucked into a place that is very hard to climb out of.

Here is the way out. Occupy a high quality survey control. You mentioned TBC - you likely have survey grade GPS so, drop a nail outside your door, occupy for 2 or 4 hours, run OPUS and when you have that sheet, occupy the same point with any GPS. In ArcGIS, start a fresh dataframe in your reference frame. Add to that frame an unadulterated coordinate from OPUS( you have lat/long in both reference frames and utm/state plane in NAD83 (2011). Then drop your exported data into that frame. Quietly test your on-the-fly transformations. You will know very quickly what is biting you and what is not.

Joel

Re: Issues with WGS_1984_(ITRF00)_To_NAD_1983 Datum transformation

JohnDeLoretta3 — Tue, 11 Dec 2018 14:40:13 GMT

Thanks Joel, yes it does seem like that would be a good idea to do a compare. I always thought it was a black & white issue- where you apply the 'most accurate' datum transformation when migrating between coordinate systems. I've been applying datum transformations to my data for years and now I'm not sure if that's the correct thing to do. Especially when working with KMZ files from my clients- I guess it's impossible to know whether a datum transformation was applied for their output to a kmz, and thus impossible for me to know if I should apply one taking it out of the KMZ.

Re: Issues with WGS_1984_(ITRF00)_To_NAD_1983 Datum transformation

JohnDeLoretta3 — Tue, 11 Dec 2018 21:38:29 GMT

I'm not seeing the NAD_1983_To_WGS84 transformation, do you just mean NAD_1983_To_WGS_1984_1?

Re: Issues with WGS_1984_(ITRF00)_To_NAD_1983 Datum transformation

JoelCusick — Tue, 11 Dec 2018 22:39:11 GMT

John,

I'm pretty familiar with PFO/TBC and have both down cold depending on what you want your final export to be "in". Then there is the issue of what is your input datum, and how you are correcting the data. Your screen shots from PFO are absolutely correct. TBC also has the same exact ones. But, and I'll emphasize again is there is no way around testing on control set to the output you want to be "in". I've trained hundreds, and every student by the 3rd day doesn't get "it" until they plop their data onto a point in ArcMap and they are 1.5 meters away, while the other students are 10cm away. Your getting bit by thee most common confusion in GIS today. Thank god we got Melita, but you can imagine her inbox from all over the world. Your original question is very specific to handling in TBC, PFO.

So, I don't even want to guess why you want to be in "WGS84". If your in State Plane, your an obvious US citizen, so most common is you want to be in NAD83, and since your using top dollar GPS devices, you most specifically want to be "in" NAD83 (2011) Epoch 2010.0. Thats what you get when you run OPUS or PFO with a CORS base station and you use the reference from Base file. You have with each software, 2 datum gates you must pass thru, and thirdly ask what you are setting in ArcMap. So that is a 3x3 matrix or 9 possible combinations. You can't do this correctly without TRUTH. You must calibrate your GPS work, and Accuracy means doing the right thing with datum. You can't evaluate Accuracy without comparing to TRUTH, and its so darn easy these days to set a nail, occupy with a GPS running 2 hours and comparing.

Oh, and your last comment on KMZ production. Nothing in Google Earth ever is to be trusted. I wouldn't export data from 2,000$ software just to be compliant with GE. Export to NAD83 (2011) and let others suffer with any supposed shifts in GE.

Just some thoughts. Joel

Re: Issues with WGS_1984_(ITRF00)_To_NAD_1983 Datum transformation

MelitaKennedy — Tue, 11 Dec 2018 23:51:26 GMT

Yes, sorry about that.

Re: Issues with WGS_1984_(ITRF00)_To_NAD_1983 Datum transformation

MitchJohnson — Mon, 17 Jun 2019 15:34:18 GMT

Hi Melita,

Can you explain why one would use this "Null" transformation VS. another? Currently, I am working with county coordinates in HARN and want to project wgs84 data to this coordinate system. The "Null" transformation is about 3 feet different than the "_2" or "_3" transformations. How should one decide what to use? The ESRI transformations .pdf describes the area of use as "HARN" for the "Null" transformation and "CONUS" for the "_2" and "_3" transformations. My guess is that "_2" or "_3" should be used since I am in the contiguous US, but it seems there is no way to be completely certain which is more accurate for the data.

Thank you,

Mitch

Re: Issues with WGS_1984_(ITRF00)_To_NAD_1983 Datum transformation

MelitaKennedy — Mon, 17 Jun 2019 20:16:02 GMT

Hi Mitch,

There isn't an easy solution. You have to take into account the data accuracy, currency (how recent is the data), and so on. Current "WGS84" data (really ITRFxx, unless you're military) is about 1 m offset from the various NAD83 realizations. Can you match the WGS84 data against the HARN data with the 2 transformations? Which one fits better?

Melita

Re: Issues with WGS_1984_(ITRF00)_To_NAD_1983 Datum transformation

MitchJohnson — Mon, 17 Jun 2019 20:24:36 GMT

Hi Melita,

Thanks so much for responding!

I don't think I understand how to match the data of different datums -- I thought the purpose of the transformation was to establish that fit you describe. The problem is that there are multiple fits (at least 2 in my case) and I am not sure how to choose between them since it is all relative. I thought one transformation would be a better fit based on where you are in the world, but that does not seem to be the case.

If current wgs84 data is 1m offset, wouldn't that suggest an "accurate" transformation to HARN should shift about 1m? Thus, if it's not shifting when you change datums to HARN then it is displaying inaccurately? I may be thinking about this the wrong way.

Edit: Let's assume data accuracy is sub-foot and it is collected in 2019.

Thanks again,

Mitch