Issues with WGS_1984_(ITRF00)_To_NAD_1983 Datum transformation

[JohnDeLoretta3]

Hi,  

Recently been having issues while working with this datum transformation.  My company is working on a project where data comes from Trimble GPS receivers and also survey data.  The survey data is both exported from Trimble Business center and also transformed with PostGIS using Proj4 into local state plane coordinates.  The GPS data is post-processed with Trimble Pathfinder office and exported as shapefiles into the local state plane coordinate system.  The shapefiles are then imported into our SDE which is in WGS84 using the WGS_1984_(ITRF00)_To_NAD_1983 Datum transformation. 

When comparing data exported from PostGIS to the data in our SDE they line up fine in ArcMap, until I apply the WGS_1984_(ITRF00)_To_NAD_1983 Datum transformation to my map document (ArcMap 10.5). We see a 1m shift between the datasets when the datum transformation is applied.

It appears that there is no datum transformation applied to the survey data that is transformed using PostGIS/Proj4.  Digging deeper I found that between WGS84 and NAD83 there is essentially no difference between the two- at least the original iteration of WGS84 and NAD83.  Going even further, I confirmed in Pathfinder Office that there is no transformation between WGS84 and NAD83 (see attachment) .  I noticed that the transformation values applied in ArcGIS using WGS84 using the WGS_1984_(ITRF00)_To_NAD_1983 match the transformation values exactly to what Trimble Pathfinder office uses for NAD 1983 (Conus) CORS96 (see attachments). 

It is at this point where I am questioning everything I know about datums and coordinate systems.

Does anyone have any input on this workflow? Should Survey be exporting using a transformation in Trimble Business Center/Proj4 or should I ignore the datum transformation?

Any input or clarification would be greatly appreciated, I've run myself into an infinite loop.

[MelitaKennedy · Accepted solution]

You're correct that the original NAD83 and WGS84 were considered coincident, but that was 30 years ago and both have moved on. Pun, probably intended. 

WGS84 has had several re-adjustments/realizations to keep it in line with the International Terrestrial Reference Frame (ITRF) until the current version is probably within a cm or so. The realizations are identified by the GPS week that they were defined in: G730, G873, G1150, G1674, G1762. 

Meanwhile, NAD83 is tied to the North American plate, so it's been drifting farther from WGS84 and ITRF over time.The National Geodetic Survey has also done several re-adjustments/realizations: HARN, CORS96, NSRS2007, 2011.

Esri added the WGS84 realizations only recently for ArcGIS Desktop 10.5.1, so every transformation has been to a "generic" WGS84 version. To compound issues, unless you're US military or contractor or ally, you don't get accurate WGS84 coordinates. You either have general coordinates, or if the data went through any RTK or post-processing, it was aligned with the control points that were used--usually either an ITRF system or some realization of NAD83. At that point, the data is in the same coordinate system as the control point network, but software sometimes doesn't say that.

Melita

[JohnDeLoretta3]

Thanks Melita! I was really hoping you would see this! So basically, anything in or out of WGS84 is "generic" or only on the GPS receiver end?  I should still maintain use of the WGS_1984_(ITRF00)_To_NAD_1983 Datum transformation though, right? Should we just eliminate use of WGS84 in our workflow?  I'm worried that we've introduced a 1m shift into our data because GIS adds the transformation when converting to and from WGS84 and it seems all other programs (Trimble Business Center/ PostGIS/Proj4/ AutoCAD) do not. 

[MelitaKennedy]

If you have control of the transformation (using Project Tool or on-the-fly in a map, or setting the default transformation in geoprocessing environment), you could use NAD_1983_To_WGS_1984, which is basically a null transformation.

[JohnDeLoretta3]

I'm not seeing the NAD_1983_To_WGS84 transformation, do you just mean NAD_1983_To_WGS_1984_1?

[MelitaKennedy]

Yes, sorry about that.

[MitchJohnson]

Hi Melita,

Can you explain why one would use this "Null" transformation VS. another?  Currently, I am working with county coordinates in HARN and want to project wgs84 data to this coordinate system.  The "Null" transformation is about 3 feet different than the "_2" or "_3" transformations.  How should one decide what to use?  The ESRI transformations .pdf describes the area of use as "HARN" for the "Null" transformation and "CONUS" for the "_2" and "_3" transformations.  My guess is that "_2" or "_3" should be used since I am in the contiguous US, but it seems there is no way to be completely certain which is more accurate for the data.

Thank you,

Mitch

[MelitaKennedy]

Hi Mitch, 

There isn't an easy solution. You have to take into account the data accuracy, currency (how recent is the data), and so on. Current "WGS84" data (really ITRFxx, unless you're military) is about 1 m offset from the various NAD83 realizations. Can you match the WGS84 data against the HARN data with the 2 transformations? Which one fits better?

Melita

[MitchJohnson]

Hi Melita,

Thanks so much for responding!

I don't think I understand how to match the data of different datums -- I thought the purpose of the transformation was to establish that fit you describe.  The problem is that there are multiple fits (at least 2 in my case) and I am not sure how to choose between them since it is all relative.  I thought one transformation would be a better fit based on where you are in the world, but that does not seem to be the case.

If current wgs84 data is 1m offset, wouldn't that suggest an "accurate" transformation to HARN should shift about 1m?  Thus, if it's not shifting when you change datums to HARN then it is displaying inaccurately?  I may be thinking about this the wrong way.

Edit: Let's assume data accuracy is sub-foot and it is collected in 2019.

Thanks again,

Mitch

[JoelCusick]

May I make a suggestion for times like these.  Much like our tidal flats up here in Alaska, the more you struggle the deeper you get sucked into a place that is very hard to climb out of.  

Here is the way out.  Occupy a high quality survey control. You mentioned TBC - you likely have survey grade GPS so, drop a nail outside your door, occupy for 2 or 4 hours, run OPUS and when you have that sheet, occupy the same point with any GPS.  In ArcGIS, start a fresh dataframe in your reference frame.  Add to that frame an unadulterated coordinate from OPUS( you have lat/long in both reference frames and utm/state plane in NAD83 (2011). Then drop your exported data into that frame.  Quietly test your on-the-fly transformations.  You will know very quickly what is biting you and what is not.  

Joel