Conversion of Trimble GPS Data to NAD83(1986) in ArcGIS

[KyleHeulitt]

I realize there are other posts on this topic but I'm not understanding the logic of the consensus so I'm hoping that someone can enlighten me.

I'm using a Trimble Geo7X GPS unit and post-processing the data using the base station positions from Trimble's base station list.  Therefore, I am confident that my data is in a newer realization of WGS84 which is essentially equivalent to ITRF00.  Following post-processing I export my data from Pathfinder Office to ArcGIS keeping it in the WGS84 datum.  All of my GIS map documents and data are in NAD83(1986).

I understand that:

NAD83(1986) = original WGS84

NAD(1986) <> current WGS84/ITRF00

In ArcGIS, I either let the program reproject my GPS data "on the fly" or reproject to NAD83(1986).  The consensus seems to be that no datum transformation/shift should be applied in this conversion.  If this is correct, can anyone explain why?  If NAD(1986)<>current WGS84/ITRF00  how is a datum shift not needed?

Thanks!

Kyle

[MelitaKennedy]

I'm sorry that I didn't respond sooner.

Is the NAD83 data really in the original 1986 version? If so, is it using EPSG:4269 or projected coordinate systems that are based on 4269? 

You could use 108190 which approximates the difference between WGS84 at ITRF00 and NAD83. It converts to 4269, but is to CORS96 realization than the original NAD83. At the time we added this transformation, we didn't have a specific CORS96 definition in the software. Depending on when your state (if your data is in a single state), there's an equivalent transformation that converts to NAD83 HARN which may be similar to CORS96. So you might try converting to NAD83, but via the WGS_1984_(ITRF00)_To_NAD_1983_HARN + NAD_1983_To_HARN_XX where XX is the state abbreviation. 

Melita

[KyleHeulitt]

Thanks for the response, Melita!  Sorry I did not specify that I am using a projected coordinate system based on 4269 - usually NJ State Plane or another State Plane coordinate system.  In the case of NJ State Plane I am using: NAD_1983_StatePlane_New_Jersey_FIPS_2900_Feet, which I understand to be the 1986 version.

I have been using 108190 but a colleague suggested that I should not be applying a transformation, citing your responses to this thread: Issues with WGS_1984_(ITRF00)_To_NAD_1983 Datum transformation.  This is what prompted me to post my question.  Your response to that thread seems to indicate that a null transformation should be used to convert NAD83(1986) to WGS84 at ITRF00 - am I misinterpreting?

Thanks again for your help!

Kyle

[FrancesBiles1]

Hi Kyle –

I’ll take a stab at this…

ITRF00 (which is equivalent to WGS84(G1150) at the decimeter level), differs from NAD83(1986) by ~1 to 2+ meters horizontally, depending on your location.

To answer your question, deciding whether to transform your GPS data from ITRF00 to NAD83(1986) all depends on the accuracy requirements of your project.  If 2 m falls within your error budget, then there is likely no need to transform. In this case, just assume ITRF00 = NAD83(1986). Also, consider the accuracy of your existing NAD83(1986) GIS map data. Is it really better than 2 meters?

Or, project from ITRF00 to NAD83(CORS96) and just assume that NAD83(CORS96) = NAD83(1986). In my area, NAD83(1986) differs about 44cm horizontally from NAD83(1992/FBN), NAD83(2007), NAD83(CORS96), and NAD83(2011).  You can check for NGS bench mark datasheets in your project area that list coordinates for multiple NAD83 adjustments to see what these differences are in your area. If it is within your error budget, then try this scenario.

On the other hand, if your project requires high accuracy coordinates you may want to consider applying datum transformations. Here are some options that come to mind:

1.

You can get from ITRF00 to NAD83(1986) by stepping through 2 transformations: 1) ITRF00-->ITRF90, then 2) ITRF90-->WGS84(original). As you mentioned, WGS84(original) -- also known as WGS84(TRANSIT) -- is essentially equivalent to NAD83(1986).

In ArcGIS Desktop 10.7 and ArcGIS Pro 2.4 (not sure about other versions) there is an ITRF00-->ITRF90 transformation, but there is not an ITRF90-to-WGS84(TRANSIT) transform. However, you can create a custom transform in ArcGIS (the parameters are available here) using the “Create Custom Geographic Transformation” tool. After you create the custom transformation and open the ‘Project’ tool, ArcGIS should automatically offer a composite transformation using its existing ITRF00-->ITRF90 + your custom ITRF90-->WGS84(TRANSIT) transformations when you go to select the ‘Geographic Transformation’ method.  After transforming you can then “Project Define” the WGS84(TRANSIT) output to NAD83. 

To sum, ArcGIS does not have an ITRF00-->NAD83(1986) transformation available, so you cannot project your GPS data “on the fly” or using the “Project” tool … unless you create the needed 2^nd ITRF90-->WGS84(TRANSIT) transform.

Keep in mind however, that there is also error inherent in the various transformations methods that may make transforming not worthwhile. I don’t know what the errors associated with these 2 transforms are in your area, but my understanding is that transformations involving NAD83(1986) are problematic and may have distortions >1m (see here).

2.

In ArcMap you can project from ITRF00 to NAD83(HARN) [if in conterminous U.S.] or from ITRF00 to NAD83(2011).  Export the NAD83(HARN) or (2011) coordinates, then use the NGS NCAT tool to convert coordinates to NAD83(1986).  Note that the ITRF00 to NAD83(HARN) or (2011) transform is actually the published transformation parameters for ITRF00 to NAD83(CORS96) at epoch 1997.0, so assuming NAD83(CORS96) = NAD83(HARN) or (2011) will introduce some error. In my locale the differences between these NAD83 realizations are typically 4 to 12 cm, but can be up to .5m. The NCAT tool is used to convert between various NAD83 adjustments, but NAD83(CORS) is not one of the options.

In the same vein, in ArcMap you can convert from ITRF00-->ITRF05-->ITRF08, then from ITRF08 to NAD83(2011). Export the NAD83(2011) coordinates, then use the NGS NCAT tool to convert to NAD83(1986).

3.

If your accuracy needs are <2-3 meters you should consider not only accounting for the datum shift (i.e., the position difference between ITRF00 and NAD83(1986)), but also accounting for position shifts due to crustal motion between the epoch dates associated with the 2 coordinates. For example, the ITRF00 coordinate from your PFO differential correction probably has an associated epoch date of 1997.0. The standard reference epoch associated with WGS84(TRANSIT) and NAD83(1986) is 1984.0, but you would have to research the epoch date associated with your existing GIS datasets to be really accurate.  You can account for both the datum shift and crustal motion (i.e., how much a point moved due to plate tectonics between 1984 and 1997) using the NGS HTDP tool. I think there is also an R package. Using HTDP you can enter a file of coordinates and convert from ITRF00 (epoch 1997.0) to WGS84(TRANSIT) (epoch 1984.0).  However, on the HTDP site when you choose WGS84(original/transit) as the output there is a note saying “NAD83(2011) will be used”. Honestly, I’m not sure what that means but it sounds like the result might not actually be in WGS84(TRANSIT)≡NAD83(1986), in which case you are back to being maybe 1-2 meters off?

This topic gets real deep, real fast, and we haven’t even talked about the vertical! Getting modern GPS data back to NAD83(1986) is tricky and I have not found any good answers to this quest. I would be interested to hear other ideas (and corrections to anything above).

Here are some links that may be handy:

http://proceedings.esri.com/library/userconf/proc15/papers/935_537.pdf

https://desktop.arcgis.com/en/arcmap/latest/map/projections/pdf/geographic_transformations.pdf

https://confluence.qps.nl/qinsy/files/latest/en/138708701/138708702/1/1556890336000/ITRF_Transformat... 

[KyleHeulitt]

Thank you Frances!  I really appreciate your thorough response!  You are not kidding about this topic getting real deep, real fast!  

I think bullet items 1 through 3 in your response are just too complex for my purposes.  I am glad you shared them though - I feel more confident that I understand all options available and they also highlight the complexity of the issue.

So that leaves me with the following choices:

1. No datum transformation.  My understanding is that this may introduce up to 1m of error.
2. ITRF00 to NAD83(CORS96) and just assume that NAD83(CORS96) = NAD83(1986).  I checked the NGS benchmarks in my area (New Jersey) and NAD83(1986) differs about 35cm horizontally from NAD83(1992/FBN), NAD83(2007), NAD83(CORS96), and NAD83(2011).

The GPS units we use are considered "sub-meter" but in optimal conditions we generally see accuracy in the 6-12" (15-30cm) range.  So I think Option 2 is my best option - the 35cm difference between NAD(1986) and NAD(CORS96) mentioned above is close to being within my error budget.  

That being said, it would still be great to hear additional perspectives and also how others are dealing with this issue, so I will not mark any of the responses as the answer just yet.  

Thanks again!

Kyle

[JoelCusick]

Kyle,

Your not kidding it gets thick.  Frances gives the guidance, while I'll take a stab at your original workflow, ie., using a Geo7x and Post Processing against a CORS station.  This is my expertise and have what I think is a workflow that won't get you to 1986 (original NAD83), but holding a CORS published position through a workflow when collecting a position with that bomber system you have.

Assumptions:  You are using TerraSync.   You are using either Trimble Pathfinder office 5.4 or higher.  Ideally, today, your using both TS and PFO at the 5.86/5.85 or 5.9 version.  This ensures the differential correction wizard is optimized.   Finally, you are using a CORS station, not a UNAVCO or local base station.  

Okay, Step 3 above has to be explained further, and I'll take an opposite tact, that holding "NAD83" is paramont to getting data into NAD83 (and the highest form).   I worked with Trimble years ago on this issue of the "Use reference position from base provider" vs "Use reference position from base files".  Trimble has not corrected this error and continues to propogate a NON published position, hindcasting to a version of non-NAD83 that, when converting to NAD83 from that version is going to be wrong. 

This has to be understood that the former "provider" is NOT the published position of a CORS station, but a calculated shift that frankly should never be used and was intended by Trimble to be a stop gap shifter when in 2002, CORS96 datum moved to a new reference frame.   NGS would state - always use the published position for a CORS.  I agree, and hence, when using that Geo7x, the best thing you can do (the right thing) is to use the modern, published position of that ARP for that antenna certified by NGS.  That position is what differential correction means to "hold" the CORS and calculate your rover "difference" from the coordinates you stored in TerraSync and what the CORS stored and differentially held during that same epoch at the base.  You should always use the latter "Use reference position from Base File".  Today, that is NAD83 (2011) epoch 2010.  You get that coordinate from the seeded RINEX file that CORS so elegantly stores in all the RINEX header of the zip file you are retrieving from CORS.  Run the PFO wizard, stalling at the confirmation step, and call up the CORS station, Position and Velocity page, and open the ARP coordiantes for that station. Scroll to NAD83 (2011) and compare the coordinates in the confirmation box and CORS  YOu will see nearly exact coordinates.  (Any difference is usually out to the 5th decimal seconds, way under the resolution of your gps).  Proceed thru differential, and today, your COR file is now "In" NAD83 (2011) Epoch 2010.0.  

By doing this, you skip around what Frances says your in "1997".  Your COR file is now at today's epoch (the day you collected the data) and are now holding NAD83 (2011) at Epoch 2010.  You do have 9 years between your time of collection, but that is the premise of NAD83 (holding at an epoch).

I know, this doesn't get you to NAD83 (1986), but you have to exclusively use "Use reference position from base file" to use the Published coordinate of a CORS.  That is the key to start putting GPS data collected today into the most current form of NAD83.  This is a serious Datum Gate many users are skipping by, and Trimble has not corrected this naming convention and shifting what you should be doing.  Since this issue started in 2002, we are now 2 reference frames past that construct, and, now in ITRF14.  Here is a link to the coordinates for a CORS in AK.  Do this for your CORS you are using.  YOu don't see ITRF97 do you?? https://www.ngs.noaa.gov/cgi-cors/CorsSidebarSelect.prl?site=tsea&option=Coordinates

Once you do these steps, then Use HTDP tool.  I would take the top choice, hindcasting your coordinate back to 1986.  This would be as close to assigning a "plain vanilla" NAD83.  Then, shift that coordinate from NAD83 to WGS84 using a bookeeping Molendensky transform like WGS_1984_To_NAD_1983_1.  

Joel

[KyleHeulitt]

Thanks Joel - I appreciate the response.  I am familiar with the "Use reference position from base files" vs. "Use reference position from base provider" issue.  I have been using "Use reference position from base provider" to keep the COR file in WGS84/ITRF00, exporting from Trimble Pathfinder Office to MS Access or SHP as WGS84, and then converting to NAD83 in ArcGIS using the WGS_1984_(ITRF00)_To_NAD_1983 transformation.  Per the discussion above I believe this conversion results in approximately 35cm of error.  I will look into/try your suggested workflow and report back.  Thanks again for the suggestions!

[JoelCusick]

Kyle,

Okay - You have got me hooked.

You state in original thread (last sentence of first paragraph). "All of my GIS map documents and data are in NAD83(1986)."

What is the source of these original map documents and data.  Lets start there.  Deployment of modern GPS receivers, bound to latest reference frame of WGS84, when selecting "correct realtime" in PFO means you are assuming latest reference frame which starts the game of shifts.  Lets start over.  What is your data really "in" that you need to get back too.

Thanks,

Joel

[KyleHeulitt]

Hi Joel,

That was a bit of an oversimplification.  The truth is that the projects this post relates to stretch back to the early 2000's and we have/use data from various sources (GPS, surveyors, NJDEP, federal agencies, and others sources).  The coordinate system for all of our MXDs are defined as NJ State Plane NAD83(1986).  I believe most NJDEP GIS data is also NJ State Plane NAD83(1986).  For much of the rest of the "old" data there is no metadata indicating the epoch, but NJ State Plane NAD83(1986) has always been assumed.  It seems like getting GIS data with these varied origins/timeframes into a single consistent NAD83 epoch is not even possible.

Thanks,

Kyle

[JoelCusick1]

Kyle,

You are correct.  The rabbit hole (and I've been in it hundreds of times), brings you popping back up from underground way more knowledgeable, but in a situation where it is not possible to project from one reference frame to another for all your data.  You could DEFINE Projection a bunch of your data to HARN, CORS96 or NAD83(2011) and call it good (the easiest recourse if you could tease out the different sources), but the bulk of your data is in one of those three, and probably a very small percentage in NAD83 (1986). 

urvey data collected by surveyors in 90's was HARN (most likely).  GPS data (what kind, but even a bunch of garmin data SAVED to Shapefile in DNRGPS since 2003 are in CORS96 (DNRGPS uses _5), Trimble data, well, done right and PP to CORS was in CORS96 since 2001, then switched to NAD84(2011) in 2010.  .. On and on.  The problem in essence, and were all wrestling with this is the lack of capability for ESRI Feature classes to have more than one feature in different datums, projectrions.  A FC has to be defined one way.  Aggregation sucks, but were all in that boat. Feature level metadata (MAPSOURCE attribute) is about all you can do, or seperate the geodetic data from all else.  

Here is what we are considering in AK.  We will redefine (not project) our bulk of our facility data from "83" to NAD83(2011) when it comes to the year 2022 (the year of a new datum).  we won't harm the good stuff, move just 2 cm on the real early GPS data, and disregard all the late 80's digitizing data (digitized off of a topo). Who cares about the TOPO derived data.  Raster data is another matter, and we have to handle a LIDAR dataset from 2002 differently from 2019 because those two datums are different enough (especially in the vertical that is bound to those) to make a difference.

Was alot of fun. Good luck.