Cordinate systems and transformations with collector and high accuracy GNSS GPS - best practice

11519
31
Jump to solution
05-22-2018 07:47 PM
LanceCole
MVP Regular Contributor

We have been testing a high accuracy GNSS GPS with Collector (Aurora and 18.0.1) in a development environment and utilizing ArcGIS online as our Portal.  As we plan to move to the production environment questions have arose pertaining to minimizing data translations to maintain accuracy.  The GPS utilizes and outputs in  the GCS NAD 1983 2011 coordinate system from our State VRS network.  We have been translating this to WGS 1984 Web Mercator Auxiliary Sphere for use with ArcGIS Online and its related basemaps.  Our production system also utilizes the GCS NAD 1983 2011 coordinate system, specifically the NAD 1983(2011) State Plane and no translations are needed between the GPS and production servers; however, we plan on continuing to utilize ArcGIS online as our Portal for collector.  What is the best or recommend approach to maintain the collected coordinate data accuracy?

We are considering:

1) generating our own basemaps on our local servers using the GCS NAD 1983 2011 coordinate system and pushing these up to ArcGIS online and sideloading them directly onto our Collector systems (when/if they will fit).  This would save on credits used for building the tiles but we will still be utilizing credits for storage.  Subsequently, we would publish our features utilizing the same coordinate system. This is currently our preferred approach as there would not be any translations required.

2) continue with the current approach of collecting GPS data using GCS NAD 1983 2011, translating to WGS 1984 Web Mercator Auxiliary Sphere on ArcGIS Online and then translating back to the NAD 1983(2011) State Plane we utilize internally.  We have noted a definite shift in the data collected utilizing this method when compared to data collected by our surveyor utilizing a total station or survey grade GPS on our own local network.  I have experimented with different translations but the WGS_1984_(ITRF08) to NAD_1983_2011 we are utilizing provides the best match.  

3) a question for the ESRI team is it possible to utilize the ArcGIS online basemaps without Collector automatically using the WGS 1984 Web Mercator.  We did try and publish a feature using the GCS NAD 1983 2011 coordinate system but the Maps/Collector automatically adopts the basemap coordinate system.  We were also not positive on how the point data was being handled with a mixed coordinate system and most likely introducing further translations and potential errors.  

It is ironic that a few years ago GIS data that was accurate to a couple of meters we considered fantastic and now we are looking at a target accuracy of 10cm and achieving an actual accuracy of less then 2cm

31 Replies
AllisynHudson_Dunn
New Contributor II

Hi Lance.

I am not sure if you are still actively monitoring this thread, but I am impressed with your knowledge and thoroughness in your testing and collection workflows and thought I would post my question in the case that you are. 

I am currently transitioning our inspectors from Trimble TerraFlex to Esri Collector. To ensure we limit the amount of potential error introduced due to unnecessary transformations, I want to make sure I am setting up the profile and the project correctly. Here is the situation that we are facing: 

  • Client DB is in WGS84
  • Client wants our deliverable in NAD83 because the RTK network we are using collects in NAD83
  • Once the client receives the data he then performs another transformation to convert the data back to WGS84
  • The client uses Esri basemaps, as do we. 
  • Our current profile is set up as follows
    • GNSS Coordinate System -- GCS NAD 1983 2011 (6318)
    • Map Coordinate System -- WGS 1984 Web Mercator Auxillary Sphere (3857)
    • Datum Transformation -- WGS_1984_(ITRF00)_To_NAD_1983_2011
  • We are using Trimble R2 units and Iphones
  • We are looking for subfoot accuracy

I have been testing out different scenarios but am not seeing the benefit of any of them. My main concern is how all the transformations will effect the final collection product. Considering the above bulleted info, should my collection database be set to NAD83, or mimic the clients setup in WGS84? It is my understanding the Trimble R2s collect in WGS84 by default, how does this affect all the other moving parts including the fact that the RTK is in NAD83? Also, what are your thoughts about our client's existing workflow? Should the data just be delivered in WGS84 rather than undergoing another transformation? 

I apologize for any confusion, my head spins just trying to wrap my head around all the transformations that are occurring in this process. 

Thank you for any advice you can provide. 

Allisyn

0 Kudos
LanceCole
MVP Regular Contributor

Hi Allisyn‌, 

Two quick questions and I will work on a reply later today.

1) When you say the client wants the deliverable in NAD83 is that NAD83(86) or NAD83(2011).  Also is this GCS or on a different coordinate system such as a State Plane system

2) Are you interested in XY data only or Z data as well?

0 Kudos
AllisynHudson_Dunn
New Contributor II

Hi! 

1.) NAD83(2011)

2.) Just XY data as of now

0 Kudos
LanceCole
MVP Regular Contributor

Allisyn‌, 

Your configuration is very similar to the setup we are currently using for our operations.  Our RTK is running on GCS NAD 1983(2011).   We are using ESRI base maps for field work on the collector units which are WGS 1984 Web Mercator Auxiliary Sphere.  The transformation we are using is WGS_1984_(ITRF00)_To_NAD_1983_2011.  One item you do not note in your question is what coordinate system is your feature data service published to AGOL for collection.  As our enterprise database is using NAD83(2011) Ohio State Plane North, this is what we publish our feature services to AGOL.  AGOL and Collector handle the transformation from GCS NAD 1983(2011) to PCS Ohio State Plane North NAD 1983(2011).  This has worked well for us.


We also have to provide our data to several other agencies in WGS84 and NAD83(1986) so we have projection scripts running to process this information on a nightly schedule and generate deliverable files.  To insure there have not been issues introduced somewhere in the process, we have about 30 control points over a 35 square mile area that at least two control points are collected in the field on a daily basis for each device.   We have a set of confirmed points, for each coordinate system, that the script looks for variance between the data and the controls outside the accuracy of the GPS.

These control points serve two functions:


1) used internally to pick up potential errors in the collection process or equipment
2) for transformation verification


A couple of examples how we used this approach.


One morning we came to find an email from our GIS server that the previous day’s data collected from one device was outside our acceptable limits (substantially).  After reviewing the data and the device it was found that the user had selected the correct GPS but forgot to change the profile and wrong RTK was being utilized. The problem was quickly found and we only lost a day’s production data for that device.  We now run a check on the data as it is posted to AGOL several times a day to catch this sooner.


Another example as when we first attempted to project our data to from NAD83(2011) to NAD83(86) it did not go well.  No matter what we tried the data was always off after translation.  After a little research, it turned out it was too big of a step to go from NAD83(2011) to NAD83(86). We need to add an intermediate step of converting to NAD83(HARN).  However, because we had control points, available in the deliverable datum, this was blatantly obvious before we delivered the data.

In setting up our program, we chose to minimize the number of transformations while collecting and storing the field data and perform post collection transformations to get the data in the form needed.  Each transformation in the field introduces a potential point of error that cannot be undone.  We collect our field data in NAD83(2011) Ohio State Plane North, however, since we also collect our GPS metadata we have the raw GPS data (lat, long and altitude) from our RTK service in GCS NAD 1983(2011), height above elipsoid.

One other factor to consider, in comparing your data to control points and day to day repeatability, is the accuracy of your equipment and correction service.  If you are collecting data to 1 foot accuracy, keep in mind that is typically a plus or minus or a 2 foot diameter circle.  If one day you collect a point that is on the plus 1 side and the next day collect the same point on the minus 1 side.  These two points will be 2 feet apart.  Furthermore, that 1 foot accuracy is if the accuracy was 1 foot and not collected under a tree near a building reducing your accuracy to maybe 2 or 3 feet (a 4 or 6 foot diameter circle).

AllisynHudson_Dunn
New Contributor II

Thank you Lance! This information is incredibly helpful and I really appreciate you taking the time to help me work through this!!  

 

Also, to respond to your note regarding the coordinate system of our feature service -- after some comparative testing, we opted to go with GCS NAD 1983 (2011). So far so good. 

0 Kudos
Kyle_LocGov
New Contributor III

Lance,

I created a hosted feature layer in our local SPCS,OR South NAD83 HARN, and used that layer in Field Maps to collect a +/- 7cm point with a Trimble R2 (using NAD 1983 2011correction) on an aerial control point that should be within 2.5' accuracy. When I view the aerial and my layer together in Pro, there is a 4 foot shift if the correct zero based transform (NAD83 HARN to NAD83 2011) is used but data lines up correctly if the 7-parameter NAD83 HARN to WGS84 + WGS84 (ITRF00) to NAD83 2011 is used. Was the shift you described in #2 that significant? 

0 Kudos
LanceCole
MVP Regular Contributor

My Apologies for the delay.  When ESRI moved from GeoNET to ESRI Community platforms, notifications were disabled on old posts.

We were experiencing a constant 2.2' shift in our northern survey area to a 1.5' shift in the southern portion.  All shifts were to the NW in our data.  As you noted, a multi-step transformation often resolves this issue, especially from older datums.  Many of the older datums do not directly transform to the newer and a intermediary step or steps need to be implemented. You can also manually apply a series of transformations if the precompiled, multistep transformation do not accurately meet your needs.

0 Kudos
Kyle_LocGov
New Contributor III

Thank you very much for the explanation. We are facing similar offsets but I am having a hard time understanding why this shift exists. Do you have an explanation? 

 

I am curious if there is an undocumented process being implemented by hosting non-Web Mercator data in AGOL. If our data is in NAD 83 HARN and hosted online, and I add it to a map with the control imagery which references NAD 1983 2011, I would think they should line up without a transformation that includes WGS.  

0 Kudos
LanceCole
MVP Regular Contributor

Kyle, 

I am not familiar with OR state plane systems nor am I a surveyor, but from a lot of beating my head on the desk, I think I can explain why including WGS works.  You are attempting to directly transform between NAD83 (HARN) and NAD83(2011) which are both NAD83 datums; however, NAD83(2011) is on a different epoch (2010.00).  An epoch is the national adjustment of passive control applied at a point in time.  By applying the NAD83 HARN to WGS84 + WGS84 (ITRF00) to NAD83 2011 transformation you are also including the epoch transformation.  For someone like me, in Northeastern USA, the change is not that significant, but in the Pacific Northwest this would be substantially larger.

Out of curiosity, have you applied the most recent ArcGIS Coordinate System Data to your ArcGIS Desktop or Pro installation.  This is available from your my.esri.com page under downloads, data and content, and will provide the most comprehensive set of transformations with the newest data.

LanceCole_0-1637264620860.png

Just think what we have to look forward to with NGS and NIST to Retire U.S. Survey Foot in December 2022 and New Datums: Replacing NAVD 88 and NAD 83 the end of 2023 with four new terrestrial reference frames and a geopotential datum.   Originally scheduled for 2022 but delayed because of the pandemic. 

0 Kudos
Kyle_LocGov
New Contributor III

Lance, 

That is very good information. Do you know of any supporting documentation identifying that transformation as the preferred step between NAD 83 HARN and NAD 83 (2011), by chance? 

I think I do have the newest Arc CS Data installed--2.9.31593. Lastly, I am not positive, but think the topic at this link could use this information:

https://community.esri.com/t5/arcgis-pro-questions/arcgis-pro-now-has-default-transformations-which/...

 

**edit** This may not be relevant to this topic, as it discusses a technically different transformation, but I thought I would bring it to your attention. this thread in which Melita Kennedy clarifies that WGS_84(ITRF00)_To_NAD_1983 + WGS_1984_(ITRF00)_To_NAD_1983_HARN ends up basically a null transformation, also related to this topic. 

0 Kudos