Measure resolution tolerance vs. measure field definition

2745
9
Jump to solution
06-02-2021 07:29 AM
KevinHunt
New Contributor III

NYSDOT has seen a number of very small overlaps generated by the concurrency API and the Calculate Route Concurrencies (CRC) tool.   We have not been able to uncover an underlying cause in the ALRS network that explain these overlap anomalies.  

A discrepancy between the measure field definition and the LRS Network Resolution and Tolerance Properties was recently pointed out to us.   All of our ALRS network and event feature classes have measure fields defined as Double 38,8.   However, our ALRS network measure precision is set to 7 as shown below.  

I'm trying to figure out if this precision discrepancy between the field definition and the ALRS has the potential to cause the (apparently invalid) route concurrency results.   

Using Roads and Highways 10.7.1.   The Tolerance and Resolution properties of the LRSN_Milepoint network are shown below.   The ALRS is maintained using the NAD83 UTM Zone 18 meters coordinate system.

 

KevinHunt_0-1622643321389.png

1 Solution

Accepted Solutions
NathanEasley
Esri Regular Contributor

Hi Kevin,

For any floating point number in a database (such as a double field type used for measures in Roads and Highways), the number of significant digits stored is going to often be 10 or more.  All operations in Roads and Highways (including the route editing operations) utilize the M tolerance to do the calculations and the precision to determine how many decimals to display in tools such as Identify Routes.

There isn't any need to make the database changes you're discussing above as the software is designed to leverage the M tolerance to do calculations already.  Even though 8 digits are stored, only 7 are used during calculations within Roads and Highways (the 7th digit is used to determine whether to round up or down for the 6th digit for display purposes).

-Nathan
ArcGIS Roads and Highways product team

View solution in original post

9 Replies
CliveReece
Esri Contributor

Hello Kevin,

I would look elsewhere for tracking down overlaps identified by CRC.  The numeric precision/scale of 38,8 just means you have a numeric container that can handle lots of decimal precision, but given the XY resolution/tolerance settings in the geodatabase, Roads and Highways is saying that any Mile measure only contains accurate digits out to the 7th decimal (any digit after the 7th can be considered merely noise). 

My colleague @AmitHazra and I have seen data cases where CRC can return unintended overlaps when centerlines of routes get too close (within the M tolerance).  But it would be hard to comment whether that is happening in your situation or whether something else is at play. 

-Clive

KristineBarry
New Contributor

Hi Clive, 

When you say that "any digit after the 7th can be considered merely noise", does that mean the data beyond 7th digit is truncated or is it rounded?  I am working with NYSDOT on this issue, and based on what I'm seeing here, all downstream apps and calculations using R&H data need to be configured to match the Property setting for the ALRS Network, regardless of the datatype of measures in the Enterprise Geodatabase.  If the ALRS property is at 7 decimal precision, then we need to either truncate or round data to 7 decimals.  Do you know which it is?

0 Kudos
CliveReece
Esri Contributor

Hello @KristineBarry 

What I mean is that you will see digits beyond the 7th decimal but those digits are not significant because of the XY resolution of the feature classes in the RH geodatabase.   When calculating numbers, it is always good to perform the math operations with full decimal precision.  And then when reporting measures, the only significant digits in any LRS measure value is out to the 7th decimal place. 

So it's not really about truncating or rounding, it's about performing calculations at full data type precision and then reporting LRS measures only to the most significant digit (7th decimal). 

0 Kudos
AmitHazra
Esri Contributor

Hi Kevin - Yes @CliveReece is correct with respect to the field precision/scale versus LRS M tolerance settings. I would also second his suggestion to look more closely at the underlying centerline geometry causing the Calculate Route Concurrency (CRC) geoprocessing tool to return very short overlap sections. The first thing I would suggest is interrogating the geometry of those short segments by dynamically segmenting (Make Route Event Layer) the output of the CRC GP against your network. You'll likely need to zoom to scales at 1:1 are larger to see the issue. Common occurrences of this issue can be seen at shallow angle gores between main lines and ramps, digitization overshoots as well as in cases where concurrent routes were loaded but those routes did not exhibit a complete vertex-vertex match. In all cases, running the LRS Centerline De-Duplication tool causes the centerlines originally used to load routes to collapse into a single short centerline segments since they are within the XY tolerance. Once this happens, the centerline sequence table is also updated so you'll need to trace through the centerline sequence and centerlines to resolve the issues. Here are some examples (Scale: 1:1):

Ramp gore point within XY tolerance:

AmitHazra_0-1622647485799.png

Routes loaded vertex mismatch on concurrency

AmitHazra_2-1622647538103.png

-If you need assistance advise contacting technical support or your Advantage Program TA and they can connect you either with an analyst or one of us.

-Amit
Enterprise LRS Transportation Team

 

 

KevinHunt
New Contributor III

Amit, Clive, 

I'm grateful to both of you for your quick responses.  They supported my suspicions that Roads and Highways is respecting the LRS Network Resolution and Tolerance Properties regardless of the definitions of the measure fields.   My fear was that some R and H editing operations could create m precision in that eighth database position, but a scan of the calibration point measures and min and max M value did not show M precision beyond the sixth decimal place.  

Good news on this concern but we still have these unexplained CRC and concurrency API results.  We have not found any issues with our centerlines and centerline sequence records in the area in question.   May post more details in a separate article here or get some help from our EAP TA as suggested.     

 

0 Kudos
KevinHunt
New Contributor III

I have some concerns again.  We realized that ArcGIS was only displaying six decimal places for ALRS measure fields but there is 8 decimal places of precision stored in the database. 

ArcMap presents the Measure field for the Calibration Points in ArcMap with 6 decimal places:

KevinHunt_0-1622666366156.png

SQL Server Management Studio presents the same Calibration Point records with 8 decimal places: 

 

KevinHunt_3-1622667250027.png

This is due to a field property setting shown below.

KevinHunt_1-1622666593352.png

 

Clive mentioned that "any digit after the 7th can be considered merely noise" but we are using these measure fields in our analysis of the LRS network  (ie. dynamic segmentation, de-normalizing event data, generating state milepoint measures from county milepoint measures, etc)

My sense at this point is that we need to somehow make an update to our network to eliminate the 8th decimal place on measure fields by redefining the measure fields to Double 38,7 (in place of Double 38,8).    Does the Community and Product Team agree that it a best practice to set ALRS measure fields to have the same precision (right of the decimal point) that is set up in the LRS Network Resolution and Tolerance properties?  

0 Kudos
NathanEasley
Esri Regular Contributor

Hi Kevin,

For any floating point number in a database (such as a double field type used for measures in Roads and Highways), the number of significant digits stored is going to often be 10 or more.  All operations in Roads and Highways (including the route editing operations) utilize the M tolerance to do the calculations and the precision to determine how many decimals to display in tools such as Identify Routes.

There isn't any need to make the database changes you're discussing above as the software is designed to leverage the M tolerance to do calculations already.  Even though 8 digits are stored, only 7 are used during calculations within Roads and Highways (the 7th digit is used to determine whether to round up or down for the 6th digit for display purposes).

-Nathan
ArcGIS Roads and Highways product team

KevinHunt
New Contributor III

Thank you for the additional information and background Nathan.  I do trust that everything inside Roads and Highways is respecting the M tolerance but we need to be aware of and address this extra precision in our other integrated applications that extend Roads and Highways functionality.   

0 Kudos
KevinHunt
New Contributor III

I meant to follow up on this article sooner when the details were fresher in my head but better late than never.

We completed additional testing and closer inspection on this measure precision issue on the NYSDOT Milepoint ALRS and confirmed the notes Nathan and Amit documented here.   Specifically...

1.  The Roads and Highways geoprocessing tools we tested, including Calculate Route Concurrency (CRC), generate results that respect the measure precision set in the LRS Network properties.  (7 in our case)

2.  We did eventually locate a series of very small geometry errors in the Centerline feature class, similar to what is shown in Amit's 6/2/21 post above.   Once these errors were repaired in the ALRS, the small overlaps generated by the concurrency API and the Calculate Route Concurrencies (CRC) tool cleared up. 

This issue turned up last spring while we were working on our "Roadway Data Mart" (RDM) which is being developed to publish the LRS network, events and dynamic segmentation results from NYSDOT's transactional Roads and Highways environment.  One of the things we realized contributed to some confusion and concern is that a new field in the SQL Server 2016 geodatabase created by ArcGIS with type "Double" will create a "Number" field with Precision = 38 and Scale = 8.    As a result, all the measure fields in our transactional ALRS are currently defined as Number 38,8   

In the Roadway Data Mart (RDM), we wanted to completely avoid even the perception of a eighth significant digit right of the decimal point so we redefined all measure fields in the RDM to be Number 38, 7.    We believe this will help mitigate potential data integrity issues related to measures when we use the RDM for both new dynamic segmentation products (ie. LRS event overlay results) and downstream system integrations.