Measure resolution tolerance vs. measure field definition

3 weeks ago
New Contributor II

NYSDOT has seen a number of very small overlaps generated by the concurrency API and the Calculate Route Concurrencies (CRC) tool.   We have not been able to uncover an underlying cause in the ALRS network that explain these overlap anomalies.  

A discrepancy between the measure field definition and the LRS Network Resolution and Tolerance Properties was recently pointed out to us.   All of our ALRS network and event feature classes have measure fields defined as Double 38,8.   However, our ALRS network measure precision is set to 7 as shown below.  

I'm trying to figure out if this precision discrepancy between the field definition and the ALRS has the potential to cause the (apparently invalid) route concurrency results.   

Using Roads and Highways 10.7.1.   The Tolerance and Resolution properties of the LRSN_Milepoint network are shown below.   The ALRS is maintained using the NAD83 UTM Zone 18 meters coordinate system.



8 Replies
Esri Contributor

Hello Kevin,

I would look elsewhere for tracking down overlaps identified by CRC.  The numeric precision/scale of 38,8 just means you have a numeric container that can handle lots of decimal precision, but given the XY resolution/tolerance settings in the geodatabase, Roads and Highways is saying that any Mile measure only contains accurate digits out to the 7th decimal (any digit after the 7th can be considered merely noise). 

My colleague @AmitHazra and I have seen data cases where CRC can return unintended overlaps when centerlines of routes get too close (within the M tolerance).  But it would be hard to comment whether that is happening in your situation or whether something else is at play. 


New Contributor

Hi Clive, 

When you say that "any digit after the 7th can be considered merely noise", does that mean the data beyond 7th digit is truncated or is it rounded?  I am working with NYSDOT on this issue, and based on what I'm seeing here, all downstream apps and calculations using R&H data need to be configured to match the Property setting for the ALRS Network, regardless of the datatype of measures in the Enterprise Geodatabase.  If the ALRS property is at 7 decimal precision, then we need to either truncate or round data to 7 decimals.  Do you know which it is?

0 Kudos
Esri Contributor

Hello @KristineBarry 

What I mean is that you will see digits beyond the 7th decimal but those digits are not significant because of the XY resolution of the feature classes in the RH geodatabase.   When calculating numbers, it is always good to perform the math operations with full decimal precision.  And then when reporting measures, the only significant digits in any LRS measure value is out to the 7th decimal place. 

So it's not really about truncating or rounding, it's about performing calculations at full data type precision and then reporting LRS measures only to the most significant digit (7th decimal). 

0 Kudos
Esri Contributor

Hi Kevin - Yes @CliveReece is correct with respect to the field precision/scale versus LRS M tolerance settings. I would also second his suggestion to look more closely at the underlying centerline geometry causing the Calculate Route Concurrency (CRC) geoprocessing tool to return very short overlap sections. The first thing I would suggest is interrogating the geometry of those short segments by dynamically segmenting (Make Route Event Layer) the output of the CRC GP against your network. You'll likely need to zoom to scales at 1:1 are larger to see the issue. Common occurrences of this issue can be seen at shallow angle gores between main lines and ramps, digitization overshoots as well as in cases where concurrent routes were loaded but those routes did not exhibit a complete vertex-vertex match. In all cases, running the LRS Centerline De-Duplication tool causes the centerlines originally used to load routes to collapse into a single short centerline segments since they are within the XY tolerance. Once this happens, the centerline sequence table is also updated so you'll need to trace through the centerline sequence and centerlines to resolve the issues. Here are some examples (Scale: 1:1):

Ramp gore point within XY tolerance:


Routes loaded vertex mismatch on concurrency


-If you need assistance advise contacting technical support or your Advantage Program TA and they can connect you either with an analyst or one of us.

Enterprise LRS Transportation Team



New Contributor II

Amit, Clive, 

I'm grateful to both of you for your quick responses.  They supported my suspicions that Roads and Highways is respecting the LRS Network Resolution and Tolerance Properties regardless of the definitions of the measure fields.   My fear was that some R and H editing operations could create m precision in that eighth database position, but a scan of the calibration point measures and min and max M value did not show M precision beyond the sixth decimal place.  

Good news on this concern but we still have these unexplained CRC and concurrency API results.  We have not found any issues with our centerlines and centerline sequence records in the area in question.   May post more details in a separate article here or get some help from our EAP TA as suggested.     


0 Kudos
New Contributor II

I have some concerns again.  We realized that ArcGIS was only displaying six decimal places for ALRS measure fields but there is 8 decimal places of precision stored in the database. 

ArcMap presents the Measure field for the Calibration Points in ArcMap with 6 decimal places:


SQL Server Management Studio presents the same Calibration Point records with 8 decimal places: 



This is due to a field property setting shown below.



Clive mentioned that "any digit after the 7th can be considered merely noise" but we are using these measure fields in our analysis of the LRS network  (ie. dynamic segmentation, de-normalizing event data, generating state milepoint measures from county milepoint measures, etc)

My sense at this point is that we need to somehow make an update to our network to eliminate the 8th decimal place on measure fields by redefining the measure fields to Double 38,7 (in place of Double 38,8).    Does the Community and Product Team agree that it a best practice to set ALRS measure fields to have the same precision (right of the decimal point) that is set up in the LRS Network Resolution and Tolerance properties?  

0 Kudos
Esri Contributor

Hi Kevin,

For any floating point number in a database (such as a double field type used for measures in Roads and Highways), the number of significant digits stored is going to often be 10 or more.  All operations in Roads and Highways (including the route editing operations) utilize the M tolerance to do the calculations and the precision to determine how many decimals to display in tools such as Identify Routes.

There isn't any need to make the database changes you're discussing above as the software is designed to leverage the M tolerance to do calculations already.  Even though 8 digits are stored, only 7 are used during calculations within Roads and Highways (the 7th digit is used to determine whether to round up or down for the 6th digit for display purposes).

ArcGIS Roads and Highways product team

New Contributor II

Thank you for the additional information and background Nathan.  I do trust that everything inside Roads and Highways is respecting the M tolerance but we need to be aware of and address this extra precision in our other integrated applications that extend Roads and Highways functionality.   

0 Kudos