topic Re: Distance between related points in Geoprocessing Questions

Distance between related points

SamuelFonseca — Wed, 05 Dec 2012 15:55:18 GMT

Here is the thing, I have two point shapes, one containing ???proposed??? sampling stations and the second one with the ???real??? coordinates where the samples were taken. The proposed and real points are linked together by a common code (5K_ID). What I need is to calculate the distance from all points in the ???proposed??? dataset to the corresponding point (by 5K_ID) in the ???real??? dataset. Any help would be much appreciated ???

Re: Distance between related points

markdenil — Wed, 05 Dec 2012 17:28:17 GMT

use GenerateNearTable using the same FC for both in_features and near_features and the 'ALL' option on closest.
That give you a (potentialy very big) table of distances between every point and every other.
You can also get the bearing angle too, if you want.

PointDistance may work for you too.

You may want to make selections on the 5K_ID field before running it, to get (many) smaller tables to work with.

you need an advanced license to use GenerateNearTable or PointDistance

Without that:

AddXY_management will give you locations for each point.
You could then do some trig on the X,Y location differences between pairs of matched points

Re: Distance between related points

SamuelFonseca — Wed, 05 Dec 2012 17:52:40 GMT

use GenerateNearTable using the same FC for both in_features and near_features and the 'ALL' option on closest.
That give you a (potentialy very big) table of distances between every point and every other.
You can also get the bearing angle too, if you want.

PointDistance may work for you too.

You may want to make selections on the 5K_ID field before running it, to get (many) smaller tables to work with.

you need an advanced license to use GenerateNearTable or PointDistance

Without that:

AddXY_management will give you locations for each point.
You could then do some trig on the X,Y location differences between pairs of matched points

Thx for the answer, I have already done what you suggested, but as you pointed out I get a huge table containing distances to all the possible combinations and the process takes a lot to execute, Now I need either a way to calculate distances only to the corresponding points or a way to filter the result table to get only those results I need.

Re: Distance between related points

DanLee — Wed, 05 Dec 2012 21:30:47 GMT

Here is a workaround idea:

1. Make sure the two sets of points are in the same projection/coordinate system; then use the Merge tool to combine them into one input.
2. Use the Points To Line tool with the 5k_ID as the Line Field to obtain lines between each pair of proposed and real points. The line output carries the 5k_ID field and the Shape_Length values would be the distances you need. For shapefile output, you would need to add a field and then calculate the shape length, using the Add Field and Calculate Field tools.
3. Run the Join Field tool to join the line output with any of the two point inputs via the common 5K_ID to transfer the Shape_Length field to it.

Does it work for you?

Re: Distance between related points

DarrenWiens2 — Wed, 05 Dec 2012 21:55:29 GMT

Along the lines of Dan's idea, you could Add XY Coordinates to each of your point feature classes, Join one to the other by the common field, then use Geometry to calculate the distance between the two sets of coordinates. As long as your feature classes are in the same projection, I'm pretty sure this is valid (although I'm open to explanations as to why not), but you should double-check a few pairs in ArcMap with the distance tool.

Re: Distance between related points

SamuelFonseca — Thu, 20 Dec 2012 21:46:05 GMT

Here is a workaround idea:

1. Make sure the two sets of points are in the same projection/coordinate system; then use the Merge tool to combine them into one input.
2. Use the Points To Line tool with the 5k_ID as the Line Field to obtain lines between each pair of proposed and real points. The line output carries the 5k_ID field and the Shape_Length values would be the distances you need. For shapefile output, you would need to add a field and then calculate the shape length, using the Add Field and Calculate Field tools.
3. Run the Join Field tool to join the line output with any of the two point inputs via the common 5K_ID to transfer the Shape_Length field to it.

Does it work for you?

I´m so far sounds like a good way to get it... i{ll let you know if have any more problems.. THX for the help

Re: Distance between related points

AnthonyGiles — Fri, 21 Dec 2012 10:34:20 GMT

Samuel,

Have you had a look at Hawths analysis tools - Distance between Points tool:

http://www.spatialecology.com/htools/pntdistbetw.php

Regards

Anthony

Re: Distance between related points

NobbirAhmed — Mon, 24 Dec 2012 21:32:13 GMT

If Anthony's or Dan's suggestion hasn't solved your issue - please let me know which version of ArcGIS you are using.

Re: Distance between related points

SamuelFonseca — Wed, 17 Jul 2013 18:53:35 GMT

Samuel,

Have you had a look at Hawths analysis tools - Distance between Points tool:

http://www.spatialecology.com/htools/pntdistbetw.php

Regards

Anthony

In fact I have Anthony but the problem is the same than with the point distance tool... The output tells me the distance to any other point in the shape and I need to know only the distance to one of them. Having more than 5K pints on my shapes makes unpractical to do it in that way.

As for the "point to line" method, it does solve the problem to some extent but it has one important limitation and that is that the two 5K codes used to generate the lines must be identical and here is where my problem gets more complicated...

Let´s add another variable to the original problem. In the original shapes, the two related points were named the same (same 5k_ID) and I was looking for a way to calculate distance from point "a" in shape "A" to shape "a" on shape "B", but now I have another field on my shape (or another table/shape) where the points are related in such a way that the 5_KID´s don match any more. And I must calculate the distance from point �??a�?� on shape �??A�?� to the point �??b�?� on shape �??B�?� and I need to do it once a day for 5000 points...

NobbirAhmed

If Anthony's or Dan's suggestion hasn't solved your issue - please let me know which version of ArcGIS you are using.

I'm on ArcGis 10.1 and have Advanced Lic.

BTW:
Coordinated system and datum�??s are no problem

Re: Distance between related points

NobbirAhmed — Wed, 17 Jul 2013 20:05:50 GMT

Are the 5k_ID values unique for each of the point feature classes?

If possible, could you please share your data (keeping only the 5k_ID field)? You can send the data to my email as well: nahmed@esri.com

It will be easier if you use Python script - do you want that?

Re: Distance between related points

DanLee — Wed, 17 Jul 2013 22:05:58 GMT

If the 5K_ID for related points in two files don't match, are there any other criteria or conditions (spatial or attribute information) that can be used to "match" them? If not, you may need to step back and look into the processes that have caused the unmatched IDs. Sometimes the problem occurs in early processes and may be solved more easily.

Re: Distance between related points

SamuelFonseca — Thu, 18 Jul 2013 21:39:00 GMT

If the 5K_ID for related points in two files don't match, are there any other criteria or conditions (spatial or attribute information) that can be used to "match" them? If not, you may need to step back and look into the processes that have caused the unmatched IDs. Sometimes the problem occurs in early processes and may be solved more easily.

The thing is that it is not a problem in the data. IÂ´ll try to give you some more background:
The points come from two sources of somewhere around 6500 moving sampling points each one. Each data set is created by a different company and they were supposed to be almost identical, but they are not. And we must check on a daily basis how far apart are the two companies taking the â??relatedâ?� measurements.
In the beginning the two companies were using the same codes (5_K) to identify the sites, but now they have different IDÂ´s (5_K and 6_K) and one of them is not doing a good job managing their IDÂ´s so equivalence between 5_K ID and 6_K ID is not always the same but they do provide what I call a Link table telling me which points on the daily results are supposed to be equivalent

Are the 5k_ID values unique for each of the point feature classes?

If possible, could you please share your data (keeping only the 5k_ID field)? You can send the data to my email as well: nahmed@esri.com

It will be easier if you use Python script - do you want that?

A Python script will be just fine!!... and yes, I can share a little data... IÂ´ll attach a excel file with 3 pages, the first one is a subset of my 5_k data, the second page contains the coordinates from what Iâ??ll call the 6_K points and the third page is the link table.
Please note than in the file IÂ´m giving you, the points are in the same (correct) order, but that is not the way it normally is, thats why I need to use the link table.

Any other ideas are still quite welcome!! And before anyone askâ?¦ NO we can not kick any of the two bad companies causing me so much troubleâ?¦ lol

Re: Distance between related points

DanLee — Thu, 18 Jul 2013 23:13:21 GMT

That is an important piece of information.

I would think you can use the link table to assign the equivalent 5_k IDs to the points with 6_K IDs; and then use the 5_k IDs in both point sets to do what you need to do.

If that's the case, try Join Field tool to transfer the equivalent 5_K IDs to the points through the common 6_K IDs.

Re: Distance between related points

NobbirAhmed — Sun, 12 Dec 2021 03:10:25 GMT

In case you do not find a way to get a solution using the system tool, here is a python script you can try on.

import arcpy

proposed = r'D:\data\f.gdb\points_A'
original = r'D:\data\f.gdb\points_B'

# create an empty dictionary to store shapes
proposed_points = {}  
with arcpy.da.SearchCursor(proposed, ("5k_ID", "Shape@")) as cursor:
    
    for row in cursor:
        # populate the dictionary
        # use 5k_ID field value as the key of the dictionary
        proposed_points[row[0]] = row[1]  
        
# create another dictionary
original_points = {}
with arcpy.da.SearchCursor(original, ("5k_ID", "Shape@")) as cursor:
    
    for row in cursor:
        # populate the dictionary
        # use 5k_ID field value as the key of the dictionary
        original_points[row[0]] = row[1]  

    
# iterate one of the dictionaries with ID values
for id_5k in proposed_points:
    
    # get proposed and original points with same ID
    orig = original_points[id_5k]
    prop = proposed_points[id_5k]
    
    # now find the distance
    dist = prop.distanceTo(orig)
    
    print dist

Re: Distance between related points

SamuelFonseca — Sun, 12 Dec 2021 03:10:28 GMT

That is an important piece of information.

I would think you can use the link table to assign the equivalent 5_k IDs to the points with 6_K IDs; and then use the 5_k IDs in both point sets to do what you need to do.

If that's the case, try Join Field tool to transfer the equivalent 5_K IDs to the points through the common 6_K IDs.

hmm... sorry but I don�??t get it ... I do the join and then what? I can´t find a way to calculate distance between the two related points, a near analysis will give me the closest point but that point may not be the one I´m interested on. Using distance tools I´ll get the huge distance matrix again...

In case you do not find a way to get a solution using the system tool, here is a python script you can try on.

import arcpy

proposed = r'D:\data\f.gdb\points_A'
original = r'D:\data\f.gdb\points_B'

# create an empty dictionary to store shapes
proposed_points = {}  
with arcpy.da.SearchCursor(proposed, ("5k_ID", "Shape@")) as cursor:
    
    for row in cursor:
        # populate the dictionary
        # use 5k_ID field value as the key of the dictionary
        proposed_points[row[0]] = row[1]  
        
# create another dictionary
original_points = {}
with arcpy.da.SearchCursor(original, ("5k_ID", "Shape@")) as cursor:
    
    for row in cursor:
        # populate the dictionary
        # use 5k_ID field value as the key of the dictionary
        original_points[row[0]] = row[1]  

    
# iterate one of the dictionaries with ID values
for id_5k in proposed_points:
    
    # get proposed and original points with same ID
    orig = original_points[id_5k]
    prop = proposed_points[id_5k]
    
    # now find the distance
    dist = prop.distanceTo(orig)
    
    print dist

That script looks good... i´ll try to look intoit and let you know how it goes... THX a lot!!

Re: Distance between related points

TOMKONG — Tue, 23 Jul 2013 22:38:11 GMT

hmm... sorry but I don�??t get it ... I do the join and then what? I can´t find a way to calculate distance between the two related points, a near analysis will give me the closest point but that point may not be the one I´m interested on. Using distance tools I´ll get the huge distance matrix again...

That script looks good... i´ll try to look intoit and let you know how it goes... THX a lot!!

The 5# post is correct:
1.Add X, Y fields for each feature class, then calculate to add X,Y values for each point.
2.Join two point feature classes by using sampling location ID, export the joined layer.
3.In the joined layer, add one new field (distance), calculate the distance for each pair point by using field calculator based on their X&Y.
If using spatial join, the result may have some errors for some sampling locations.

If points X&Y are same(convert to string value and put X, Y to one field, then summarize this field to find duplicates), they should be duplicate.----it's an another useful application by calculating geometry.