topic Re: Calculate Distance multipoints to multiple Polylines in Geodatabase Questions

Calculate Distance multipoints to multiple Polylines

ChristophBaumeister — Fri, 15 Dec 2017 11:03:22 GMT

Hello everyone,

I have a problem calculating average distances between multiple points to multiple polylines. I have two tables: one is with mapped points, the other is with mapped polylines. The ID number of both tables match. I would like to calculate the average distance of a certain set of points that match to the ID of the certain set of polylines. Example: I would like to calculate the average distance of two homes of one city dweller to three roads in the city he likes most. So, two points (homes) and three polylines (roads) have the same ID since they belong to this unique person. Is there an easy way to calculate the respective distances in ArcGIS? Since I have a set of data with many attributes like this, I cannot calculate it manually by pre-selecting attributes. The near tool does not work.

Could somebody help me to resolve the problem?

Thanks a lot in advance,

Christoph

Re: Calculate Distance multipoints to multiple Polylines

XanderBakker — Fri, 15 Dec 2017 12:22:15 GMT

I noticed that you also posted this question in the Web AppBuilder Custom Widgets space (calculate distance multipoints to multiple polylines ). Where do you want to implement this? What should the end result be (a field in one or both of the participating featureclasses). This can be easily implemented with some Python code, but if the idea is to implement this as a Widget then Python is not the way to do this.

Re: Calculate Distance multipoints to multiple Polylines

ChristophBaumeister — Fri, 15 Dec 2017 14:09:32 GMT

Dear Xander,

Thank you very much for your quick response. Sorry for not posting my question correctly–I am new to the community platform. In order to answer your question: a Python code would be perfect for me–I would like to do statistical analysis with the data after calculating distances to see wether I can find correlations. Hence, a table that shows distances between every set of points to the respective polylines would be fine. Would you have some advice on the Python code?

Thank you,

Christoph

Re: Calculate Distance multipoints to multiple Polylines

XanderBakker — Fri, 15 Dec 2017 14:26:45 GMT

Sure, if you can post a sample of you data we can cook some code that you can use. I assume that a field with the mean distance from the points to the lines is what you are looking for? Are the points Point (the ID that should be used to match the lines repeats for multiple records) or MultiPoints (one unique ID for per feature that contains multiple points)?

Re: Calculate Distance multipoints to multiple Polylines

XanderBakker — Mon, 18 Dec 2017 11:26:13 GMT

So Christoph Baumeister , do have some sample data that you can share? I already created some code, but would like to test it with some data.

Re: Calculate Distance multipoints to multiple Polylines

ChristophBaumeister — Mon, 18 Dec 2017 12:54:49 GMT

Dear Xander,

Thank you very much for your help. I ‘ve created two example files. One represents homes (points), the other represents streets (lines). In order to analyze distances, we would like to be sure that each point matches to the respective line–for example person 2304 has 3 homes and marked two streets (ORIG_FID 1098 &1099). We would like to know the distance between home 1 and the mean center of street no 1098 & 1099, the distance from home 2 to the mean center of street no 1098 & 1099, and the distance between home 3 and the mean center of street no 1098 & 1099. Please also note that person 1821 marked 3 homes but no street (which represents quite well the mixed data we have).

Do you have any advice on the code in Python that can cope with this issue?

Thank you very much in advance,

Christoph

Re: Calculate Distance multipoints to multiple Polylines

ChristophBaumeister — Mon, 18 Dec 2017 12:56:53 GMT

...I would love to upload the original excel file–however the available functions only allow me to attach picture-files

Re: Calculate Distance multipoints to multiple Polylines

XanderBakker — Mon, 18 Dec 2017 13:23:49 GMT

To upload the files, you can edit the original post or reply to this one and click on the line "Use advanced editor" in the upper right corner when editing a response:

When you enter the advanced editor there will be an option in the lower right corner of the editor to attach files to the thread.

It would be better to attach the two featureclasses rather than the Excel file(s). That would avoid having to create a geometry from fields. Also you mention the distance to the center of the street, but shouldn't this be the nearest point on the street?

Re: Calculate Distance multipoints to multiple Polylines

ChristophBaumeister — Mon, 18 Dec 2017 13:48:05 GMT

Dear Xander,

Thank you for the helpful hint. I've uploaded the two files. I prefer excel files since it allows us to manipulate data in various ways (which we will need to do quite intensively in our research project....

Re: Calculate Distance multipoints to multiple Polylines

ChristophBaumeister — Mon, 18 Dec 2017 13:54:45 GMT

In terms of the nearest point of a street: I think this works as well for our research question. However, we need the mean center of both or more streets since we do not want the number of mapped streets to influence the average distance to points in our analysis. I hope that makes sense...

Re: Calculate Distance multipoints to multiple Polylines

XanderBakker — Sun, 12 Dec 2021 10:28:59 GMT

My recommendation would be to use geometries in case they are available. I imported the excel tables into a file geodatabase and created events in order to visualize the data you have.

I created a script to generate some results, which could guide us to define what result you're after and how to store this.See code below:

def main():
    import arcpy

    # settings
    fc_line = r'C:\GeoNet\DistLinePoints\data.gdb\lines_neu'
    fld_person_line = 'person'
    fld_orig_fid_line = 'ORIG_FID'
    fld_x_line = 'POINT_X'
    fld_y_line = 'POINT_Y'

    fc_pnt = r'C:\GeoNet\DistLinePoints\data.gdb\points_s'
    fld_person_pnt = 'person'
    fld_x_pnt = 'POINT_X'
    fld_y_pnt = 'POINT_Y'

    fc_revi = r'C:\GeoNet\DistLinePoints\data.gdb\ValidateLines02'

    sr = arcpy.SpatialReference(4326) # assuming GCS_WGS_1984

    # create list of ids in lines fc and pnts fc
    lst_ids_line = list(set([int(r[0]) for r in arcpy.da.SearchCursor(fc_line, (fld_person_line))]))
    lst_ids_pnt = list(set([int(r[0]) for r in arcpy.da.SearchCursor(fc_pnt, (fld_person_pnt))]))

    # Determine the ids that are in both lists
    lst_ids_both = list(set(lst_ids_line) & set(lst_ids_pnt))
    print "person ids to process:"
    for person in lst_ids_both:
        print " - ", person
    print

    # person ids to skip
    lst_ids_skip = list(set(lst_ids_pnt) - set(lst_ids_line))
    print "person ids to skip (no lines available):"
    for person in lst_ids_skip :
        print " - ", person
    print

    # create dictionary id vs points corresponding to lines
    dct_lines = {}
    flds = (fld_person_line, fld_orig_fid_line, fld_x_line, fld_y_line)
    with arcpy.da.SearchCursor(fc_line, flds) as curs:
        for row in curs:
            person = int(row[0])
            org_fid = int(row[1])
            x = row[2]
            y = row[3]
            pnt = arcpy.Point(x, y)
            pntg = arcpy.PointGeometry(pnt, sr)
            if person in dct_lines:
                data = dct_lines[person]
                data.append([org_fid, pntg])
                dct_lines[person] = data
            else:
                data = [[org_fid, pntg]]
                dct_lines[person] = data


    # create dictionary oid vs [person, pntg]
    flds = ('OID@', fld_person_pnt, fld_x_pnt, fld_y_pnt)
    dct_points = {r[0]: [int(r[1]), arcpy.PointGeometry(arcpy.Point(r[2], r[3]), sr)]
                        for r in arcpy.da.SearchCursor(fc_pnt, flds)}


    # process points
    lst_feat = []
    for person in lst_ids_both:
        lines = []
        lines = dct_lines[person]
        for oid, data in dct_points.items():
            if data[0] == person:
                pntg_pnt = data[1]
                min_dist = 99999
                min_dist_orig_fid = None
                lst_dist = []
                cnt = 0
                for line in lines:
                    cnt += 1
                    orig_fid = int(line[0])
                    pntg_line = line[1]
                    dist = pntg_pnt.angleAndDistanceTo(pntg_line, 'GEODESIC')[1]
                    lst_dist.append(dist)
                    # print " - ", cnt, dist, min_dist
                    if dist < min_dist:
                        pntg_near = pntg_line
                        min_dist = dist
                        min_dist_orig_fid = orig_fid
                print "oid: {0}   person: {1}".format(oid, person)
                print " - min dist: {0}   orig_fid: {1}".format(min_dist, min_dist_orig_fid)
                print " - line points: {0}  mean dist: {1}   max dist: {2}\n".format(len(lst_dist),
                            sum(lst_dist) / float(len(lst_dist)), max(lst_dist))
                polyline = arcpy.Polyline(arcpy.Array([pntg_pnt.firstPoint, pntg_near.firstPoint]), sr)
                lst_feat.append(polyline)

    # store revision lines
    arcpy.CopyFeatures_management(lst_feat, fc_revi)

if __name__ == '__main__':
    main()
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

This results in the following information being printed:

person ids to process:
 -  2304
 -  2305

person ids to skip (no lines available):
 -  1821

oid: 1   person: 2304
 - min dist: 1201.03805599   orig_fid: 1099
 - line points: 25  mean dist: 2472.93651947   max dist: 3463.50017465

oid: 2   person: 2304
 - min dist: 2065.10483882   orig_fid: 1099
 - line points: 25  mean dist: 3408.33879204   max dist: 4573.81500576

oid: 3   person: 2304
 - min dist: 2289.66409625   orig_fid: 1099
 - line points: 25  mean dist: 3747.19318314   max dist: 4891.74059608

oid: 4   person: 2305
 - min dist: 3191.65276967   orig_fid: 1100
 - line points: 23  mean dist: 4244.04063308   max dist: 5040.45046165

oid: 5   person: 2305
 - min dist: 6699.3446908   orig_fid: 1100
 - line points: 23  mean dist: 7721.68587451   max dist: 8449.74244931

oid: 6   person: 2305
 - min dist: 6168.04853814   orig_fid: 1100
 - line points: 23  mean dist: 7218.10916206   max dist: 8018.98016489

oid: 7   person: 2305
 - min dist: 3356.12213621   orig_fid: 1100
 - line points: 23  mean dist: 4526.67225294   max dist: 5352.57178966‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Since I also create lines between the point and "line" features this help to check the result:

The larger points are the original points (houses) and the smaller points are those that represent the streets. The dashed lines connect the house to the nearest street point.

Re: Calculate Distance multipoints to multiple Polylines

ChristophBaumeister — Tue, 19 Dec 2017 12:17:12 GMT

Dear Xander,

Thanks a lot for all your effort. The result is great and works for the sample data. We will now 'translate' the script for our original data set to see, whether we can address everything we are looking for.

Thank you–that's great for the moment!

All the best,

Christoph

Re: Calculate Distance multipoints to multiple Polylines

XanderBakker — Tue, 19 Dec 2017 13:07:08 GMT

Your are welcome. Please post back your findings.

Re: Calculate Distance multipoints to multiple Polylines

ChristophBaumeister — Thu, 21 Dec 2017 12:55:01 GMT

Dear Xander,

we have used your script for our data analysis and it seems to work perfectly. Thank you again! However, we realized that the distance is calculated between

C:\GeoNet\DistLinePoints\data.gdb\points_s' and C:\GeoNet\DistLinePoints\data.gdb\lines_neu'

Is there an easy way you would think of to manipulate your script in order to calculate the nearest distance to the line? I have uploaded a picture (above) to illustrate what I mean (look at the distance in the upper middle [0.000579]).

Let me know what you think. Thanks a lot,

Christoph

Re: Calculate Distance multipoints to multiple Polylines

XanderBakker — Thu, 21 Dec 2017 13:59:08 GMT

In fact it will be more easy, since the script will not have to evaluate the distance to multiple points representing the line, but can use a single polyline to do this. Can you upload a small sample of the lines of the same area to adjust the code and test it?

Re: Calculate Distance multipoints to multiple Polylines

XanderBakker — Sun, 12 Dec 2021 10:29:02 GMT

Never mind, I just converted the points to vertices using ORIG_FID to indicate the separate lines and using the order of the OBJECTID after which I added the field person with the required data.

Since there can still be multiple streets there is not much that changes. See code below:

def main():
    import arcpy

    # settings
    fc_line = r'C:\GeoNet\DistLinePoints\data.gdb\lines_neu2'
    fld_person_line = 'person'
    fld_orig_fid_line = 'ORIG_FID'

    fc_pnt = r'C:\GeoNet\DistLinePoints\data.gdb\points_s'
    fld_person_pnt = 'person'
    fld_x_pnt = 'POINT_X'
    fld_y_pnt = 'POINT_Y'

    fc_revi = r'C:\GeoNet\DistLinePoints\data.gdb\ValidateLines03'

    sr = arcpy.SpatialReference(4326) # assuming GCS_WGS_1984

    # create list of ids in lines fc and pnts fc
    lst_ids_line = list(set([int(r[0]) for r in arcpy.da.SearchCursor(fc_line, (fld_person_line))]))
    lst_ids_pnt = list(set([int(r[0]) for r in arcpy.da.SearchCursor(fc_pnt, (fld_person_pnt))]))

    # Determine the ids that are in both lists
    lst_ids_both = list(set(lst_ids_line) & set(lst_ids_pnt))
    print "person ids to process:"
    for person in lst_ids_both:
        print " - ", person
    print

    # person ids to skip
    lst_ids_skip = list(set(lst_ids_pnt) - set(lst_ids_line))
    print "person ids to skip (no lines available):"
    for person in lst_ids_skip :
        print " - ", person
    print

    # create dictionary id vs points corresponding to lines
    dct_lines = {}
    flds = (fld_person_line, fld_orig_fid_line, 'SHAPE@')
    with arcpy.da.SearchCursor(fc_line, flds) as curs:
        for row in curs:
            person = int(row[0])
            org_fid = int(row[1])
            polyline = row[2]
            if person in dct_lines:
                data = dct_lines[person]
                data.append([org_fid, polyline])
                dct_lines[person] = data
            else:
                data = [[org_fid, polyline]]
                dct_lines[person] = data


    # create dictionary oid vs [person, pntg]
    flds = ('OID@', fld_person_pnt, fld_x_pnt, fld_y_pnt)
    dct_points = {r[0]: [int(r[1]), arcpy.PointGeometry(arcpy.Point(r[2], r[3]), sr)]
                        for r in arcpy.da.SearchCursor(fc_pnt, flds)}


    # process points
    lst_feat = []
    for person in lst_ids_both:
        lines = []
        lines = dct_lines[person]
        for oid, data in dct_points.items():
            if data[0] == person:
                house = data[1]
                min_dist = 99999
                min_dist_orig_fid = None
                lst_dist = []
                cnt = 0
                for line in lines:
                    cnt += 1
                    orig_fid = int(line[0])
                    street = line[1]
                    pntg_street = street.snapToLine(house)
                    dist = house.angleAndDistanceTo(pntg_street, 'GEODESIC')[1]
                    lst_dist.append(dist)
                    # print " - ", cnt, dist, min_dist
                    if dist < min_dist:
                        pntg_near = pntg_street
                        min_dist = dist
                        min_dist_orig_fid = orig_fid
                print "oid: {0}   person: {1}".format(oid, person)
                print " - min dist: {0}   orig_fid: {1}".format(min_dist, min_dist_orig_fid)
                print " - line points: {0}  mean dist: {1}   max dist: {2}\n".format(len(lst_dist),
                            sum(lst_dist) / float(len(lst_dist)), max(lst_dist))
                polyline = arcpy.Polyline(arcpy.Array([house.firstPoint, pntg_near.firstPoint]), sr)
                lst_feat.append(polyline)

    # store revision lines
    arcpy.CopyFeatures_management(lst_feat, fc_revi)

if __name__ == '__main__':
    main()‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Example result (in red the new connections)

Re: Calculate Distance multipoints to multiple Polylines

ChristophBaumeister — Fri, 22 Dec 2017 11:28:41 GMT

Dear Xander,

Thanks a lot again. That's awesome. I wish you a Merry Christmas well in advance,

Christoph

Re: Calculate Distance multipoints to multiple Polylines

XanderBakker — Fri, 22 Dec 2017 14:13:03 GMT

Hi Christoph.Baumeister , You're welcome. Just wondering, did you try it and if so did it yield the results you are looking for? If that is the case can you mark the answer and the "correct answer". If you haven't tested it yet, you can leave it open. Anyways I will be monitoring the thread for any follow-up. Happy Holidays!