how can i find points with same x,y data and different z value

Kostas

I am reposting this question which was deleted by my mistake.(but a bit different)

 

I have a shp with over 13.000.000 points with x.y.z data. My data are decimals (333456.765432)and i want them to be integer (333456). After that i want to keep only data with same x,y. I want to find points which are one under the other (vertical). I managed to do that exporting it in csv and writing a script in python 3.4.7. 64bit. However i am using ArcGis Desctop 10.2.2. and this version uses python 2.7. 32 bit I didn't manage to write it at the other version because i am running out of memory ( Richard Fairhurst), although my computer has 8 GB Ram, free space on hard disc and at least good processor.

 

Answering to your questions

! I made them round to make comparing them easier ( Joshua Bixby ), i think it won't be a problem not having accuracy (from .1 to .9), am i wrong?

! how can i make it run without exporting it in csv.

! i cannot figure it out why i cannot write such script in Python 2.7 which my gis is using.

! if i find a solution at that i want to sort them by x,y data and only by pairs which differ 100meter (vertical). If a point is common between two pairs (because it may differ with another point also 100 meters) i want to be written again, as in the line 7 of x y z table. For example

 

i     x    y   z

1   33 42 600

2   33 42 500

3   33 42 200

4   33 42 100

5   56 12 900

6   56 12 800

7   56 12 800

8   56 12 700

9   56 12 200

10 56 12 100

I am posting the code.

 

import math

f=open("C:\\Users\\Desktop\\\\DEM\\Sik\\points_3m.csv","r")

#g=open("C:\\Users\\Desktop\\test.csv","w")

 

 

data={}

 

 

for line in f:

    line=line.split(";")

    x=math.trunc(float(line[0]))

    y=math.trunc(float(line[1]))

    z=math.trunc(float(line[2]))

    if (x,y) in data:

        if data[(x,y)]['max']<z:

            data[(x,y)]['max']=z

        if data[(x,y)]['min']>z:

            data[(x,y)]['min']=z

    else:

        data[(x,y)]={'max':z, 'min':z}

 

 

#print("finished reading\n")

 

 

f.close()

i=0

for key in data:

    zmin=data[key]['min']

    zmax=data[key]['max']

    while zmin<zmax:

        i+=1

        #g.write("OID,X,Y,Z\n")

        #g.write(str(i) + "," + str(key[0]) + "," + str(key[1]) + "," + str(zmin) + "\n")

        print("OID,X,Y,Z\n")

        print(str(i) + "," + str(key[0]) + "," + str(key[1]) + "," + str(zmin) + "\n")

        zmin+=1

    print(str(key[0]) + " " + str(key[1]) + " " + str(zmin) + "\n")

 

 

#print("finished writing\n")

#g.close()

XanderBakker

There are several ways to do this. The nice thing about having spatial information is that you can treat it as such. Since you are evaluating difference in height for the "same" location you could process parts of the file (selected by location). Now this will be a little less slow, however creating a spatial selection might be slow anyway when processing 13 million points (in a shapefile which is I believe slower than a file geodatabase that is properly indexed).

Now what I wonder is this; the 13 million points sound as LiDAR data. If you have a LAS file you can create a LAS dataset and use the LAS Point Statistics as Raster tool (ArcGIS Help (10.2, 10.2.1, and 10.2.2) ) which has an option to extract the Z_RANGE.

If you don't have the LAS file, you can create one using LAStools converting a text file to a LAS file.