topic Re: Speeding up a large spatial join in Data Management Questions

Speeding up a large spatial join

ThomasCaruso — Tue, 12 Aug 2014 12:26:03 GMT

Hi folks.

I want to join a layer of elevation points (~47 million of them for this given county) to a layer of features (~1700) that I want to know the elevation of. Is there a way to speed this process up or just generally make it less awful? My machine has 8gb of ram and 3+ ghz of processing power and it got through less than 1/6 of the points in about an hour. Do I just need to brute force it and let it run for a day?

Thanks in advance,

Tom

Re: Speeding up a large spatial join

XanderBakker — Tue, 12 Aug 2014 12:39:57 GMT

I wouldn't join those 47 million points with your 1700 features. For these types of analysis it is custom to convert your elevation points to a Digital Elevation Model (DEM). Use Spatial Analyst of 3D Analyst for this purpose.

Depending the type of geometry of your features you can do various things:

For polygons perform a zonal statistics (as table), using your polygons as zones and the DEM as value raster. This will yield statistics (min, max, mean, etc) for each polygon (Spatial Analyst required)
In case of points you can use the Extract Values to Points tool that will retrieve a single elevation for each point (Spatial Analyst required)
For lines I would probably convert them to raster using a unique identifier and do the Zonal Statistics (Spatial Analyst required).

In case you don't have 3D Analyst and/or Spatial Analyst, select relevant features by a select by location, using a small buffer. Export that selection to a separate layer and perform the join...

Kind regards, Xander

Re: Speeding up a large spatial join

ThomasCaruso — Tue, 12 Aug 2014 12:56:41 GMT

Hi Xander,

I actually already have DEM's of the county in question. I am rather new to GIS and was unsure of how to go about this. I downloaded the DEM's online, converted them to raster and then did a raster to point shapefile. Currently I also have a full raster image (converted from the ~30 DEM's I downloaded and mosaic'd together) of the county. Can I perform the extract values to points on that raster image?

Thanks for your advice,

Tom

Re: Speeding up a large spatial join

XanderBakker — Tue, 12 Aug 2014 15:24:39 GMT

Yes, you should be able to do this if you have a Spatial Analyst license available.

More details in the Help topic on "Extract Values to Points (Spatial Analyst)"