topic Re: splitting up large shapefile in Geoprocessing Questions

splitting up large shapefile

SaraThompson2 — Wed, 10 Oct 2012 17:16:31 GMT

Hello,

I have a large point shape file (112,000 points) that I need to temporarily split up. I need to split it up in to small sections so I can run this tool that will not process the lump sum of points, and I am thinking if i give the tool less points to process, it will actually function. I can't write script or code, and am wondering if there is a simple way to do this without ruining my shapefile forever. Thanks, help greatly appreciated 🙂

Re: splitting up large shapefile

JonathanQuinn — Wed, 10 Oct 2012 18:36:00 GMT

Do you have any values within within the attributes of the points that can be grouped? If so, you can use the Iterate Feature Selection tool within Modelbuilder to iterate through each group, make the selection, run any other tools. If you don't have a field to group values on, you can create a fishnet that will cover the extent of your data and then use the Iterate Feature Selection tool again to select each grid, use the Select by Location to select points that fall within the grid, process the data, then use the Collect Values tool to collect the output. Finally, use the this model in a separate model and use the output of the collect values tool as the input to a Merge tool to merge the data back together.

Here's a image of a sample workflow:
[ATTACH=CONFIG]18338[/ATTACH]

The larger model was just dragged into a new model from ArcCatalog. The last output of the submodel is a parameter so it can exposed within the main model.

Re: splitting up large shapefile

DaleHoneycutt — Thu, 11 Oct 2012 15:18:21 GMT

Take a look at the Split tool -- it might be just what you need.

In addition, there's a sample tool in the Analysis and Geoprocessing Gallery on arcgis.com: http://www.arcgis.com/home/item.html?id=db1e328dce6948d08ee42a2b52a17daa

Re: splitting up large shapefile

SaraThompson2 — Thu, 11 Oct 2012 16:40:14 GMT

Ah thanks for your help! I don't have access to model builder or split, no arcinfo license. However, it's a pain but I've been using the clip tool, and making assigned polygons, then running the tool on the smaller clumps of points in those polygons, then I'll merge them back together. Thanks for the help again!

Sara

Re: splitting up large shapefile

DaleHoneycutt — Thu, 11 Oct 2012 16:46:45 GMT

You should have access to ModelBuilder -- it's available at every license level

Re: splitting up large shapefile

EricRice — Thu, 11 Oct 2012 17:45:41 GMT

Sara,

What tool is having a problem with 112,000 points? I routinely work with tens of millions of points, sometimes billions.

Best,
Eric

Re: splitting up large shapefile

SaraThompson2 — Fri, 12 Oct 2012 15:42:27 GMT

I was using the multiple minimum distance tool (an esri addin) because I can't access the analysis near tool without an arcinfo license, but it turns out i just had to use another computer, and the tool worked flawlessly on this computer! So this whole effort was not even necessary I just needed a better computer.

Hurray for success!
Sara