Handling large dataset for spatial join

SyedAdil — Wed, 13 Sep 2023 15:17:34 GMT

hello everyone. I have a csv file which is approximately 6gb and has 80 million rows and I want to convert it to a shapefile so that I can do a spatial join with another feature layer. however, I had to split it into multiple files and do that. the process is extremely manual and slow and I was hoping if there was a faster and more automated way to do this. my current workflow is as follows:

1) Split csv file to multiple files ~ 80 files

2) Upload each file one by one to arcmap and convert them to a shapefile

3) do a spatial join between each shapefile created with the input feature.

4) Export the resulting shapefile to a csv file

Thank you in advance for your help!

Re: Handling large dataset for spatial join

SLouq — Wed, 13 Sep 2023 15:20:52 GMT

Have you tried converting them to a feature class instead of a shapefile? A feature class allows more attributes than a shapefile

topic Handling large dataset for spatial join in ArcGIS Pro Questions

Handling large dataset for spatial join

Re: Handling large dataset for spatial join