Spatial join with 50 million observations...

Discussion created by ajhart109 on Aug 3, 2011
Latest reply on Aug 5, 2011 by ajhart109

I am using ArcGIS 9.3.1, and I am trying to match 50 million addresses that I have already geocoded to a polygon feature class.  I am using the GP tool Spatial Join, but after working with a tiny subset of my data, I think that this could take a month on the whole dataset...

Does anyone know of useful techniques I could use to speed up the process?  I have 8 processors and 16 GB of RAM on my machine, so I am thinking of splitting the two tables I'm joining into eight pieces each and running 8 simultaneous spatial joins in separate ArcMap windows, one for each thread.  Or can the "Add spatial index" geoprocessing tool help me here?  I am relatively new to ArcGIS and I only need to perform this process once, so I am trying to avoid writing a Python script, though if someone happened to have a premade solution for 9.3.1 that I could tweak, that would be very helpful.

Thank you very much for your time,

Andrew Tschirhart