My experience with these two tools is that Intersect runs much faster than Spatial Join. For 100 files, intersect took about 40 seconds, while Spatial Join took almost 4 minutes. Why the difference in processing time? It seems like Spatial Join is doing a more simplified overlay, and should take less time.
I'm using ArcGIS 10 and running the analysis on shapefiles.
I tried importing the shapefiles to a geodatabase, but it actually took look longer for the analysis (I only tested on the Intersect), another thing that puzzled me.
An update on this: I was running the script from a created toolbox in ArcMap...I have since tried running the scripts from the command line on my computer, and they run much faster, possibly as much as 5 times faster.
Interesting. Would you be able to provide me with a reproducible case for this so I can take a look? I don't see this behavior in testing here ... so maybe you're data is triggering something that we haven't hit before. I'd need everything to be able to run the case ... the toolbox with script tool and the data. If the data is large it can be uploaded to our ftp server (let me know if you need instructions).