I found this old post and I am having the same problem as described above.
I am on the latest ArcMap 10.8 - yet with 290k points and 20k objects, the spatial join takes about 20 hours - running on Xeon E5 2680 based cloud.
Appreciate any advice
GDB actually helped for the 290k case - it did process in less then 10 minutes.
However my larger file with 600k points as GDB joint with the same 20k objects in GDB will still not complete.
Current run time is ~30 hours and it is stalling. ArcMap allocated ~254 MB out of 64 GB of server memory.
Just some thoughts