I've found that, due to persistent memory leaks, any kind of serious iterative work with ArcGIS requires a lot of extra effort. If running the same loop consistently crashes on the same iteration, and increasing the size of input data set makes it crash, consistently, on an 'earlier' iteration, then you probably have a memory leak issue.
In my experience, the only way to handle these is to build the analysis as a wrapper script for the operation in python, and then code an additional driver routine to launch that wrapper. For first try, look at the 'os.system' method. Once you've got that down, try the subprocess module.
Bonus? If you can pay the cost of book-keeping, you can launch multiple instances (loop steps), and keep things moving. Penalty? Extra helping of book-keeping.
In my experience: Even with the simplest operations, if you run it 1000 times, you can expect some of them to crash anyway (Error 999999). So do code some logic to keep track of failed evaluations, and generate a list of what needs to be re-rendered down the line.
