I have a script that processes ~50,000 records, taking about 30 seconds to process each record. My problem is that my script crashes at a random point, sometimes at the 10th record, 200th record or 1254th record, etc. I wrote this code in 2017, and it never crashed then.
I also see that multiple people had this issue. For example here the op explains that
"I converted tables into dictionaries or lists and carried out the InsertCursor and UpdateCursor functions with pure Python"
Another user reports that applying a patch [ArcGIS (Desktop, Engine) Background Geoprocessing 64-bit Arcpy Exit and Shutdown Patch] fixed the issue. That patch didn't work for me.
I have been trying the same script on multiple machines with ArcMap 10.3.1 and ArcMap 10.5.1. Same random crash at some point in the run.
The quoted op reported this issue on Jun 11, 2015, and after four years it is still going on. Will ArcGIS support actually fix this problem, or should I circumvent this problem by using less of Arcpy like the quoted op, or by starting to use Python for QGIS? In my opinion, It is a great disservice to the ArcGIS users to give them an unreliable library that would fail them after writing lengthy amounts of code that used to work a couple of years ago that became unfunctional after some updates or some other technical problem.
