I have a Python script that converts CAD files to geodatabase feature classes and annotation files, and then merges them all into two line and polygon feature classes at the end. It gathers hundreds of .dwg files from nearly 100 different buildings during the process. I started using the script in an ArcGIS Pro toolbox, and moved to a standalone Python script for testing. I have never been able to get the script to run in it's entirety in ArcGIS Pro or in IDLE. It throws up random errors 1.5 or 2 hours into the process. When I rerun the script without making any changes, I'll get a different random error. The last one was from arcpy.CopyFeatures_management: XML document must have a top level element. ERROR 000021: Failed to create the output annotation feature class.
When I split up the buildings into batches of 20 or so and run the script 5 times into 5 different geodatabases, everything works like a charm. I'm wondering if the geodatabase schema is getting locked, or something is getting caught up in memory. I can't really pin down what's happening.
Anybody experienced something similar with arcpy and geodatabases?