arcpy.mapping.ListLayers causes Python script to suddenly exit without any errors

Question

I'm trying to write a program to get all the data sources of all layers in a .mxd map document. I am using Python 2.7.10 with ArcGIS 10.4.1. On some .mxd's it seems to work just fine, while on others, calling ListLayers on line 16 simply causes the Python script to suddenly exit. It doesn't say why it stopped, it doesn't give an error. I have tried placing the function where I call ListLayers in a try except statement and it doesn't throw an exception either. I have put print statements to figure out where the program stops and it always stops right after the call to ListLayers (it seems to never return) Is there a bug with ListLayers or am I using it wrong? import arcpy, os, datetime, sys root_directory = r"M:\path_to_mxd\document.mxd" data_sources = {} # mapping from resource names to an array of paths to resources with that name resource_locations = [] # a running list of all the drive letters and unc locations used in resources def store_data_sources(mxd_path): global data_sources, resource_locations print("signal 0: before MapDocument") mxd = arcpy.mapping.MapDocument(mxd_path) print("signal 1: returned from MapDocument") if mxd: layers = arcpy.mapping.ListLayers(mxd) print("signal 2: returned from ListLayers") if layers: for lyr in layers: print("signal 3: looping through layers") if lyr.supports("dataSource"): print("signal 4: supports dataSource") data_path = lyr.dataSource # keep track of resource drive letters and unc locations splitdrive = os.path.splitdrive(data_path) if not (splitdrive[0] == ""): if not splitdrive[0] in resource_locations: resource_locations.append(splitdrive[0]) # keep a list of all resource paths in the mxd if lyr.name in data_sources: if data_path in data_sources[lyr.name]: continue else: data_sources[lyr.name].append(data_path) else: data_sources[lyr.name] = [data_path] continue del layers del mxd def dump_str_to_arr_hash(hashmap): for key in hashmap: print(" " + key) array = hashmap[key] for string in array: print(" " + string) def str_to_arr_hash_count_items(hashmap): count = 0 for key in hashmap: array = hashmap[key] for item in array: count += 1 return count start_time = datetime.datetime.now() print(start_time) print("") print("Exploring path: '" + root_directory + "'") arcpy.env.workspace = root_directory try: store_data_sources(root_directory) except: print("An exception has occurred!") print("") num_sources = str_to_arr_hash_count_items(data_sources) # number of unique data sources found print("Number of data sources found: " + str(num_sources)) print("") # print the drives that are used by resources print("Drive letters and network paths used by resources:") for loc in resource_locations: print(" " + loc) print("") print("Paths to resources:") dump_str_to_arr_hash(data_sources) print("") end_time = datetime.datetime.now() print(end_time) ‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

bixb0012 · Answer

I ended up creating two lists, one for the actual multiprocessing pool and a second with metadata I created and populated about the jobs I put into the pool.  One of the items I tracked in the metadata pool was the start time of when the job was submitted to the pool.  As I looped over the pool to retrieve results, I would check the jobs that were still running and see when their start time was, and I would calculate elapsed time.  If a running job passed a designated elapsed time, I would issue a terminate command.  I would also use the metadata pool list to check whether an existing process had crashed by keeping a list of PIDs.  If a process had crashed, I would print a warning.

IraKoroleva1 · Answer

Hello Joshua, it seems that I'm working on a very similar process and I'm facing very similar issues, so my last idea is to add time-out to the script. While walking through a list of MXDs, my script is able to skip an MXD and continue running if it hits an error but if an MXD is just taking hours to load, it will keep trying forever. Can you please share an example of a time-out code? Are you using multiprocessing module for that?

bixb0012 · Answer

Last time I ran the script in our data center on a large volume of MXDs, I am fairly certain it was ArcGIS 10.3.1.  Yes, each worker process was given a string path to an MXD and used MapDocument and ListLayers to enumerate the layers.

I think MapDocument has some kind of memory-management issue that shows itself after a process has opened a lot of MXDs.  What is a lot?  It seemed to vary depending on the content of the MXDs.  Sometimes a single worker process could make it through hundreds before running into an issue, sometime less than 25.  I knew it wasn't MXD corruption alone because if I re-submitted the MXD that was being processed when the process hung or crashed, it would work just fine.  Speaking of corruption, it was also a problem.  When processing hundreds of thousands of MXDs from thousands of users over years, it even surprised me what some end users could do in an MXD.  How do you think an MXD with 1,200 layers would perform?

I tried numerous tactics to deal with these issues.  I did worker process pool recycling after so many MXDs, that mostly addressed the first issue, but it didn't address the second issue.  I eventually just settled on a time-out for each worker process.  If a worker process took more than so many minutes, I would kill the process and have the pool spool up a new one.  At the end of the script, I would have a list of MXDs that didn't get processed the first time, and I would re-run them to determine which ones are valid and which ones corrupt.