AnsweredAssumed Answered

Python, arcpy and memory use

Question asked by granth on Aug 12, 2014
Latest reply on Oct 25, 2016 by Dan_Patterson

I have a script which searches subdirectories for mxds and reports on them. Nothing new, except that I am having trouble with memory management.

 

There seems to be a memory leak, although I have tried hard to del variables as I go, and separate the MXD parsing to a separate function in the script so that they are all local variables although I hold a reference to the logging and reporting text files as globals.

 

This has been working well, until I hit a certain MXD in my test folders (I just grabbed a handful of folders off our network drives, so they are real MXDs). This MXD  is 2MB in size and has nothing wrong with it that I can see, although it  has a number of broken links it opens fine in ArcMap, and I can access it fine from the python window using "current", but when I access it with arcpy from my script, memory use jumps up 300-400MB for a moment (peaking at 600MB+). This usually crashes the process even though I have 16GB of RAM. The python process is 32bit.

 

The MXD in question has a broken events layer at the top of the TOC, but I can access it OK from PyScripter (with the same memory jump) if I do so directly and before accessing any other MXDs.  If I open the MXD and use Arcpy in the python window I have no trouble accessing it either. If I have already processed an MXD and am using around 300MB of memory then the spike often kills the python.exe process. Making a copy of the MXD with the events layer deleted seems to work OK in my script, even after processing with other MXDs, suggesting that the events layer itself may be the problem, but I am at a loss what to do about it. I cannot tell if there is a problem until I access it and I can't seem to catch it when there is (it skips my try block and windows reports that python has stopped working).

 

Any ideas of what I can try?

Outcomes