Clear Memory Cache - ArcPy

Business_IntelligenceSoftware — Sun, 12 Dec 2021 07:42:19 GMT

Some of the Python scripts I run can take 24 - 78 hours to complete. During the beginning stages, these scripts consume a reasonable amount of Memory (about 16% - 20%). But the longer they run the more Memory they consume, until Task Manager says that 99% of the Memory is being used. (Just to clarify, this is Memory not CPU). I recently learned about garbage collection in Python, but that has not seemed to fix the problem. I have also tried

arcpy.Delete_management‍‍

but that has not solved the problem either.

Does anyone know how to keep the Memory from continuously building up?

Here is an example of some code where I incorporated garbage collection and arcpy.Delete_management:

import arcpy
import time
import datetime
from datetime import timedelta
import gc

#Document Start Time in-order to calculate Run Time
time1 = time.clock()

#project location
p = arcpy.mp.ArcGISProject(r'C:\arcGIS_Shared\Python\CenterHeatMaps4.aprx')
#Name of layout that the PDFs will be based on
lyt = p.listLayouts("Layout_King")[0]
#second object is the name of the mapframe in the project
mf = lyt.listElements("MAPFRAME_ELEMENT", "Map*")[0]
bkmks = mf.map.listBookmarks()
m = p.listMaps()[0]
#layers to be used in the PDFs
OnLyrList = m.listLayers('Topographic')
OnNxtList = m.listLayers('OpenCenters')
OnList = m.listLayers('SumWithin1000*')


####Loop through all bookmarks until finding a match for the active center

for bkmk in bkmks: 
    
     
        
        
        #Turn off all layers
    lyrList = m.listLayers()
    
    for lyr in lyrList:
        lyr.visible = False
        

    #Turn on desired layers
    for lyr in OnLyrList:
        lyr.visible = True
    for lyr in OnNxtList:
        lyr.visible = True
    for lyr in OnList:
        lyr.visible = True


    #Zoom to active bookmark
    mf.zoomToBookmark(bkmk) 
    
    ### Zoom in or out further from bookmark
    mf.camera.scale = mf.camera.scale * 1.50
    
       
    #Change Layout Title
    for elm in lyt.listElements("TEXT_ELEMENT"):

        elm.text = "Zip Code Out 50 Percent - " + bkmk.name #[22:]
        
           

    #Export to PDF
    lyt.exportToPDF(r"C:\arcGIS_Shared\Python\Export\Geographic_Boundaries" + "\\Zip Code Out 50 Percent " + bkmk.name + ".pdf") #+ "\\NY"

    print(bkmk.name)
     
    ### Clear Memory
    arcpy.Delete_management(r"C:\arcGIS_Shared\Python\Export\CensusTracts")
    del lyrList, OnList, OnLyrList, OnNxtList
    gc.collect()        
               
#Document End Time
time2 = time.clock()

#Run Time in seconds
runtime = (time2-time1)

#Print total run time for the job
print (str(timedelta(seconds=runtime))) ‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Clear Memory Cache - ArcPy

JakeSkinner — Wed, 25 Apr 2018 18:54:44 GMT

Can you post the code you are executing? Here is a helpful link on posting code in GeoNET:

https://community.esri.com/docs/DOC-8691-posting-code-with-syntax-highlighting-on-geonet?sr=search&searchId=a94a6fe4-8c1d-4c4e-9cae-b2f8dca166e6&searchIndex=0

Re: Clear Memory Cache - ArcPy

Business_IntelligenceSoftware — Wed, 25 Apr 2018 19:10:08 GMT

Hi Jake,

Thanks for responding. I have posted a script I use for creating and exporting PDFs. Towards the bottom of the script I try and use the arcpy.Delete_management and the garbage collector.

Re: Clear Memory Cache - ArcPy

DanPatterson_Retired — Wed, 25 Apr 2018 19:18:43 GMT

Not sure what you are actually deleting using Delete_management (hard to follow from the code.

This .... lyt ... appears outside the loop, and it is used to produce the pdf files, so it doesn't ever get deleted nor I suspect things that reference it. Garbage collection, I am sure that you discovered, is absolutely no guarantee that anything gets deleted until it is permitted. If something is being referenced and not deleted, the trash accumulates.

topic Re: Clear Memory Cache - ArcPy in Python Questions

Clear Memory Cache - ArcPy

Re: Clear Memory Cache - ArcPy

Re: Clear Memory Cache - ArcPy

Re: Clear Memory Cache - ArcPy