64-bit ArcPy Not releasing Python Process on Completion

EricFulcher1 · ‎06-11-2020

I'm using a command script to run an arcpy process on ArcMap 10.6 - 64-bit arcpy. On successfully completing the script, the process ends - but doesn't release/kill the python.exe (or pythonw.exe), therefore the bat script hangs and doesn't finish. In the script below, its the arcpy.mapping.ListDataFrames that causes the process to hang (that was fun narrowing down...). Comment it out and python terminates correctly. If i run this in 32 bit python terminates correctly - so just 64 bit. I'm thinking esri bug? Am I missing something?

Thanks,
Eric

Cleaned up script (may have missed something here) but it says the script is Finished.

import arcpy
import string
import gc

def main():
print("Completion Script is Starting")

mxd = arcpy.mapping.MapDocument("D:\WOBufferScripts\LeakSurveyScripts.mxd")
# Running this causes python to not close.
dataFrame = arcpy.mapping.ListDataFrames(mxd)[0]
del dataFrame
del mxd
gc.collect()
print("Completion Script Has Finished")

if __name__ == "__main__":
main()

JoshuaBixby · ‎06-12-2020

64-bit ArcGIS (ArcPy and other software) doesn't support all workspaces that 32-bit ArcGIS does, e.g., personal geodatabases. If the MXD contains one of these workspaces, a 64-bit process can easily hang trying to process it.

View solution in original post

PaulBatley · ‎01-22-2021

Hey, We're getting the same issue for both 32 & 64bit and it only started happening around an installed Windows update KB4593226. Prior to the update our python jobs running via Scheduled Tasks runs through but will not run on the next schedule as the process is already running, due to the fact the process never terminates. We have debugged sys.exit() and it is firing correctly. Our only workaround is to have the following code:
os.kill(os.getpid(),9)
This ensures the process terminates and that batch processing continues.

View solution in original post

PaulBatley · ‎02-23-2021

Hi,
Just to update further. After KB4598243 which superceded the patch mentioned above, the issue started affecting the Interop tool too. With the FME.exe process now not closing down and causing a further problem as the file handles for the logging were inherited and therefore the python Scheduled Task couldn't re-run because of the orphaned FME process. The code below resolved the orphaned FME not shutting down with the Standard signum (,9) kill signal.

## Python for dealing with child processes
parentProcess = psutil.Process(os.getpid())
children = parentProcess.children(recursive=True)
for ofspring in children:
localLogger.write("Child ProcessID: {}".format(ofspring.pid) + " terminating...")
os.kill(ofspring.pid,9)
localLogger.write("Python Process PID: {}".format(os.getpid()) + " terminating...")
os.kill(os.getpid(),9)

View solution in original post

DanPatterson · ‎06-11-2020

You mention batch file. Does the script terminate outside of the batch file?

... sort of retired...

EricFulcher1 · ‎06-11-2020

its a runscript.bat file and no it never returns from the python process.

This is just me typing quick but

arcpy script1.py

arcpy.script2.py

It never gets to script2 even tho script1 is in fact finished.

JoshuaBixby · ‎06-12-2020

How many MXDs have you tried it with? Does it fail on all of them or just this one?

EricFulcher1 · ‎06-12-2020

You have value Joshua! Created new mxd, added 2 layers and the process closed successfully. It was a 10.2.1 mxd originally. wonder what it doesn't like.....

Thank You!

DanPatterson · ‎06-12-2020

I marked Joshua Bixby‌ 's answer correct since he provided the hint as to the check you didn't try

... sort of retired...

EricFulcher1 · ‎06-24-2020

I don't see how you have the POV to mark this correct. Must get reputation points or something.

JoshuaBixby · ‎06-12-2020

64-bit ArcGIS (ArcPy and other software) doesn't support all workspaces that 32-bit ArcGIS does, e.g., personal geodatabases. If the MXD contains one of these workspaces, a 64-bit process can easily hang trying to process it.

EricFulcher1 · ‎06-24-2020

Thanks Joshua. Actually - the mxd has about 20 SDE Feature classes. I found if I rebuild the mxd from scratch (same FCs) - same issue. But by trial and error - if I remove one FC (Oracle SDE GDB) it magically works now. Go figure.....Doubt I'll get esri to debug this. Other FCs in the same feature dataset dont' impact it. weird......

ChrisWalsh · ‎10-12-2020

Is there a way to identify incompatible workspaces? I'm currently running into this similar issue where an automated publishing script running on 64 Bit Arcpy doesn't release it's process. This script has been working for years for multiple mxd's, but has only just now shown this behaviour after we upgraded to ArcServer 10.8.