Fault Tolerant Python Services -- arcpy

JasonTipton

I have Windows Services that need to run 24x7 (except for reboots). They use the arcpy package and a Floating License. I have run into issues with the service obtaining a license. For example, the license manager might be down.

What I need is to try and import arcpy. If that fails, keep trying every x amount of seconds until it is successful. So, you would think something like this works:

arcpy_imported = False
while arcpy_imported is False:
    try:
        import arcpy
        arcpy_imported = True
    except Exception as ex:
        print ex
        time.sleep(2)

But it does not. You can import arcpy all day long and if it fails the 1st time, it will fail infinitely. I tested this by disconnecting from the network, running "import arcpy" a few times, then connect back to the network and running "import arcpy" a few times.

Traceback:

import arcpy

Traceback (most recent call last):   File "<stdin>", line 1, in <module>   File "C:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\__init__.py", line 21, in <module>     from arcpy.geoprocessing import gp   File "C:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\geoprocessing\__init__.py", line 14, in <module>     from _base import *   File "C:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\geoprocessing\_base.py", line 592, in <module>     env = GPEnvironments(gp)   File "C:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\geoprocessing\_base.py", line 589, in GPEnvironments     return GPEnvironment(geoprocessor)   File "C:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\geoprocessing\_base.py", line 545, in __init__     self._refresh()   File "C:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\geoprocessing\_base.py", line 547, in _refresh     envset = (set(env for env in self._gp.listEnvironments())) RuntimeError: NotInitialized

It would never succeed with the same python session open. I could open a separate session, and successfully import arcpy, come back to the failing session, and it continues to fail.

This means, that when the LM goes down for whatever reason, no amount of error handling can keep my services running without manually restarting them. Well, OK, I could write something to monitor these services and automatically restart them when this problem arises, but I would rather handle it within the actual service.

Any ideas?

JasonTipton

So, this seems ridiculous, but what I have so far is to make a seperate module (let's say foo.py for now) and all it will do is to import arcpy.

Then, call that module by:

while subprocess.call(r'python.exe \\path\to\foo.py') <> 0:
    pass
import arcpy

This will prevent the code from loading arcpy until another process is able to load arcpy. When it finally succeeds, it the process will return 0 and exit the loop. At that point, we assume that it is safe to proceed and then we can import arcpy in our own code.

I would really like to handle this a little better.

EDIT:

I am marking this answer as correct until somebody comes up with a better working solution.