Fault Tolerant Python Services -- arcpy

2989
5
Jump to solution
11-16-2015 09:27 AM
JasonTipton
Occasional Contributor III

I have Windows Services that need to run 24x7 (except for reboots). They use the arcpy package and a Floating License. I have run into issues with the service obtaining a license. For example, the license manager might be down.

What I need is to try and import arcpy. If that fails, keep trying every x amount of seconds until it is successful. So, you would think something like this works:

arcpy_imported = False

while arcpy_imported is False:

    try:

        import arcpy

        arcpy_imported = True

    except Exception as ex:

        print ex

        time.sleep(2)

But it does not. You can import arcpy all day long and if it fails the 1st time, it will fail infinitely. I tested this by disconnecting from the network, running "import arcpy" a few times, then connect back to the network and running "import arcpy" a few times.

Traceback:

import arcpy

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

  File "C:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\__init__.py", line 21, in <module>

    from arcpy.geoprocessing import gp

  File "C:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\geoprocessing\__init__.py", line 14, in <module>

    from _base import *

  File "C:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\geoprocessing\_base.py", line 592, in <module>

    env = GPEnvironments(gp)

  File "C:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\geoprocessing\_base.py", line 589, in GPEnvironments

    return GPEnvironment(geoprocessor)

  File "C:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\geoprocessing\_base.py", line 545, in __init__

    self._refresh()

  File "C:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\geoprocessing\_base.py", line 547, in _refresh

    envset = (set(env for env in self._gp.listEnvironments()))

RuntimeError: NotInitialized

It would never succeed with the same python session open. I could open a separate session, and successfully import arcpy, come back to the failing session, and it continues to fail.

This means, that when the LM goes down for whatever reason, no amount of error handling can keep my services running without manually restarting them. Well, OK, I could write something to monitor these services and automatically restart them when this problem arises, but I would rather handle it within the actual service.

Any ideas?

0 Kudos
1 Solution

Accepted Solutions
JasonTipton
Occasional Contributor III

So, this seems ridiculous, but what I have so far is to make a seperate module (let's say foo.py for now) and all it will do is to import arcpy.

Then, call that module by:

while subprocess.call(r'python.exe \\path\to\foo.py') <> 0:

    pass

import arcpy

This will prevent the code from loading arcpy until another process is able to load arcpy. When it finally succeeds, it the process will return 0 and exit the loop. At that point, we assume that it is safe to proceed and then we can import arcpy in our own code.

I would really like to handle this a little better.

EDIT:

I am marking this answer as correct until somebody comes up with a better working solution.

View solution in original post

5 Replies
JasonTipton
Occasional Contributor III

So, this seems ridiculous, but what I have so far is to make a seperate module (let's say foo.py for now) and all it will do is to import arcpy.

Then, call that module by:

while subprocess.call(r'python.exe \\path\to\foo.py') <> 0:

    pass

import arcpy

This will prevent the code from loading arcpy until another process is able to load arcpy. When it finally succeeds, it the process will return 0 and exit the loop. At that point, we assume that it is safe to proceed and then we can import arcpy in our own code.

I would really like to handle this a little better.

EDIT:

I am marking this answer as correct until somebody comes up with a better working solution.

View solution in original post

JasonTipton
Occasional Contributor III

Yeah, I went down that route already. The problem is that it fails on import so "arcpy" doesn't exist so there's nothing to reload.

reload(arcpy)

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

NameError: name 'arcpy' is not defined

It seems to be something cached at a non-python level.

0 Kudos
LukeSturtevant
Occasional Contributor III

Have you tried to capture the error explicitly?

import_arcpy = False

while import_arcpy == False:

     try:

         import arcpy

         import_arcpy = True

     except Exception as e:

         for error in e:

             if error == 'No module named arcpy':

                 continue

Edited: Oh I see in your code that you set arcpy_imported = True first. That way it would never enter your while loop? I think setting it to arcpy_imported = False first would fix your code as you have it.

0 Kudos
JasonTipton
Occasional Contributor III

Sorry, that was supposed to be False. Edited my post to reflect.

And it doesn't seem to matter about the Exception. The Exception is: "RuntimeError: NotInitialized". But once it hits the exception once inside the session, you will never be able to import arcpy without starting a new session.

Added full traceback to my post.

0 Kudos