Exposing Arcpy Python Script to Web

[JamieJackson]

Hi Folks,

I've got some Python code that takes an address as an input, calls the Bing geocoding service, then does some geoprocessing on the location, then outputs the geoprocessing results.

I set up IIS so that the code runs as a CGI, and this works fine, except it doesn't work well under the load of multiple concurrent users.

Problems:
1. For every request, a new arcpy is imported, etc., and each one of the resulting processes uses ~80MB of RAM.
2. Even one process pegs the CPU at 100% for a while, so running more than one really bogs things down.

So, questions:

A. How can I get this thing to perform better, with respect to concurrent requests?
B. Should I be looking at something other than CGI?

Thanks,
Jamie

[JasonScheirer]

There's WSGI, which is an open standard that allows you to develop in Django, or Pyramid, or web.py, or flask, or any of a number of other options. Flask or web.py are probably your best bet for porting from CGI.

If you run a web application that handles multiple requests in the same process (that is, running a Python WSGI server behind your IIS instance) then you don't incur the overhead of importing arcpy over and over.

[JamieJackson]

There's WSGI, which is an open standard that allows you to develop in Django, or Pyramid, or web.py, or flask, or any of a number of other options. Flask or web.py are probably your best bet for porting from CGI.

If you run a web application that handles multiple requests in the same process (that is, running a Python WSGI server behind your IIS instance) then you don't incur the overhead of importing arcpy over and over.

Thanks a lot for the tip.

I chose Flask. I made my way through its integration, but I ran into some hurdles, and now a brick wall:

If I run my script stand-alone from the command line, everything goes okay, and I get the output I would expect.

However, if I start up the werzeug server, and run the same script in the Flask context, it gets to a line that calls RefreshCatalog on a CSV file, and then dies silently. If I try/catch around that line, and print the error, I get a useless 99999 error:

[INDENT]

Error: ERROR 999999: Error executing function.

[/INDENT]

FWIW, yes, the CSV is present, and readable/writable. (As I mentioned, the same script runs if called directly.)

Finally, I'm on ArcGIS 10.x, and I happen to be using a virtualenv based on ArcGIS's python installation.

Let me know if you have any ideas.

Thanks,
Jamie

[olivierthomas]

Hi Jamie,

Do you succeed with arcpy + flask?
I would like to test this web solution, but I'd like to have some background experience on this before starting. The goal is to create a simple map export with arcpy.mapping through a web browser.

Thanks,

Olivier.