Exposing Arcpy Python Script to Web

Question

Hi Folks,

I've got some Python code that takes an address as an input, calls the Bing geocoding service, then does some geoprocessing on the location, then outputs the geoprocessing results.

I set up IIS so that the code runs as a CGI, and this works fine, except it doesn't work well under the load of multiple concurrent users.

Problems:
1. For every request, a new arcpy is imported, etc., and each one of the resulting processes uses ~80MB of RAM.
2. Even one process pegs the CPU at 100% for a while, so running more than one really bogs things down.

So, questions:

A. How can I get this thing to perform better, with respect to concurrent requests?
B. Should I be looking at something other than CGI?

Thanks,
Jamie

olivierthomas · Answer

Hi Jamie,

Do you succeed with arcpy + flask? 
I would like to test this web solution, but I'd like to have some background experience on this before starting. The goal is to create a simple map export with arcpy.mapping through a web browser.

Thanks,

Olivier.

JamieJackson · Answer

There's WSGI, which is an open standard that allows you to develop in Django, or Pyramid, or web.py, or flask, or any of a number of other options. Flask or web.py are probably your best bet for porting from CGI.

If you run a web application that handles multiple requests in the same process (that is, running a Python WSGI server behind your IIS instance) then you don't incur the overhead of importing arcpy over and over.

Thanks a lot for the tip.

I chose Flask. I made my way through its integration, but I ran into some hurdles, and now a brick wall:

If I run my script stand-alone from the command line, everything goes okay, and I get the output I would expect.

However, if I start up the werzeug server, and run the same script in the Flask context, it gets to a line that calls RefreshCatalog on a CSV file, and then dies silently. If I try/catch around that line, and print the error, I get a useless 99999 error:

[INDENT]Error: ERROR 999999: Error executing function.

[/INDENT]

FWIW, yes, the CSV is present, and readable/writable. (As I mentioned, the same script runs if called directly.)

Finally, I'm on ArcGIS 10.x, and I happen to be using a virtualenv based on ArcGIS's python installation.

Let me know if you have any ideas.

Thanks,
Jamie

JasonScheirer · Answer

There's WSGI, which is an open standard that allows you to develop in Django, or Pyramid, or web.py, or flask, or any of a number of other options. Flask or web.py are probably your best bet for porting from CGI.

If you run a web application that handles multiple requests in the same process (that is, running a Python WSGI server behind your IIS instance) then you don't incur the overhead of importing arcpy over and over.