Select to view content in your preferred language

Best Practices Document for setting up automated stand alone python scripts?

1044
4
09-29-2023 11:50 AM
EllenGuiberson1
New Contributor II

I am looking for some sort of 'Best Practices' document that outlines the ideal setup for running automated python scripts in the ArcGIS environment. I need something that outlines everything from machine setup to licensing to task automation.

Background: Up to now the majority of our python scripts were developed for ArcMap and python 2.7. We have a number of scripts that are run nightly that move data between sde geodatabases as well as run geoprocessing tools to manipulate the data. We also pull data from external sources (FEMA, AGOL) to update layers in our geodatabases.  The scripts are stored on the server and run on the server using task scheduler. ArcMap and ArcGIS Pro were installed on the server to allow the scripts access to the python installs and licensing. This is also the server where with our SQL Server install, as well as file shares for individuals.

My role: I did not set up this system, I hired into it and need to advocate for a better set up. I am not the server manager and cannot update software on the server or manage ArcGIS licenses. I write the python scripts that are implemented by others.

 Problem: I am in the process of updating our existing scripts to run with the python 3x version of ArcPy as well as develop new processes like Vector Tile Caching that rely on ArcGIS Pro to function.  I am running into issues with licensing timeouts, where scripts won't run because it's been too long since ArcGIS Pro was opened, as well as issues with ArcGIS Pro Versions. For instance a Vector Tile Caching script won't run on the server because the .aprx document it needs to read was saved with Pro 3.1 but the server Pro install is still on 2.7. 

Other employees have also started using scheduled Model Builder tools that are saved and run on the server. those tasks are running into the same licensing issues. 

What I need: I need a document that outlines a 'Best Practices' for setting up ways of running automated python scripts with ArcGIS Pro python 3x. I've done lots of Google searches but haven't found something I can present to my supervisor.  A how-to document meant for organizations just setting up a GIS department would be awesome. Where should scripts be run, what type of licensing, does ArcGIS Pro need to be installed on the machine or just the python install. I don't want a document that looks at our existing setup and tries to fix it, I want something that will give me an Optimal Set-up that we can then scale for our needs.

Does something like this exist? Is there a list of web sites or help pages I can look at to create a summary? 

My next step is to put in an Esri Tech Support request if no one has any leads. I just don't like doing that because this is all theoretical and I am not a sys admin that can implement changes. 

 

4 Replies
BlakeTerhune
MVP Regular Contributor

We have pretty much the same setup for our Python scripts. They run on a server with Pro installed. I've never run into a licensing issue, but we use concurrent use licensing with the ArcGIS License Manager. The future seems bleak for the license manager though, so I am not recommending that as a solution. I too would like to know if anyone has a better solution.

0 Kudos
DonMorrison1
Occasional Contributor III

We switched to a "Single Use License" for the user that runs the script in order to eliminate the licensing timeouts.

I'm not sure if this is best practices but we also store all of the required credentials in the Windows keystore instead of hardcoding them in the script

PaulCyr1
New Contributor III

We used to have a similar setup with many python scripts running on a SQL Server but recently migrated all to separate virtual boxes: Script_Server, new SQL_Server and two Enterprise portal servers.  We would advocate for a best practice guide from ESRI too as the migration was a painful time-consuming experience.

0 Kudos
BlakeTerhune
MVP Regular Contributor

Although fairly high-level, I found this ArcGIS Architecture Center, which has a section for Automation.

0 Kudos