Danger Will Robinson! Your Pro Python Code is Wrecking the Spaceship!

ThomasColson — Wed, 01 May 2019 15:21:49 GMT

For years, I have with great success been updating my AGOL feature services with on-prem data using https://www.esri.com/arcgis-blog/products/analytics/analytics/updating-your-hosted-feature-services-with-arcgis-pro-and-the-arcgis-api-for-python/ as a nightly scheduled task. Emphasis on years. Great tool Kevin Hibma ! In my work flow, for folks needing to make a "Map" (not edit), we push them to use AGOL hosted feature services which allow me to dial down the number of service instances I need to maintain on my GIS server farm, which I limit to just "Editors". In this scenario, any edits the editors make will show up in the mappers maps the next day. We have a few scenarios where that doesn't work obviously, so I spawn more instances by default to support the mappers. I control all of this with AD groups driving the permissions on the GIS servers, as well as the Portal (no federation!). This allows me to not need to deploy about 20 GIS servers, as we have discovered there is a finite number of service instances a single server can push, despite memory and CPU.

Until this past weekend. The server this was running on, as a scheduled task using a domain service account, went on a protest strike. 2 days of troubleshooting narrowed it down to an issue with the local profile of the service account, which we could no longer troubleshoot, as we couldn't even get access to the profile, not even with Powershell. Long story short, here's what was happening.

Python runs at 1910:

Success is written to a log

But....

A massive plethora of junk ends up in C:\Users\[domainsvcaccount]\AppData\Local\Temp!

8 items to be exact. Times 100-ish hosted feature services being updated daily. Times 2 years.

What had happened is the local\temp folder filled up with several hundred million of these things, several hundred gb worth. It simply blew up the OS. I was able to clear it out using Snake Oil.

It's well known that clearing local\temp solves many ESRI problems, and we do so frequently with desktop. But this is the first time I've seen the problem using Python.

So the warning here is, if you're a heavy user of https://www.esri.com/arcgis-blog/products/analytics/analytics/updating-your-hosted-feature-services-with-arcgis-pro-and-the-arcgis-api-for-python/ , make sure to automate cleaning C:\Users\[domainsvcaccount]\AppData\Local\Temp.

As a side note, I'm curious (concerned) as to why Py is throwing that much stuff in the temp folder, but like I said, automated clean up in the solution here.

Re: Danger Will Robinson! Your Pro Python Code is Wrecking the Spaceship!

FraserHand1 — Wed, 01 May 2019 21:39:08 GMT

Hi there,

yep it can fill up fast - a lot of those file get created when you are manipulating ArcGIS Pro projects or ArcMap documents and during the create sd draft, service publishing process. What I usually do is I have a class that cleans up a directory and I run this as the last step in my Python workflow making sure to clean up any files that may have been created during the run process. Note that you need to del any references to file items to clear handles on them.

Re: Danger Will Robinson! Your Pro Python Code is Wrecking the Spaceship!

DanPatterson_Retired — Thu, 02 May 2019 00:10:31 GMT

Now... don't go blaming python... probably something in the ArcGIS API

Re: Danger Will Robinson! Your Pro Python Code is Wrecking the Spaceship!

DuncanHornby — Fri, 24 May 2019 14:52:33 GMT

+1 for title of page! and thanks for the advice!

topic Re: Danger Will Robinson! Your Pro Python Code is Wrecking the Spaceship! in ArcGIS Pro Questions

Danger Will Robinson! Your Pro Python Code is Wrecking the Spaceship!

Re: Danger Will Robinson! Your Pro Python Code is Wrecking the Spaceship!

Re: Danger Will Robinson! Your Pro Python Code is Wrecking the Spaceship!

Re: Danger Will Robinson! Your Pro Python Code is Wrecking the Spaceship!