Help running python script through task scheduler with Exception:'LOCALAPPDATA'

[LindsayRaabe_FPCWA]

I've got a number of python scripts that download copies of ArcOnline feature services as part of a nightly backup routine (in lieu of anything simpler). At present, the python script logs into ArcOnline, accesses the hosted feature service and exports a copy of the data to a feature class in a GDB on our server. This python script is then called by Task Scheduler on a nightly basis (user logged off with highest privledges). Last month (while I was on leave), the server was moved from physical on-premises to cloud based, and we started getting the below Error:

Exception: 'LOCALAPPDATA'

I can't find anything online that really points out what the error is or how to fix it. I assume it's an issue accessing the temporary data environments. When I log in and run the script manually (even through Task Scheduler), it works without any problems. It only happens once logged off. 

I tried running the following script in IDLE, but that also didn't yield any positive results. 

import arcpy

# Reset geoprocessing environment settings
arcpy.ResetEnvironments()

I've tried finding inbuilt environment settings in IDLE (for ArcGIS Pro) but couldn't so don't know where else to look. I've included my script below so that you can analyse it more in depth.  

from datetime import datetime, timedelta, date

# *****Update the below 6 lines*****

name = "FS Name" #Name of dataset for use in error email notification
url_fl = "https://services3.arcgis.com/#X#X#X#X#X#X#X#X#X/arcgis/rest/services/FSName/FeatureServer/0" # Service URL for feature layer to download as feature class
destGDB = r"\\server\ArcGIS Online\AGOL Backups\Backups.gdb" #The GDB where the backup feature class will be created
destFC = "FC Name" #The backup feature class name (no spaces - user _ or CamelCase)
monthlimit = datetime.today() - timedelta(days=30) # number of days to keep daily backups
yearlimit = datetime.today() - timedelta(days=365) # number of days to keep monthly backups (1st day of month only) - everything older will be deleted

while True:  #If something fails in the main script under "try", the "except" section emails a notification to the GIS Inbox
   try:
      import arcpy
      from arcpy import env
      from arcgis import gis 
      from arcgis.gis import GIS
      from arcgis.features import FeatureLayer
      import getpass
      import json
      import requests
      from time import strftime
      import datetime
            
      def getSecrets():
          # Secrets from file stored outside of revison control
          with open(r"\\server\file.json") as f:
              secrets = json.load(f)
          return secrets
      secrets = getSecrets()

      # Get login credentials       # http://docs.python-requests.org/en/latest/user/advanced/
      s = requests.Session()
      url_gis="https://org.maps.arcgis.com"
      s.user = (secrets["username"])
      s.pw = (secrets["password"])

      #SIGNING INTO ARCGIS ONLINE
      print ("Signing into ArcGIS Online")
      source = gis.GIS(url_gis, s.user, s.pw) #signing in
      print ("Signed into ArcGIS Online")

      # CREATING BACKUP OF FEATURE SERVICE      # https://community.esri.com/t5/python-questions/using-arcpy-to-copy-a-portal-feature-service-to-a-fgdb-feature/m-p/4285#M394
      fl = FeatureLayer(url_fl)
      fs = fl.query()
      print ("Exporting backup of feature service")
      Outputfs = destFC + "_" + strftime("%Y%m%d_%H%M%S")
      fs.save(destGDB, Outputfs)
      time.sleep(10) #add 10 seconds delay to allow export to complete
      print (name + " feature service exported to backup GDB: " + destGDB + "\\" + Outputfs)
      
      print ("Filtering past backups")

      arcpy.env.workspace = r"\\server\ArcGIS Online\AGOL Backups\Backups.gdb"
      FClist = arcpy.ListFeatureClasses(destFC + "*")

      for fc in FClist:
          datestamp = datetime.strptime(('{}'.format(fc))[-15:],  "%Y%m%d_%H%M%S")
          day = (datestamp.day)
          print (fc + "..........Backup date:" + str(datestamp))
          if datestamp < yearlimit: #data more than 365 days old
              print ("     Older than 12 months: delete backup")
              arcpy.management.Delete(fc)
              print ("Deleted")
          elif datestamp < monthlimit: #fc more than 90 days old
              if day == 1:
                  print ("     1st of month & 4-12 months old: retain as monthly backup") #fc from 1st of Month and more than 90 days old
              else:
                  print ("     Older than 3 months and not the 1st of the month: delete backup")
                  arcpy.management.Delete(fc) #fc NOT from 1st of Month and more than 90 days old
                  print ("Deleted")
          else:
              print ("     Less than 3 months old: retain backup")
      print ("Script finished")
      break # Stops script here
   except Exception as e:
      from email.mime.multipart import MIMEMultipart
      from email.mime.text import MIMEText
      import smtplib

      fromaddr = "us@org.com.au"
      toaddr = "us@org.com.au"
      msg = MIMEMultipart()
      msg['From'] = fromaddr
      msg['To'] = toaddr
      msg['Subject'] = name + " backup process failed"
      # Enter email body text below
      body = "There has been an error backing up the feature service. Please check the script to troubleshoot any problems. Exception: " + str(e)
      msg.attach(MIMEText(body, 'plain'))    
      server = smtplib.SMTP('smtp.org.com.au')
      #No login required so this section is commented out
      #server.login("youremailusername", "password")
      server.sendmail(fromaddr, toaddr, str(msg))
      print ("Script failed - email notification sent")
      print ("Exception: " + str(e))
      break # Stops script here

[JeffK]

Instead of print statements, implement a logger so you can review what is happening when it's ran in Scheduler and your not logged on.

What type of ArcGIS licensing do you have on the cloud server?

This line doesn't do anything because it will always be True and since it is not a variable and nothing can change it, is the same as just not having it.

while True:  #If something fails in the main script under "try", the "except" section emails a notification to the GIS Inbox
  

---

[LindsayRaabe_FPCWA]

Hi @JeffK . Thanks for the tips. I'm not experienced with the logging side of things. I ended up including the following code at the begininning of my script, tested it (which worked - got a file output with lots of stuff), reset the schedule, logged off and waited - with no joy. Just got the same LOCALAPPDATA error via the email. 

import logging
logging.basicConfig(filename='python.log', level=logging.DEBUG)
logging.debug('Debug Message')
logging.info('Info Message')
logging.warning('Warning Message')
logging.error('Error Message')

As for the line that you say isn't required, I removed it and reduced the indentation of all the following lines but got the following error when checking, so put it back in. 

[JeffK]

You can remove that break along with the while True: lines.  The script will stop executing after it executes the last line so these are redundant.

What was in the logging file? Just the few lines that you added or was there more dumped in there? 

You can change the print statements to something like this:

logging.info(f'{fc} ..........Backup date: {str(datestamp)}') so it will log those lines.

You could also try scheduling another script that just creates a file (no arcpy) or something simple and see if that runs. If the script/task doesn't even fire off, then there is something you'll need to track down with the user/server. If it runs, then I would comment out everything in this script and start un-commenting the imports one by one. Log them too- if it makes it through the imports, keep going un-commenting the code one process at a time. If it fails on one of the imports, then that is where you start looking. 

[LindsayRaabe_FPCWA]

I was wondering if the breaks were also superfulous. I'll try that out. 

As for the logging file, when I tested it whilst logged in, it did spit out a heap of text (which for the most part I just glossed over as it was working as expected). I deleted the log file, scheduled and logged off, and when it failed, logged in to find that it didn't even create the log file - so I feel it's not actually even getting to the point of executing the script? 

[LindsayRaabe_FPCWA]

Hi @JeffK. I've managed to isolate the cause of my issue - but need some guidance on a solution! I took the approach of commenting out all the lines, scheduling the code to run and one-by-one, uncommenting a line and rescheduling until I found which one was causing the problem. The below code shows that line 3 is causing the issues!

import arcpy
   from arcpy import env
   #from arcgis.gis import GIS ###### This line is causing the Exception
   import os
   import time
   from time import strftime
   import datetime
   import zipfile
   from zipfile import ZipFile

   # Access the stored password with keyring and sign into the GIS      # https://community.esri.com/t5/arcgis-online-blog/connect-to-the-gis-in-python-scripts-without/ba-p/889867
   import keyring
   pw = keyring.get_password("ArcGISOnline", "Username") # Note: Username is Case Sensitive
   gis = GIS("https://org.maps.arcgis.com", "Username", pw) # Note: Username is Case Sensitive
   print("Connected to the GIS")

Obviously this line is needed (first referenced in line 12 and strangely it works fine when running the script through IDLE or Task Scheduler when logged in - it only fails and throws the Exception: LOCALAPPDATA issue when scheduled through Task Scheduler and logged out.

Tested that it worked by only commenting out Line 3 and got the following - "Exception: name 'GIS' is not defined". 

[JeffK]

I wonder if you try to use a cloned environment saved to a common location (instead of the users local appdata), it wont have to look in localappdata. maybe have to set the env up using conda and exporting to a general location. This post has some good steps for the process: what-s-new-with-python-in-arcgis-pro-2-8