Issue with Python Script calling other .py files in a folder via a list

1793
4
Jump to solution
07-08-2021 11:35 PM
LindsayRaabe_FPCWA
Occasional Contributor III

Hi All. I've just tried to implement a single python script that iterates through a list of other python files with the aim of not having dozens of independent Task Schedule items. 

I've got the below code working - with 1 flaw (so far);

 

 

import glob,os
#https://stackoverflow.com/questions/64016426/how-to-run-multiple-python-scripts-using-single-python-py-script

os.chdir(r"\\my\server\folder")  # locate ourselves in the directory containing the python files to run
for script in sorted(glob.glob("*.py")):
   if not script.startswith (("1.", "2.")):
      print ("Found: " + script)
      with open(script) as f:
          contents = f.read()
      print ("Running " + script)
      exec(contents)
      print ("Script complete")

 

 

The script successfully lists all the python files in the folder (and also to exclude 2 which are a template and the script being run above), and it successfully runs through the first script found. When it gets to the 2nd, 3rd, ... scripts though I get the below error;

"Exception: type object 'datetime.datetime' has no attribute 'datetime'"

Obviously, you need to see the script to understand context, so here it is. From what I can see, it is failing somewhere between Lines 50 and 58 (based on the print statements that come out before the error occurs). 

 

from datetime import datetime, timedelta, date

# *****Update the below 6 lines*****

name = "My Hosted Feature Service" #Name of dataset for use in error email notification
url_fl = "https://services3.arcgis.com/SERVICEID/arcgis/rest/services/SERVICENAME/FeatureServer/0" # Service URL for feature layer to download as feature class
destGDB = r"\\My\output\folder\and\GDB.gdb" #The GDB where the backup feature class will be created
destFC = "BackupID1" #The backup feature class name (no spaces - use _ or CamelCase)
monthlimit = datetime.today() - timedelta(days=30) # number of days to keep daily backups
yearlimit = datetime.today() - timedelta(days=365) # number of days to keep monthly backups (1st day of month only) - everything older will be deleted

while True:  #If something fails in the main script under "try", the "except" section emails a notification to the GIS Inbox
   try:
      import arcpy
      from arcpy import env
      from arcgis import gis #Instead of signing in as follows (gis = GIS("https://abcde.maps.arcgis.com", "Username", "Password") this breaks it up so that you are able to call the sign in details independently
      from arcgis.gis import GIS
      from arcgis.features import FeatureLayer
      import getpass
      import json
      import requests
      from time import strftime
            
      def getSecrets():
          # Secrets from file stored outside of revison control
          with open(r"\\none\of\your\bees.json") as f:
              secrets = json.load(f)
          return secrets
      secrets = getSecrets()

      # Get login credentials       # http://docs.python-requests.org/en/latest/user/advanced/
      s = requests.Session()
      url_gis="https://abcde.maps.arcgis.com"
      s.user = (secrets["username"])
      s.pw = (secrets["password"])

      #SIGNING INTO ARCGIS ONLINE
      print ("Signing into ArcGIS Online")
      source = gis.GIS(url_gis, s.user, s.pw) #signing in
      print ("Signed into ArcGIS Online")

      # CREATING BACKUP OF FEATURE SERVICE      # https://community.esri.com/t5/python-questions/using-arcpy-to-copy-a-portal-feature-service-to-a-fgdb-feature/m-p/4285#M394
      fl = FeatureLayer(url_fl)
      fs = fl.query()
      print ("Exporting backup of feature service")
      fs.save(destGDB, destFC + "_" + strftime("%Y%m%d_%H%M%S"))
      time.sleep(10) #add 10 seconds delay to allow export to complete
      print (name + " feature service exported to backup GDB: " + destGDB + "\\" + destFC + strftime("_%Y%m%d_%H%M%S"))
      
      print ("Filtering past backups")

      arcpy.env.workspace = destGDB
      FClist = arcpy.ListFeatureClasses(destFC + "*")

      for fc in FClist:
          datestamp = datetime.datetime.strptime(('{}'.format(fc))[-15:],  "%Y%m%d_%H%M%S")
          day = (datestamp.day)
          print (fc + "..........Backup date:" + str(datestamp))
          if datestamp < yearlimit: #data more than 365 days old
              print ("     Older than 12 months: delete backup")
              arcpy.management.Delete(fc)
              print ("Deleted")
          elif datestamp < monthlimit: #fc more than 90 days old
              if day == 1:
                  print ("     1st of month & 4-12 months old: retain as monthly backup") #fc from 1st of Month and more than 90 days old
              else:
                  print ("     Older than 3 months and not the 1st of the month: delete backup")
                  arcpy.management.Delete(fc) #fc NOT from 1st of Month and more than 90 days old
                  print ("Deleted")
          else:
              print ("     Less than 3 months old: retain backup")
      print ("Script finished")
      break # Stops script here
   except Exception as e:
      from email...#does more stuff

 

 

 

Lindsay Raabe
GIS Officer
Forest Products Commission WA
0 Kudos
2 Solutions

Accepted Solutions
LindsayRaabe_FPCWA
Occasional Contributor III

I just tried your solution and ran the problem script by itself and got the following error;


Exception: module 'datetime' has no attribute 'strptime'

So instead I added "import datetime" line at Line 23 and left the rest of the script as it was originally. It worked standalone, and also in the batch run as desired.

Thanks for the pointer. One day I'll get my head around all of the in's and out's of this!

Lindsay Raabe
GIS Officer
Forest Products Commission WA

View solution in original post

0 Kudos
Luke_Pinner
MVP Regular Contributor

You could also:

from datetime import datetime, timedelta, date, strptime
...
datestamp = strptime(('{}'.format(fc))[-15:],  "%Y%m%d_%H%M%S")

View solution in original post

0 Kudos
4 Replies
Luke_Pinner
MVP Regular Contributor

You are importing the datetime class not the entire datetime module.

from datetime import datetime, timedelta, date

But then try to use the datetime module

# line 56
 datestamp = datetime.datetime.strptime(('{}'.format(fc))[-15:],  "%Y%m%d_%H%M%S")

So just use the datetime class that you imported

datestamp = datetime.strptime(('{}'.format(fc))[-15:],  "%Y%m%d_%H%M%S")

 

 

0 Kudos
LindsayRaabe_FPCWA
Occasional Contributor III

I just tried your solution and ran the problem script by itself and got the following error;


Exception: module 'datetime' has no attribute 'strptime'

So instead I added "import datetime" line at Line 23 and left the rest of the script as it was originally. It worked standalone, and also in the batch run as desired.

Thanks for the pointer. One day I'll get my head around all of the in's and out's of this!

Lindsay Raabe
GIS Officer
Forest Products Commission WA
0 Kudos
Luke_Pinner
MVP Regular Contributor

You could also:

from datetime import datetime, timedelta, date, strptime
...
datestamp = strptime(('{}'.format(fc))[-15:],  "%Y%m%d_%H%M%S")
0 Kudos
LindsayRaabe_FPCWA
Occasional Contributor III

Tried that but got the following error immediately;

ImportError: cannot import name 'strptime' from 'datetime' (C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\datetime.py)


I assume that's because strptime is coming from within the datetime class and not the datetime module (something someone else pointed out to me in an unrelated post. 

Lindsay Raabe
GIS Officer
Forest Products Commission WA
0 Kudos