Select to view content in your preferred language

Running item.export from notebook sometimes hangs for no reason

392
1
01-02-2024 01:17 AM
Labels (1)
JoeBullard
Emerging Contributor

Hi,

I have a notebook that runs 5 nights a week after working days. The script takes a copy of a feature layer and exports it as a FileGDB. This generally works, however on occasion, some files will take an excessively long time to export, resulting in the script timing out. I can't for the life of me work out what causes this to happen - the same files that are unchanged from one day to the next can go from taking <1 minute to several times that. 

The issue seems similar to this one here: Re: Scheduled Notebook sometimes exceeds execution... - Esri Community 

 

now = datetime.datetime.now()

existing_backups = [item.title for item in user.items(folder='Backups')]

query_string = "tags: 'to_backup'"
feature_services = gis.content.search(
    query=query_string, 
    max_items=-1, 
    item_type='Feature Service'
)

for fs in feature_services:
    
    print(f'Processing: {fs.title}')
    file_name = generate_name(fs.title, now)
    
    if file_name not in existing_backups:
        try:
            fs.export(file_name, 'File Geodatabase', parameters=None, wait=False, tags='backup')
        except:
            print(f"Couldn't export {fs.title} - {fs.id}")

        export = gis.content.search(file_name, item_type='File Geodatabase')
        
        if export:
            try:
                fgdb = gis.content.get(export[0].itemid)
                fgdb.move("Backups")
                print(f'Backup of {fs.title} complete\n')
            except:
                print(f"Not able to move item: {file_name}, check if item already exists in 'Backup' folder")
        else:
            print(f"No backups with name {file_name} found")
    else:
        export = gis.content.search(file_name, item_type='File Geodatabase')
        print(f"Backup: {file_name} already exists, skipping to next item\n")
        
print("Backup processing complete")

 

Tags (3)
0 Kudos
1 Reply
MappyIan
Frequent Contributor

Hi @JoeBullard, I'm wondering if you're seeing more occurrences of this problem since the start of the year?  We have the same problem, and in January our Notebook failed four times (because the export process in the script times out), which is much more frequent than was the case last year.  Looking back, we didn't experiece this timeout problem at all in the last four months of 2023 (it happened twice in August 2023 on the 1st and 14th).

I'd be very interested to know if you've experienced a similar increase in the number of timeouts this year, and also if they occur at the same time as the ones we experience.  In January, we had issues on the following dates/times:

  • 10th Jan, around 6:45
  • 11th Jan, around 10:45
  • 22nd Jan, around 14:00
  • 31st Jan, around 11:00

I'm hopeful that if we're experiencing the same problem at the same time then we might be able to get ESRI to look in to it further.

Thanks - Ian

0 Kudos