Hi,
We have a workflow that uses an export from a FileMaker-database, preprocesses it, and then "updates" (overwrites) a layer in a feature layer collection. The script below runs fine, until it doesn't! It usually runs for some 2-3 days before something fails during the publishing phase. This is scheduled through a *.bat-file to run once every hour, but once it has failed, the layer becomes empty and it's not possible to "repair" in any way.
The only way is to republish the data from zero, which then of course creates a new item-ID etc. Alas, this is not a viable solution.
# Import used libraries
import pandas as pd
from arcgis.gis import GIS
from arcgis import features
from arcgis.features import FeatureLayerCollection
# Preprocessing of the CSV-file
df = pd.read_csv(r'//a network drive/Path/To/A/Folder//YODA_2_0_FM.csv',header=None, sep=',', encoding='ANSI')
#Print the df to new csv incl header
df.to_csv(r'//a network drive/Path/To/A/Folder/YODA_2_0.csv', header=["LOTS","OF","COLUMNS"...], index=False)
# Step 2. Publish the csv as feature layer
# Connect to the GIS
gis = GIS(url='https://portal.my_company.com/portal', username='A Username', password='A password')
# Find the Yoda Feature Layer Collection in Portal
Yodafeaturelayer = gis.content.get('49604ab1b4854dca9734eca0a079358c')
Yoda_collection_layer = FeatureLayerCollection.fromitem(Yodafeaturelayer)
# Call the overwrite() method to overwrite with the new CSV
Yoda_collection_layer.manager.overwrite(r'//a network drive/Path/To/A/Folder/YODA_2_0.csv')
The csv that gets created on line 11 is always fine so there's no problem there.
Is there maybe some way of introducing a "check" in the script that performs before the overwrite function?
I guess there are more ways of making this workflow run, but this has worked fine for us, until it doesn't that is 🙂
If anyone has any ideas on how we could secure this it would be highly appreciated as it is a highly demanded layer internally, that needs to be up and running asap...
Thanks!
BR,
Marcus
