How to back up a feature layer along with the attachments

426
5
Jump to solution
03-24-2021 12:02 PM
RaufShaikh
New Contributor II

Hello Friends, 

I have a feature layer in ArcGIS online and it has 180,000 records and most have the features (records) have attachments as well. As the feature layer size is growing, we are trying to archive the data submitted until 31-Dec-2020. I tried two different methods as explained in the below articles

https://support.esri.com/en/technical-article/000012232  (It creates replica using rest endpoint of the Feature service)

In the above method, the process kept running for about an hour, it extracted data but looks like the process failed when extracting the attachment. I am telling this because the status was showing as extracting attachments. It only displays message : Exporting data for layer failed. and doesn't give any details about the issue and the replica doesn't get created.

https://support.esri.com/en/Technical-Article/000015786 (it uses geoprocessing tool in ArcGis Pro to extract the data with attachment)

In the above method, ArcGIS pro keeps running for sometime and then freezes and shut's down. This process also fails in creating a local geodatabase from the ArcGIS online feature layer.

 

So my question is how I can extract the data from my feature layer along with the attachments and publish it as an archive feature layer.

Thanks ,

Rauf

0 Kudos
1 Solution

Accepted Solutions
RaufShaikh
New Contributor II

Hi Tim,

Sorry for the late reply as got busy with other stuff but I ended up writing my own python script which loop through all the features from the source dataset and then add them to the target dataset. For each feature in the source, I am downloading the attachments and uploading it to the target dataset as well. It took some time to complete as the dataset was too large around 173000 features + attachments but finally it was done.

Thanks,

Rauf

View solution in original post

0 Kudos
5 Replies
Tim_McGinnes
Occasional Contributor III

This article explains how to export from the rest endpoint using smaller batches of records. That should have a much better chance of working.

https://support.esri.com/en/technical-article/000014156 

0 Kudos
RaufShaikh
New Contributor II

Thanks for your response, Tim. I have stumbled upon that article earlier as well but aren't there any other way to automate or do it all at once?

0 Kudos
Tim_McGinnes
Occasional Contributor III

Not that I know of - you may have to script it in your language of choice or maybe FME if you have it? I think that long running REST processes with large attachment layers are just problematic. We have had lots of issues with replication of these types of layers as well. Scripting small batch extracts with lots of error checking and retrying on failure seems like the best option to me.

RaufShaikh
New Contributor II

Hi Tim,

Sorry for the late reply as got busy with other stuff but I ended up writing my own python script which loop through all the features from the source dataset and then add them to the target dataset. For each feature in the source, I am downloading the attachments and uploading it to the target dataset as well. It took some time to complete as the dataset was too large around 173000 features + attachments but finally it was done.

Thanks,

Rauf

View solution in original post

0 Kudos
DerrickWestoby
New Contributor III

I have a feature layer with ~9,000 point features that contain a photo attachment.  I could never get backups to succeed for this, so I tried out "CivicLens" in the ArcGIS Marketplace and ended up buying it.  It's been working great for about 6 months now.  (Backup size is currently 8.5gb)

0 Kudos