Best way to write WFS features to a feature layer

[JosephHilliard94]

Hi there, 

For the last year, I have been having extreme difficulties in extracting features from a WFS using Esri software and writing this into a Feature Layer. I want to automate this but the WFS to Feature Class Tool (https://pro.arcgis.com/en/pro-app/latest/tool-reference/conversion/wfs-to-feature-class.htm) timeouts after a while and does the same using FME Readers and Writers (even lowering down the feature count) -  https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_ReadersWriters/arcgisonlinefeatures/arc... 

I have over 80,000 features to write, I believe the quantity and Esri not wanting to support OGC may be the issue (they can't handle the complexity of the URL). Any ideas? 

Thanks!

Joe

[ILoveMap]

Hi Joe,

Have you had any luck with this? I'm running into similar problems. 

[George_Thompson]

There may be a timeout on the service side causing the error after a certain period.

I have also seen machine resources cause issues when exporting data from a service.

Do you own the WFS service?

[ILoveMap]

I do not own the service. I'm able to successfully use the WFS to feature layer tool to extract the 17000 records and manually published them as an ArcGIS Online hosted feature layer. However I'm looking to automate this process to create a dynamically updated Hosted Feature Layer copy of the WFS within our organization.

Any guidance would be appreciated!

[George_Thompson]

Not an expert in this area, but I have seen people who automate do it in batches (of like 500 - 1000), which cuts down on the potential impact of a "limit". This is mainly using Python or something like that.

[KimOllivier]

You might try my script. It extracts the data from WFS, loads it into a geopackage, uploads the geopackage, zipped up. Then turn the online geopackage into a published feature layer collection. Get out the item_ids and put them in your script to update instead of add. It is fast and reliable, only uses open source tools arcgis and a AGOL licence. Not a notebook. Now all I need to solve is how to format dates so that AGOL can recognise them as dates.