Create Vs Replace

Royce_Simpson — Fri, 18 Oct 2024 22:50:54 GMT

I've got a bunch of layers that I want to bring in as hosted feature layers from my ArcGIS Server Rest service implementation. I've got the process working but have doubts about how "create" is used vs "replace".

I see that at first, I have to use "create" to get the hosted feature layers generated, but then, as we update our enterprise data being exposed by our ArcGIS Server site, I want to replace the data in the hosted feature layers via the scheduler.

Right now, in Data Pipelines, I bring in each ArcGIS Server based feature layer and then associate that to a hosted feature layer output with the "create" option. But then, once that's done, I've been changing the "create" option to "replace".

This is a fairly laborious process. Is there a trick to this? Should I have one data pipeline for "create" on all my layers, and then have another one that uses "replace"? That way if I ever need to recreate a layer, I can use that "create" pipeline but for the standard, scheduled update process, I'd just be using the "replace" pipeline.

What's the most sensible workflow here? Ideally, I'd like to see the "create" and "replace" as a single option. So, if the layer doesn't exist, create it and if it does, just replace the data, or perhaps a toggle to allow for data replacement vs recreation of the entire hosted feature layer.

x (x)

Re: Create Vs Replace

MaxPayson — Fri, 15 Nov 2024 01:30:13 GMT

Hi Royce, thanks for the question! We made some related updates in the release this week:

The "Create" method now has an "Overwrite" option that, when selected, will recreate the entire hosted feature layer if it already exists (doc & considerations)
When switching from "Create" to "Replace" after the data pipeline is run, the "Replace" layer will automatically populate with the output from "Create" (this was inspired by your post)

Our recommendation is to use "Create" while authoring and "Replace" for automated workflows. "Replace" is quite optimized for automation; it preserves the schema to avoid breaking downstream apps, it rolls back on failures, and it has negligible disruption (via a table swap). That said, "Create" with "Overwrite" enabled can also work if you want the layer schema to change. I would author this all within one data pipeline because it will be easier to switch from "Create" to "Replace" for the newly created layers.

Hope this is helpful and please continue to share feedback as it comes up!

Thank you, Max

topic Create Vs Replace in Data Pipelines Questions

Create Vs Replace

Re: Create Vs Replace