topic Re: Is the Join permament? in Data Pipelines Questions

Is the Join permament?

RyanKelley_NA — Tue, 28 Jan 2025 20:49:24 GMT

Is the join a permanent join? I just want to do a temporary join to calculate some fields and then remove the join.

Re: Is the Join permament?

DanPatterson — Wed, 29 Jan 2025 00:00:25 GMT

Add Join (Data Management)—ArcGIS Pro | Documentation temporary

Join Field (Data Management)—ArcGIS Pro | Documentation permanent

at least in ArcGIS Pro, it is

Re: Is the Join permament?

RyanKelley_NA — Wed, 29 Jan 2025 12:59:14 GMT

Right, since it is called Add Join I figured it was temporary but just wanted clarification.

Re: Is the Join permament?

BethanyScott — Thu, 30 Jan 2025 18:04:30 GMT

Hi @RyanKelley_NA ,

Thank you for reaching out!

In ArcGIS Data Pipelines, we never modify the source datasets; this includes adding temporary or permanent joins to them. When you use a Join tool in Data Pipelines, it generates a new dataset combining both of the inputs. Instead of removing the join as you would in temporary join workflows, you can just apply tools to clean up the fields and records. For example, you could do the following in Data Pipelines:

Use the Join tool to combine two datasets
Use the tool Select fields to maintain only the fields of interest
Use the tool Filter by attribute to maintain only the records of interest
Apply any other tools you need to prepare your data
Configure an output to create a feature layer that will be available in your ArcGIS Online content
Run the data pipeline

When you run the data pipeline, the layer that is created will not have a join. It is just a completely new feature layer representing the dataset that you engineered in the Data Pipelines editor.

Please let me know if you have any additional questions!

Thanks again,

Bethany

Re: Is the Join permament?

RyanKelley_NA — Thu, 30 Jan 2025 19:43:48 GMT

Thanks @BethanyScott ! That is helpful!

Is there a way to have a hosted feature layer, join a table or a different hosted feature layer to it, and perform some field calculations without having to have a new output each time? I just want that original hosted feature layer to have attributes updated from the join...not necessarily create a new ouput. Hopefully that makes sense.

Re: Is the Join permament?

EmilyAust1 — Thu, 10 Apr 2025 17:21:11 GMT

I'm also interested in this. My team would like to schedule tasks to join two hosted feature layers together and perform a field calculation that modifies the source dataset without a new output. Right now we've tested "replacing" the source dataset with the pipelines output, but it seems like an unnecessary step that will increase the job's runtime (i.e. credit cost). Is there another way to automate something like this?