nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Allow feature services with joined hosted feature layer views to have the data schema updated without removing the view

Philip_Ohlinger

Currently, if you have a joined hosted feature layer view setup for a feature layer, you cannot edit the data schema of the layer in ArcGIS Online. This means you cannot add or remove fields, or alter the domains for the features unless you remove the view. This is extremely cumbersome, as we are using the joined hosted feature layer views for many of our features, but also need to periodically make changes to the data schema.

Please allow joined hosted feature layer views to be used while also allowing changes to the data schema of the features. This would save us so much time not having to continually delete and recreate the views.

Find more posts tagged with

Status: Open

Comments

ScottKiley

I first want to to say that I 100% agree that this needs to be implemented. We have implemented some workarounds that lessen the work required when needing to make a schema change on data that partakes in a joined view that @Katie_Macintyre_Atmos and others on this post have identified.

I fully understand that workarounds shouldn't be required for something so fundamental to utilizing this platform, but here are those workarounds.

Add a joined view to a web-map.
Do a save-as on the joined view layer to create a secondary item referencing the joined view.
When you need to make a schema change to feature services taking part int the joined view, delete the joined view, but NOT the saved item reference. Also note the exact name of the joined view as it appears in REST.
After making the necessary schema changes to the source data, you recreate the joined view with the exact name as it originally appeared in REST.
Your saved-as item referencing the joined view will re-path properly in the web-map and any subsequent app configurations a well. You'll need to do any reconfiguration of your layer to account for the schema change, especially if it's a new attribute field, but it won't completely break the web-app or any subsequent app. The best part, is that any pop-up, labelling, or other additional configurations of the saved-as item referencing the joined view get saved back to the reference item, so those are maintained after deleting and recreating the joined view.

The key is to be consistent and diligent in the naming conventions. Our firm uses naming conventions like this:

Joined View Name:

[Layers/Asset Names]_Joined_View

Use the names from both datasets partaking in the join along with “_Joined_View”

Ex: Storm_MH_Inspections_Joined_View (where the joined layers are Storm MHs and Inspections)

Our thinking here is using “Joined_View” makes it explicit that this IS the join itself.

Saved-As Reference Layer Name:

[Layers/Asset Names]_Join_Reference

Use the names from both datasets partaking in the join along with “_Join_Reference”

Ex: Storm_MH_Inspections_Join_Reference (where the joined layers are Storm MHs and Inspections)

Our thinking here is using “Join_Reference” makes it easy to understand that isn't the join itself, but the reference of it.

We didn’t like using the word “Layer” here because Layer is used so much for so many things in ArcGIS, and isn’t explicit enough.

NOTES:

Whenever a NEW joined view is made, we should immediately do a save-as on it to create the referenced layer, then ensure subsequent configuration changes to it are saved as well.
When you need to do a schema change to either dataset partaking in the join and require deleting the original join, you DON’T need to do another save-as to create another referenced layer.
In the Item description of the Joined_View, enter the datasets partaking in the join and the name of the Join_Reference created from it.

Also note that “When either of those layers require a schema change, this item must be deleted here and emptied from the recycle bin. Then recreated after the necessary schema changes are made.”

In the Item description of the Join_Reference, enter the name of the Joined_View it was created from.

Also note “DON’T DELETE THIS ITEM!”

I'm curious to know if others on this thread have used similar workarounds to deal with this massive limitation.

Katie_Macintyre_Atmos

Agree this is a massive limitation, especially when coupled with the functionality limitations in experience builder which mean you effectively have to recreate everything if you replace a layer in a web map.

We have an EB app which allows our surveyors to see bird data, which has been joined to a table so they can filter by designation status. We need to add a field to the bird data layers which now means deleting 7 view layers, recreating them, replacing them all on the web map, and reconfiguring 18 widgets in the EB app. We would need to do all of this each time we needed to add a field to the dataset or update a domain value.

At the very least there should be a warning when creating joined view layers that the schema for both datasets will be locked and updates to fields and domains will result in a lot of extra work.

csambol_cityofguelph

I have also run into this issue as I have many views/maps created for a Citizen Problem Reporter implementation. I have deleted the join views to which should have allowed me to add a field in the main hosted layer but it does not. I have looked at the JSON and there is nothing there for the "editing" items in the previous post or "schemaLock" found in another post. I need to add a new field to the main data to support the users but cannot determine why it is still locked as the main views should not have an impact on this, as well as if there is a tool to find the layer which may be causing the lock. Looking for any assistance and I agree with everything that there should be a UI tool to accomplish this as Esri is promoting this option but we cannot break/delete views all the time to add fields for solutions that are in use. Any assistance would be appreciated and I have also submitted a support ticket.

WillyG

Thank you @EricaNova that's great. The steps you have taken are really clearly laid out. I'd like to try this on a proof of concept I have time. My main concern is that this could cause issues with sync enabled hosted feature services. If it can be done by editing the JSON then I'm sure ESRI can build an 'update Joined view layer' UI

The fact it can be done in the JSON should not be a substitute for a UI. Can someone from ESRI please give an update when we are going to have the ability to update the schema on source layers as stated that we would here (June 2023 ArcGIS online release) When will we see this on the road map?

if there is a technical explanation for not enabling this then please tell us why.

EricaNova

@AbdiHassan 's post solved this issue for me. I only had two hosted feature layer views and they weren't important enough to not test this solution, so I went for it and it seems to have worked.

In my case, all I wanted to do was to update a domain; I had mispelled someone's name in my list of allowable values and needed to correct the spelling both in the label and in the code for that list value. I couldn't edit the list since I had some joined feature layer views.

I went to the joined feature layer view overview page, and clicked the "View" button to the right of the feature layer URL (on the page's right-hand panel) to access the feature service's REST information. Then I clicked 'Admin' in the upper right. At the bottom, I clicked "Update Definition". I used the keyboard (Ctrl-F) to access the page search and searched for 'schema'.

In both (2) places where I found 'schema', I changed 'false' to 'true', e.g.,

"sourceSchemaChangesAllowed" : false, --> "sourceSchemaChangesAllowed" : true,

Then, I altered the editingInfo as well, to insert empty quotes around the last edit dates, e.g., from:

"editingInfo" : {
"lastEditDate" : 1751901654885,
"schemaLastEditDate" : 1749488965825,
"dataLastEditDate" : 1749488965825

to:

"editingInfo" : {
"lastEditDate" : " ",
"schemaLastEditDate" : " ",
"dataLastEditDate" : 1749488965825

Once I'd done this for both of my hosted feature layer joined views, I was able to update the domain values in my hosted layer. I noticed that they were automatically updated in my joined feature layer views.

I'm an amateur, so I'd welcome feedback if this is incorrect for some reason.

DavidForbuss1

I'm running into this problem now as well. Luckily, we're in the testing phase of using a spatial layer joined to a table that was imported from MSAccess. It's still very cumbersome to have to delete the view, then recreate it, and trying to remember all of the symbology/labels/pop-ups. It should be as simple as updating a domain on the joined table, then clicking 'Update View'.

WillyG

Hi @AbdiHassan That's a good suggestion, I'd like to try that, but would need more guidance on exactly how to do it as I don't have a lot of time to experiment. I still believe that it needs a UI not dissimilar to the update view UI, especially as it was announced in June 2023 that we would be able to update schema. We add fields all the time and don't want to be faffing around with the JSON. Most of our Hosted feature services are sync enabled, so sync errors would also be a concern, You can also see that a lot of people are confused when a joined view layer is in place that they cannot update schema, especially as there is little warning that this is the case. This NEEDS something better than what we have currently. I don't understand why this is not prioritised.

AnneShanahan2

Agree with all of the above, this is a real showstopper to our org using live joins. A solution or effective workaround would be hugely appreciated. Then we can retire cumbersome overnight processes.

AbdiHassan

You can use the update definition on the joined view to allow schema changes to true which should allow you to make schema changes to the source. If you want to add a field and have that field to propagate to the joined view, then you would have to do like @jfisher mentioned, delete the definition of the view and add definition which you need to include the new field.

WillyG

Can someone from ESRI give an update on this please?

Philip_Ohlinger

This poses a huge problem for me every day. ESRI, can you please allow changes to the data schema without having to undo all of the joined views I've put into place?

AndrewWillingale

I'm unable to make schema edits to a hosted feature layer at the moment. I did have a number of joins and after lots of investigation came across this forum post. I've deleted the join views, which obviously stopped the web map which was accessing these from working, and I still seem to have a schema lock!

If data schemas become locked then we need a clear message to the user telling them why and how.

@LaurenBallantyne I've tagged you in as you wrote the blog linked above. Please can you provide an update about this. Thanks.

WillyG

@jfisher Thank you. So you are effectively disconnecting the joined view from the source and then reconnecting? Can you please share more of your workflow and where to find ':remove from definition' in the REST endpoint

jfisher

EDIT: I have not had time to further test this. I wrote it from memory and it may be mistaken. I'm leaving it for now, in case someone is able to do further testing.

The solution that has been working for me so far is to not delete the joined view, but rather use the REST endpoint: delete from definition. Once the joined view is removed from the service definition, the source layer schema can be edited. You can then use add to definition to re add the joined hosted feature layer(s) back to the service. As long as you add definitions in the appropriate order (so that they have the same service id number they did before removal), apps that consume the joined-hosted-feature-layer-view should continue working: the service itself has not been removed and the id has not changed. The benefit of using the REST endpoints is that you can add many joined hosted feature layers to a single service. This of course should all be done programmatically (I'm using arcgis python api), so that service definitions (joined hosted feature layer view definitions) can be added and removed with the push of a button.

WillyG

Couldn't agree more @JeffShaw When hosted view layers were re-released in June 2023 it was stated in this blog: that the ability to allow schema changes in source layers is coming Join us in welcoming joined views to the item details page (esri.com)... It has been over a year and there has been complete radio silence. Even giving us functionality to disconnect / reconnect using something like AGOL assistant or view swapping like what is in the regular hosted view workflow is better than nothing.

JeffShaw

Needing to delete joined views in order modify the joined layers makes them not worth the significant effort and risk to configured apps in almost all cases.

We NEED Esri to get serious about finally providing useful relational data structures in ArcGIS Online after all these years. As a user of Esri products for 37 years I can say that the lack of support for this is baffling and has significantly curtailed the effectiveness and efficiency of our agency in managing natural resources.

WillyG

I was hoping with the new recycle bin functionality that after we delete the joined view layer it would then unlock the schema, and then we could make our changes to the schema and then restore the joined view layer, but the schema of the master is still locked while in the recycle bin unfortunately. There must be a simple solution to this. We just need a workflow to to swap the joined view layer or disconnect it temporarily so we can unlock schema. We have an update views option in the settings for regular views. Why not joined view layers?

jfisher

One possible work around mentioned here is to programmatically find and replace item id's in the definitions of all affected items using the arcgis python api. Using this solution means you need to keep careful track of any item that references your joined-hosted layers.

We have not stared using joined hosted feature layers yet. They are potentially powerful, but this limitation is preventing me from really embracing them.

DevinCarlson

A solution to this workflow is definitely needed!

Or - a better recommended practice to be able to symbolize features based on the most recent inspection that lives in a related table.

Specific use case: Hydrant Flushing
Hydrant feature is related to Hydrant Maintenance Inspection table.
Currently, we are utilizing the joined view layer to show our field crews and department heads when a recent flushing event occurred - in Field Maps and Experience Builder. When changes to the schema are needed or domains need to be updated/changed - which is inevitable - the join view layer needs to be deleted, changes made, then recreated, resymbolized, and added back into any maps or applications. Very tedious workflow.

We have also tried to use an Arcade Expression in the parent hydrant feature that looks at the related table and populates the parent feature with information from the inspection. This could work, but I am unable to get the parent table to update automatically when the inspection record is submitted. We have to start to edit the parent feature, then the data populates (working in Field Maps).

Regardless, a better workflow to utilize inspection data that lives in a related table is very much needed!

MattyMaps

A pause and refresh view option would solve this

WillyG

We just need a button to disconnect the joined view layer from the source, make schema changes and then reconnect in a seamless manner that would not cause disruption to the end user. Perhaps this could be handled with the update view workflow, not dissimilar form what we have with regular views. How about the ability to swap the source of the hosted join layer to a temporary slave 'Layer'. We can then make changes to the master while we are in this limbo stage and the end user wouldn't notice anything. This seems like it should be reasonably straight forward to implement. Currently we are bending over backwards to provide complex workarounds for this

Kaz

Agree, this limitation needs to be VERY obvious when you create a join layer, and there needs to be a way to update the source schema afterwards.

Example: A survey is created. The survey feature layer then participates in a live join view. Then you are unable to make changes to the survey. 😞

DamienDiVittorio

As a new user on AGOL this issue was extremely frustrating to me! It took me quite a while to figure out why I couldn't edit my hosted feature layers. There has to be a way to lock only the fields used to define the joined view and allow schema and domain changes elsewhere.

At a minimum, their could at least be a pop up to warn users that creating joined hosted feature layer views will not allow schema changes on the parent features/tables. This has been a known issue for years. ☹️

Calvin_Wong

I would also like to see this. It's frustrating having to delete the joined view and re-create it. The view is usually linked to dashboards or Experience Builder and this breaks since the item ID changes after a re-creation.