Feature Layer Views: Updating Parent Feature Layer Schema

2819
20
05-02-2021 11:51 PM
GlenShepherd
Esri Contributor
8 20 2,819

The Premise:

When we create a Hosted Feature Layer View (hereafter referred to as a "View Layer") from our Hosted Feature Layer in ArcGIS Online, our View Layer schema is inherited from its parent Feature Layer.

Sometimes project design requires us to make schema changes to our parent Feature Layer and we want to be able to ensure that all dependent View Layers are inheriting the same changes. These might include:

  • Adding or deleting fields
  • Updating coded domain lists
  • Editing field name aliases

ArcGIS Online (and ArcGIS Portal) users may run into unexpected issues where updates are not being reflected in their View layers. And it is important to understand how to manage this behaviour before overhauling the content, deleting the old view layers and creating new ones from scratch.

 

The Troubleshooting:

There are a few reasons why our View Layers might not be inheriting schema changes made to the parent Feature Layer and a few troubleshooting methods at our disposal to narrow down the cause of the disconnect...

First of all, we want to ensure that we're not trying to do something that is simply not possible in ArcGIS Online. View layers that participate in table Joins performed through Map Viewer will create schema-locks that prevent the parent Feature Layer's schema from being edited. We can read more about this here: Bring your data closer together: Save analysis results from Join Features as Hosted Feature Layer Vi...

  • "feature layers that participate in view layers with joins cannot have their schema altered and are always read-only and cannot be used offline."

Once we've clarified that. Let's work through the solution in the following two scenarios to narrow the issue down...

(Note: The following will cover troubleshooting View Layers that have been configured using the "Create View Layer" button on the Feature Layer's Item Details page.)

 

Scenario 1: We've added new fields to the parent Feature Layer but cannot see them in the View Layer's attribute table

Solution: Update "Define Fields" for the View Layer

  • For the View Layer item, go to: Item Details > 'Visualisation' tab
  • Click the View Layer's 'More Options' list and go to: Set View Definition > Define Fields
  • Check the boxes next to the newly added fields at the bottom of the list.

Resource: Problem: Hosted feature view does not inherit new field added to hosted feature service

Solution: If we have previously chosen to "Define an Area of Interest" for the View Layer... Delete it.

  • For the View Layer item, go to: Item Details > 'Visualisation' tab
  • Click the View Layer's 'More Options' list and go to: Set View Definition > Define an Area of Interest
  • Click "REMOVE AREA" 

Limitation documented here: Considerations when creating hosted feature layer views

viewDef.png 

 

Scenario 2: We've added new domains to the 'List of Values' in a particular field for our parent Feature Layer and cannot see them being inherited by a View Layer

Solution: clicking on the "Reset to source" button to revert the view layer's field update property from "Overridden" to "Inherited".

  • This disconnect may have occurred due to changes being made to specific fields within the View Layer, causing the field properties to change from "Inherited" (from the parent layer) to "Overridden".
  • The simplest way this occurs is if changes are made to field names. This causes a break between the aliases in the parent table and those in the View Layer table.
  • For example:
    By default, changes made to the parent layer are "Inherited" by the view layer. Properties: Inherited
    Inherited.png

  • However, if a change is made to the field name, the view layer's update property becomes "Overridden" and changes to the parent hosted feature layer are not reflected. Properties: Overridden
    Overridden.png
  • The connection and inheritance can be re-established by clicking on the "Reset to source" button (pictured above) which will revert any changes made to the field, back to that of the parent Feature Layer. The user should check all other fields to ensure this hasn't happened elsewhere.

Read more about the Settings that influence the "Inherited"/"Overridden" field property on View Layers here: Hosted feature layer view settings (edit, courtesy @Bnewell).

Note: In some cases, users will need to carry out a combination of the solutions outlined above to resolve all inheritance issues they are encountering.

 

Some key things to remember:

  • To maintain schema inheritance, it's best not to make edits in the 'Data' > 'Fields' tab for your View Layers.
  • Once we have published our Feature Layer and created View Layers from it, we cannot use ArcGIS Pro to Overwrite the parent Feature Layer, as it now has dependent layers. Either the View Layers will need to be deleted first or the updates must be carried out in the web portal.
  • If we have subtypes and domains configured for a feature layer and symbology/filters dependent on this schema; create all templates for your layers in ArcGIS Pro prior to publishing.
  • If we have a View Layer participating in a table Join, and we want to edit the parent Feature Layer's schema, then we will need to delete the View Layer created from the Join operation.

 

20 Comments
KevinMacLeodCAI
Occasional Contributor II

Useful guide, thank you Glen for the post.

 

Sidenote enhancement idea- It would be an excellent if in a future update if joins did not lock out parent hosted feature layers. This prevents data from being able to be updated. Let's say you have Parcels. And a read-only view of Parcels joined to something. Well what if you wanted to have a notebook update the Parcels weekly. This prevents that. It would be useful if joins did not create this kind of a lock and increase the amount customer utilization of hosted feature layers.

 

 

MattHuser
New Contributor II

@GlenShepherd What about adding a new feature class or table to the overall schema? For example, our original source of the hosted feature layer is a FGDB, we continually use the "update" process to push updates from our local environment to AGOL. But when we add a new feature class to the FGDB and use the "update" process, the parent (hosted feature layer) is updated to include that new feature class but none of the hosted views inherit the new layer.  Is there something we have trigger to force a schema inheritance of a new feature class?

GlenShepherd
Esri Contributor

Hi @KevinMacLeodCAI ,

Thank you, we're certainly in agreement on that sidenote. There are currently enhancement requests and defect records lodged for this behaviour, so fingers crossed :crossed_fingers:

Hi @MattHuser ,

Can you elaborate on your hosted feature layer; is this a series of feature classes grouped into a single feature layer and you are updating records within them? Or are we adding a completely new feature class to the grouped layer?
How is your update method carried out?

KevinMacLeodCAI
Occasional Contributor II

Thanks Glen for your vote of support! I know it would be huge and increase adoption and use cases for many Orgs.

TomThomey
New Contributor

.

MattHuser
New Contributor II

Hi @GlenShepherd ,

We originally started out with a FGDB in our local environment then uploaded it to our AGOL. That created one hosted feature layer that contains many feature classes. We've then created hosted feature layer views off of it.

When we added a new feature class to our FGDB and then used the python code below to migrate the changes to AGOL for the existing hosted feature layer, the hosted layer views do not inherit the new feature class that was added.

fgdb = gis.content.get('XXXXXX')
print ("Updating database:",fgdb.title)
fgdb.update({}, theFGDBZip)
fgdb.publish(overwrite=True)

I'm happy to take this conversation offline and I can demonstrate what we are seeing if that is easier. Just let me know the best way to coordinate with you. Appreciate the help!

 

GlenShepherd
Esri Contributor

Hi @MattHuser ,

Apologies for the delayed response, are you still trying to work with this issue?
I've just been testing and confirming the behaviour in ArcGIS Online. Unfortunately, we're looking at a current limitation of the 'Overwrite' method for updating an existing hosted feature layer with pre-configured view layers and a bit of a grey area between the 'overwrite' and 'append' methods for updating our GIS data.

If we're adding feature classes to feature layer during an overwrite, these new feature classes will only appear in subsequent view layers built after the overwrite has been carried out. Meaning that we would need to re-configure new view layers to see the newly added layers.

If we're appending new records to an existing layer within the hosted feature layer (via the 'Update Data' option in ArcGIS Online), then we would see new feature layers within the views "as soon as updates are applied" - Documentation: https://doc.arcgis.com/en/arcgis-online/manage-data/manage-hosted-feature-layers.htm#APPEND

But unfortunately, the 'Append' method can't be used to add new feature classes into a grouped feature layer. Hence, the "grey area" I mentioned...
I've submitted a bug and enhancement for this behaviour:

  • ENH-000141026 - "ArcGIS Online: add ability for the 'Update Data' method of appending to add additional feature classes to a grouped hosted feature layer."
  • BUG-000141027 - "ArcGIS Online: If the "Overwrite" method is used to add new feature classes to an existing hosted feature layer, any pre-configured view layers will not inherit the newly grouped feature classes"

Hopefully future development efforts can address this behaviour.

MattHuser
New Contributor II

@GlenShepherd Thank you for the explanation and submitting a bug/enhancement on our behalf!  If you need client use cases, I would be happy to explain why this is needed and what additional steps we have to take as a workaround to "support" this grey area.  Feel free to reach out!

ADHSGIS_Admin
New Contributor

@GlenShepherd thanks for your response to @MattHuser's question. I was just bumping into the same issue myself. I need to add features classes to a published feature layer that have views built from it, and was investigating why the new feature classes weren't showing up in the view, which led me to this blog & comments/posts.

Sounds like the views need to be recreated each time that new feature classes are added, and swapped out in the map in order to update the apps. Is that correct? 

I'm also willing to supply use cases if needed - thanks!

erica_tefft
Occasional Contributor III

Hi @GlenShepherd 

I've got two questions about this:

1) for your Scenario #1 above, can it take some time for a new field to show up in the Fields view definition for a layer?

I've added a new field to my hosted feature layer. There are 6 different Views created from this. 3 out of 6 of these Views had the new field in the "Fields" list when I went to modify the View definition. The other 3 did not; these 3 did have an AOI defined, so I removed that, but the new field still does not show up in the Fields view definition list. Any thoughts?

2) I used the workflow described below to remove/hide the related table from my View layer which resulted in this bug - #BUG-000136618.

The solution was provided to me by Esri Customer Care (see below for more details). Now, I am unable to edit my pop-up. Deleting the View in question is not an option because it is used in almost 100 maps. Is there some sort of work around that would allow me to a) edit the pop-up or b) re-order fields?

From Esri Customer Care:

Go to the admin rest endpoint of your hosted view layer and used the delete from definition tool. We added in the following snippet of json:

  • {
    "layers": [{
    "id": 1,
    "relationships": [{
    "id": 0
    }]
    }]
    }

Once done the hosted view layer no longer shows the related table.

Any help would be appreciated,

GlenShepherd
Esri Contributor

Hi @erica_tefft ,

1. Are you saying that these view layers do have the new field in their attribute table but you are only unable to see it in the 'Define Fields' list?

2. Do you mean you're now unable to use the "configure popup" option as described here: https://support.esri.com/en/technical-article/000022958 ?

The following document describes what supported schema changes we can make to the JSON of an item via the REST endpoint: https://www.esri.com/arcgis-blog/wp-content/uploads/2014/10/How-to-Update-Hosted-Feature-Service-Sch...

Unfortunately, this does not include re-ordering the fields of an item and understandably for a view layer, this would likely break the relationship with the parent hosted feature layer...

Following up from removing/hiding the related table, did you open a new case with Esri Technical Support for this issue?

erica_tefft
Occasional Contributor III

Hi @GlenShepherd 

1. I am saying that the new field is not in my attribute table and also cannot be found under "Define Fields" when configuring the View Definition. Oddly enough, the new domains (for other fields) did show up when I followed your steps above for resetting the fields to the source. 

2. Yes, I mean that I am unable to "configure pop-up" at all for the View layers that went through the workflow I described in my initial post (point 2). This is what I see now - it basically locks up the View "Visualization" page and I need to refresh or change pages to 'unlock' it again. If I click "OK" or "Cancel" at the bottom, nothing happens. 

erica_tefft_0-1626182793288.png

 

As I mentioned above, I did open a technical support case for this issue and we logged #BUG-000136618. This was done via case number #02707008. 

I mentioned that 3 of my Views do not show my new field as an option under "Define Fields" under the View Definition. Two of these Views went through the process to hide/remove the related table, while one did not. So, it seems like I am having an issue with a one "regular view" that I did not do anything strange with, and then also with my 2 "modified views" where I have removed the related table. 

I do have another "modified view" (where I removed/hid the related table) that does show the new field. 

It makes no sense. Do you recommend I open another technical support case?

 

GlenShepherd
Esri Contributor

Hi @erica_tefft ,

I've read the bug you mentioned and as this is defective behaviour attributed to JSON edits without any currently known workaround, I'm sorry there's not much I can help with regarding the pop-ups. As for your case, I work for the Australian branch of Esri and don't have access to casework from other countries unfortunately.

It's certainly curious behaviour that you're describing.

Some troubleshooting suggestions:

  • Go through all the fields in the troublesome view layers and check for the "properties: overridden" message mentioned in 'Scenario 2' above
  • Reverse the remove/hide related table changes that you made.
  • Double-check that the AOIs have been deleted
  • Publish a new test copy of your parent hosted feature layer and go through all the same steps to create your view layers to try and track where the disconnect occurred.
  • Think about what has been done to those 3 view layers at any point in your workflow and strip back those changes (where possible) to see what might've broken the inheritance...

Being that there are a lot of moving parts to your workflow (i.e. updated hosted feature layer, six views, related tables, JSON edits) it would be quite difficult to diagnose over discussion board comments. I'd reach out to technical support again and outline everything that has been carried out with this content and/or allow them access to your organisation to take a look at the items' Settings.

You can email me as well if you like.

erica_tefft
Occasional Contributor III

Hi @GlenShepherd 

I did as you suggested w/ the following results:

  • Checked all "properties: overridden" and ensured that all fields showed as "properties: inherited", which they did. 
  • Created a new test view from the original hosted feature layer. All fields were present as they should be. 
  • Used the "Delete from Definition" tool at REST to remove the related table. Once done, all fields were still present, but I couldn't update the pop-up (which was expected). 
  • Attempted to undo my removal of the related table using the same code snippet as above (I wasn't sure what else to use) through the Add to Definition and Update Definition tools at REST. Neither worked - I received an error in both instances. I am not sure if I needed a different code snippet...

It seems like this modification just breaks the View's connection back to it's parent hosted feature layer. The weird part is, one of my problem layers (which doesn't have the new field) has not gone through the process to edit the REST definition....

For that layer I checked all fields for "properties: inherited" (they were), removed the AOI, removed the View Definition Filter, removed custom symbology and none of that made the missing field appear. 

I feel like my only solution is to re-create the Views. 

Thank you for your help. I might connect with technical support again. Ultimately, I feel like if I want to remove related tables in the future, I need to ensure the schema/pop-ups are 100% configured before doing so. 

One final question though...at the UC this week, it seemed like the View creation experience will be changing in the future. Will removal of related tables be supported in the new experience? Or will the workflow through REST still need to be used? It would be really great it for public Views a related table could be removed/hidden in some sort of officially supported way. In my example here, our related table tracks unauthorized trail use (e.g. ATVs, vandalism, etc.). This is for internal use only, so we remove the related table from the public View of our trail data inventory. 

GlenShepherd
Esri Contributor

Hi @erica_tefft ,

Sorry to reply weeks later. Did you end up working with technical support again on this behaviour at all? Any outcome?

Not sure about the new experience in future updates and whether this will be incorporated, but I think you have a pretty strong use-case for submitting an ArcGIS Idea through this domain. I'll certainly give it an up-vote if you do.

erica_tefft
Occasional Contributor III

Hi @GlenShepherd ,

I did create a new idea - https://community.esri.com/t5/arcgis-online-ideas/offer-official-support-for-removing-related-tables...

Thanks for your help around this issue. I would definitely love to see some improvements made so this is officially supported someday!

Erica 

JMitchell
New Contributor III

Hi @GlenShepherd ,

Is there an existing idea or somewhere that I could put my vote behind what @KevinMacLeodCAI suggested regarding getting rid of the locking of the parent when joins are used in a view?

I came across this blog post while looking into why having views for a layer locked the schema of the parent;  now I know that that is only the case with views using joins (which is the main reason I utilize views).

I am using several views with joins to help symbolize and label a feature layer with several related tables as I do field data collection within field maps.  I need to add several new fields in one of the related tables, but in order to do that I now have to get rid of the various view layers to make that seemingly simple change and thus have to spend time recreating the views, the associated label and symbology for each of those view layers and adding them back into my map.

Thanks,

GlenShepherd
Esri Contributor

Hi @JMitchell ,

I had a look around and looks like there is a relatively new(ish) Idea submitted for this functionality: https://community.esri.com/t5/arcgis-dashboards-ideas/allow-adding-fields-to-agol-hosted-layer-with/...

Definitely be sure to flick a 'Kudos' on that one... @KevinMacLeodCAI might like to as well. The suggestion for 'suspend' and 'refresh' functionality on a view layer join would be pretty cool to have.

KevinMacLeodCAI
Occasional Contributor II

@GlenShepherd   @JMitchell nice, I just +1'd

JMitchell
New Contributor III

@GlenShepherd Thanks, left a kudos for that idea!

About the Author
Esri Australia team member based in Melbourne. Delivering support and training across Enterprise, Desktop and Field Apps. Learning something new every day.
Labels