topic Re: Updating/appending data of feature layer does not properly update data in infographics? in ArcGIS Online Questions

Updating/appending data of feature layer does not properly update data in infographics?

JennyKuan — Sat, 24 Oct 2020 23:47:12 GMT

Hi,

I created a Web Map with two different infographics displaying the number of votes per political party/candidate. These numbers will need to be updated every ten minutes or so, and I tested out whether "Update Data"/"Append data to layer" would properly update the data inside the infographics. It did something very strange - instead of displaying the proper numbers in the updated table, it multiplied the values! I have appended the table multiple times now but, it keeps multiplying the values - in this instance, multiplying the number of votes.

I want to be able to click the box "Update existing features" but it is grayed out. I have unique IDs in my Excel sheet but for some reason, it does not show in the Feature Layer table.

Has anyone ever experienced something like this before?

Please see screenshots below for reference.

Thank you,

Jenny

Re: Updating/appending data of feature layer does not properly update data in infographics?

Anonymous User — Wed, 28 Oct 2020 23:53:56 GMT

Hi Jenny Kuan‌,

You will also need to make sure that the field that is unique in your spreadsheet is designated as unique from the Data tab of the hosted feature layer: Manage hosted feature layers—ArcGIS Online Help | Documentation

If Sync is enabled, update existing features will also be grayed out. You can disable Sync from the Settings tab of your layer.

Without the update existing features box checked, it makes sense that your chart counts that sort on attributes are increasing each time. This is because each feature in the spreadsheet will be treated as a new feature and added to the count.

I hope this helps, if you can send me a scrubbed version of your spreadsheet I would be happy to take a look on my end as well.

Thanks,

-Peter

Re: Updating/appending data of feature layer does not properly update data in infographics?

JennyKuan — Thu, 29 Oct 2020 13:39:50 GMT

Hi Peter,

Thank you for the reply. Despite following those steps, I still was unable to get my data to append properly. I believe it might be due to my limited roles as I am not an admin for my organization on ArcGIS Online.

Thank you for your help though,

Jenny

Re: Updating/appending data of feature layer does not properly update data in infographics?

Anonymous User — Thu, 29 Oct 2020 14:44:09 GMT

Hi Jenny,

Thanks for the response; if you own the feature layer it should not matter that you are not an administrator. If the issue were permissions related you would not be able to append at all. I would recommend getting a support case started with Esri Support - let me know and I can get that started for you.

Best regards,

-Peter

Re: Updating/appending data of feature layer does not properly update data in infographics?

DataOfficer — Wed, 02 Dec 2020 12:08:30 GMT

Hi @Anonymous User ,
I am experiencing similar issues as those described by Jenny. I am trying to update (not just append) several hosted feature layers in ArcGIS Online but the update existing features checkbox is greyed out in all the layers I have tested. This is despite following this guidance and ensuring that Sync is disabled, that there is a unique identifier present in both the feature layer and the dataset being used for the updated. I am struggling to work out what is preventing the update existing features option from being accessible. Do you have any suggestions? Furthermore, is there an alternative tool in ArcGIS Pro that allows for update, rather than append?
Many thanks,
Rob

Re: Updating/appending data of feature layer does not properly update data in infographics?

Anonymous User — Wed, 09 Dec 2020 22:22:31 GMT

Hi @DataOfficer - can you share a copy of your source data here so I can test on my end?

A few ideas:

Even if the values are unique, ArcGIS Online has to know they are unique. In Data > Fields > choose a field > does it show as unique?

You might be interested in the Update Features request in the Rest API. I've seen this used with a lot of efficacy for smaller, more frequent updates. How many total features are you looking to update and how frequently?

That said, it would be best to get Append working to update existing features - once I can test out the data you're working with I think I can provide some next steps.

Thanks,

-Peter

Re: Updating/appending data of feature layer does not properly update data in infographics?

DataOfficer — Fri, 18 Dec 2020 18:13:29 GMT

Hi Peter,

Are you happy for me to add you to a group so you can access the content?

Yes the fields show as unique as you demonstrate.
Thank you for your suggestion. I have not tried that approach yet and would prefer to get things working through the user interface as there will be other staff who are less specialist needing to perform updates and overwrites to datasets. There are several datasets in question, currently ranging from 10's of features to ~7000. In the longterm however we will be working with some layers containing >500,000 features. Frequency is also variable, but it will generally be fairly sporadic (i.e. once every 1-3 months).

I have just created a very simple point layer in ArcGIS Pro and uploaded it to ArcGIS Online to test the Append/Overwrite functionality. After setting a field as UniqueID, when I upload a slightly edited csv, it does give me the option to Update existing features, but the only field from the ArcGIS Online feature layer that it picks up as unique to match to the csv is the OBJECTID. This is despite another field being set as unique. Furthermore it appears to struggle to match numeric (integer) entries from the csv. If I try running the update using OBJECTID as the unique identifier (as it won't let me select any others), I get the following error: "Object reference not set to an instance of an object.".

I have also tried running the update with a text (string) field as the unique identifier on this test dataset, but this returns the same error as above.

Thanks,
Rob

Re: Updating/appending data of feature layer does not properly update data in infographics?

Anonymous User — Fri, 18 Dec 2020 19:38:37 GMT

Hi Rob,

Thanks for the details on this - if you would add PeterK_ess to a group that has an export-enabled version of the hosted feature layer and a sample CSV used for updates I will take a look.

Best,

-Peter

Re: Updating/appending data of feature layer does not properly update data in infographics?

DataOfficer — Mon, 21 Dec 2020 12:43:02 GMT

Hi Peter,

I have invited you to a group with the feature layer and a sample CSV for updates. I've added details of the issues and steps I have tried in the group description.

Many thanks,
Rob

Re: Updating/appending data of feature layer does not properly update data in infographics?

Anonymous User — Wed, 23 Dec 2020 20:37:06 GMT

Thanks Rob - about to drop you a private message with my testing results/observations.

Re: Updating/appending data of feature layer does not properly update data in infographics?

Anonymous User — Wed, 23 Dec 2020 20:58:43 GMT

@DataOfficer

I got a warning that you do not have private messages enabled. If you go to your profile settings > preferences > toggle the drop-down to private messenger > you can turn this functionality on if you wish. Here is what I would have sent via a DM:

Testing:

I initially published the layer from an exported FGDB
Setting Ref_uniqueID values as unique failed
- The error message was that the field was not unique - confirmed in Excel that the field is unique. Not sure what was going on with this and it may warrant some further research. However since this isn't the issue we're currently troubleshooting, worked around it for now.
Successfully used Python API method clone_items to copy the layer
- This maintained the setting of the Ref_uniqueID field as unique
- Tried to append records from incl. csv (update existing features was grayed out)
Looked at settings of cloned feature layer and noticed that Keep track of created and updated features was checked on and grayed out. This is the change tracking setting, different than keep track of who created and updated features. From the append doc: "You can only add new data to a hosted feature layer that has synchronization or change tracking (Keep track of created and updated features) enabled."
I then looked at the admin endpoint of the layer and viewed the service definition. I noticed that there were settings that indicated change tracking had been enabled on hosted feature layer views: "hasChangeTrackingEnabledViews" : true
- I'm thinking that since I cloned this layer from your hosted feature layer, it copied across the entire service definition, including information that the layer has views with certain properties.

Suggestions:

From everything I can tell, change tracking is preventing using append to update existing features at the moment.
I would look at the views created from this hosted feature layer and see if they have change tracking enabled, and if they do, evaluate if it is a requirement. If not, then you can toggle this off on the views and then ultimately on the hosted feature layer.
- I expect that this will enable updating existing features when appending.

I hope this helps, let me know how it goes.

Thanks,

-Peter

Re: Updating/appending data of feature layer does not properly update data in infographics?

DataOfficer — Thu, 24 Dec 2020 12:13:20 GMT

@Anonymous User,
For some reason Private Messenger isn't an option in my settings.
Thank you for looking in to this. I have now disabled Keep track of created and updated features in both the hosted view layers and the hosted layer itself. However, Update existing features is still greyed out in the Append Data window.

Re: Updating/appending data of feature layer does not properly update data in infographics?

Anonymous User — Thu, 31 Dec 2020 00:46:24 GMT

Hi @DataOfficer - thanks for changing this setting. Cloning the item I can confirm that the service change tracking setting is no longer there, which is good.

On the cloned service, I am also given the opportunity to append with updating features and selecting the Ref_UniqueID field as the unique identifier. However this hangs and does not complete. The oddity with the unique IDs mentioned above when republishing from FGDB (the error message that they are not unique) was added to when I downloaded the hosted feature layer as a CSV; it only exports about 1/4 of the records. However when I publish this subset as a hosted feature layer, I can append with update features and use the Ref_UniqueID field as the unique identifier.

Would it be possible to add to the Online group or send to pklingman@esri.com a source copy of the data that hasn't been published to Online as a hosted feature layer? I want to take the existing service out of the picture and try a test workflow start-to-finish with the data.

Thanks for your help and patience on this one,

-Peter