Help Understanding Attribute Indexes on Feature Layers

[PatrickMcKinney99]

I am looking for help in understanding Feature Layer Attribute Indexes.  

Specifically:

1. Does adding attribute indexes to a feature layer require more storage credits than if there are no attribute indexes added?

2. When a feature layer is overwritten, do you need to re-create the index because the data has changed?

Thanks,

Patrick McKinney

[ChristopherCounsell]

Storage credits comes in two ways:

• Feature storage (200x credits more per mb)
• Attachment storage

Enabling sync and editor tracking can increase the size of feature storage, as it creates backend tables.I have seen this, where a < 1mb layer was 10gb due to several years of editing and offline use.

I would have assumed that an index does similar however testing it on a small service, adding 3 attribute indexes did not increase the size of the item. Editing the data then increased it, but then removing the indexes did not decrease it. So as far as I could see, not a single kb was added/removed from the 144kb dataset when adding/removing indexes.

I would be more curious as to what volume of credit consumption would be a concern around this. I would assume that reducing content, generalizing geometry, reducing fields, and occasionally toggling sync to clear backend tables would be more beneficial then indexing. And at that volume of usage where these are of a serious concern, paying for a premium data store would also be a consideration (and it comes with a 'free' amount of storage capacity).

If the data is frequently edited, then general performance would be more a concern than the storage of the indexes.

I'm also hoping for more technical information on these. Assuming some of the recently released dev summit videos may have some but if anyone can identify which session or if there are other available resources that would be great for @PatrickMcKinney99 and I.

https://mediaspace.esri.com/channel/2023+Esri+Developer+Summit/292702072

Re overwriting, the documentation indicates that all web layer stuff will need to be redone. I'd assume this would include the indexes that have been added. If they somehow remain, they would still need to re-index due to the insertion of new records.

[BrianFausel]

I too wish there was more info or a video about the attribute and full text indexes created on ArcGIS Online hosted feature services. With the recent February 2024 AGOL update (and/or possibly the Oct 2023 one) there seems to have been a change to how they are created if you don't manually create them.

I can confirm the #2 question from Patrick is a YES: "When a feature layer is overwritten, do you need to re-create the index because the data has changed?" I found that if I use ArcGIS Pro to overwrite a hosted feature service in AGOL the indexes will be gone. And this will change how the end-user searching works in your apps. You will want to re-index all of the attributes used in searches. 

Interestingly it also seems that attribute indexes are created automatically when you set up searches in the AGOL web map (Settings --> Enable Search) or in Experience Builder search widget. I think the ExB part was added in the Feb 2024 update in relation to a searching bug fix in that app because I don't recall seeing that happen before the update. The indexes are created if they don't already exist on the feature service, and are given a title that indicate their source (example "exb-{app id}-widget..."). If you manually create the indexes on the hosted feature service then they are not created during the app configuration.

This 2022 blog post kind of hints at some of this.

There is also some useful info in this developers doc.

The Experience Builder search widget doc also has some info about the "breaking change" of full text indexes added last fall.