Automate task to rebuild spatial index of hosted feature layer on AGOL

[SamJakson]

Is there any way to automate the rebuilding of spatial indexes on a AGOL hosted feature layer? I assumed this would be a common task you could do with the ArcGIS API for python, but I can't find any information online. 

[DougGreen]

I don't see anything under the Python API or the REST API. When I check the network section of dev tools in the browser when running a rebuild index command, I see that there is an "updateDefinition" POST sent to my service at a url:

https://services6.arcgis.com/[sensitive_data]/arcgis/rest/admin/services/[layer_name]/FeatureServer/0/updateDefinition

There is a payload with a token and some info that specifies the shape field and to perform async. Perhaps it's something they haven't exposed yet. It might be good to suggest it as an idea.

[MichaelMacDermott · Accepted solution]

EDIT/UPDATE: As of 12JUN23, rebuilding of spatial indexes via update-definition is given as example number seven on the official REST documentation. Please note that this operation is performed on the feature layer and not on the feature service (REST api terminology) or FeatureLayerCollection (arcgis api for python terminology) as mentioned below for the example.

So, in my research to do the exact same thing, I found this reply from an ESRI employee which is in line with @DougGreen 's response:

https://github.com/Esri/arcgis-python-api/issues/259

For question 2, you can rebuild spatial indexes by passing in the index into update_definition.

Example:

flc = <some FeatureLayerCollection>.manager.update_definition( {"indexes":[{"name":"<index id>","fields":"Shape"}]})

This seems the intention was always to accomplish this via the update_definition tool, which is a direct wrapper for the same REST functionality.

Now that the intent is understood, we need to clear up usage a bit as there still seem to be some remaining questions and confusion.

As @DougGreen has illustrated, the AGOL website indeed constructs an "updateDefinition" POST request with a payload similar to {"indexes":[{"name":"<index id>","fields":"Shape"}]} from the example above.

However, as pointed out by pmacMaps, the required information to make this work is <index id>:

@achapkowski, for updating the spatial index, what should the value of <index id> be?

This information is both critical and non-obvious. The answer lies in the query the browser makes just prior to the "updateDefinition" POST request, which is a simple json request of the hosted feature layer. anyone may do the same by navigating to the REST endpoint, click on either the sublayer of interest or on 'All Layers and Tables' , then click on the JSON hyperlink in the top left:

The JSON will give you slightly more information than the HTML view.

(Optional) copy and paste the JSON into https://jsoneditoronline.org/ to make working with it easier

scroll down to 'Layers'

open the associated sublayer, likely index number 0

scroll down to 'indexes'

here you will find all the registered indexes for your feature class.

one of them will have the  value 'Shape' for the fields attribute. This is your spatial index. the value of the name attribute is what you are looking for.

hence, my updateDefinition POST request would have the payload  {"indexes":[{"name":"user_14456.ADDRESS_ADDRESSES_Shape_sidx","fields":"Shape"}]}

GREAT! Now we know where the information comes from and where it goes. But how do we get it there? If you dont like making REST requests by hand, I'll assume you want to use python.

As in the example by the ESRI employee above, we will be using a feature layer collection:

flc = arcgis.features.FeatureLayerCollection(serviceLayer.url, gis)

updateDict = {"indexes":[{"name":"user_14456.ADDRESS_ADDRESSES_Shape_sidx","fields":"Shape"}]}

Then we access the feature layer we intend to update (via the layers array) and invoke the manager method:

flc.layers[0].manager.update_definition(updateDict)

Barring any typos, that should get your spatial index updated.

Hope this helps!

---

[SamJakson]

Many thanks for the detailed response! Much appreciated. 

[JosephRhodes2]

Thanks Michael, this was super helpful and got my script working. To piggyback on this, if retrieving the index name programmatically is desired, you can do it like this:

import requests
from arcgis.gis import GIS
from arcgis.features import FeatureLayerCollection

gis = GIS(portal_url, username, password)
token = gis._con.token

item = gis.content.get(item_id)
layer_index = 0
service_definition_url = f"{item.url}/{layer_index}/?token={token}&f=json"
response = requests.get(service_definition_url)
    
service_definition = response.json()

for index in service_definition["indexes"]:
    if index["fields"].lower() == "shape":
        shape_index_name = index["name"]
        break
    
flc = FeatureLayerCollection.fromitem(item)
update_dict = {"indexes":[{"name": shape_index_name,
                              "fields": "Shape"}]}
    
result = flc.layers[layer_index].manager.update_definition(update_dict)

if result:
    print(f"\nSpatial index successfully rebuilt for {item.title} ({item.id}), Layer {layer_index}")