Poor performance using Python to update Hosted Feature Layer

[SteveJavins]

Hello, I am working on some Python code that updates a Hosted Feature Layer inside of ArcGIS Pro. However, performance is extremely slow. There are over 700k records in the feature class, but performance was acceptable when working with a FGDB. Accessing the hosted layer takes hours to run each step (e. g. and update cursor and select by location, see below)

Is this expected for a hosted feature layer? I'm using ArcGIS Pro 2.9.10 and Enterprise 10.9.1.

with arcpy.da.UpdateCursor('featureClassName', ['fieldName']) as cursor:

    for row in cursor:

        if row[0] == 'N':

            row[0] = None

            cursor.updateRow(row)

arcpy.management.SelectLayerByLocation('featureClassName', 'INTERSECT', 'fieldName', None, 'NEW_SELECTION', 'INVERT')

If more details are needed please let me know. Any help would be appreciated.

Thanks,

Steve

[EzraBosworth-Ahmet · Accepted solution]

Hi @SteveJavins , I recommend first exporting the data from the hosted feature layer to a gdb, performing the updates there, then overwriting the hosted feature layer. If you don't want to overwrite the service, truncating it then appending to it from the feature class works well too.

[SteveJavins]

thank you @EzraBosworth-Ahmet, appreciate the quick reply. I considered this but hoped to automate if possible. Can this be done with arcpy or would it require the Python for ArcGIS API? Any pointers to documentation?

[EzraBosworth-Ahmet]

This can all be done with arcpy. I suggest doing the process manually with the Pro tools and copying the Python command. I find this speeds up dev time quite a bit!

The Pro tools would be Copy Features (input hosted feature layer, output feature class), whatever you were doing locally before to the feature class, Truncate (input is hosted feature layer), and Append (input layer is updated feature class, target layer is hosted feature layer) 

[SteveJavins]

Hi @EzraBosworth-Ahmet , I'm attempting the truncate/append method. However, the truncate tool fails on the hosted feature layer. Since my original post, I upgraded Pro to version 3.3 which I thought supported truncating hosted layers. But it still fails. I'm also seeing things about the 'supportsTruncate' setting but don't know how to update that. The sync option is not enabled on the layer. Any thoughts? 

[DavidSolari]

Unfortunately hosted layers will almost always be slower than local file/mobile GDBs. If @EzraBosworth-Ahmet's method doesn't work you can adjust your cursor to use a SQL filter:

with arcpy.da.UpdateCursor('featureClassName', ['fieldName'], "fieldName = 'N'") as cursor:
    for row in cursor:
        cursor.updateRow([None])

Or combine a Select By Attribute with a Calculate Field (you can even do the calculation in SQL to run it on the server in modern Enterprise versions). But yeah, in the general case you're losing performance with cursor updates once the server's involved.

[SteveJavins]

Thank you @DavidSolari. I considered this as well. Or having other references to the layer in the map with definition queries. Using SQL might be cleaner and any selects or attempts to field calculate in arcpy have been brutally slow. 

[SteveJavins]

Hi @DavidSolari , I wanted to let you know I tested using where clauses and they made a HUGE difference. However, there are two steps where they may not be doable. It was a great idea that I will be using either way. Thanks again.

[PeterKnoop]

When working with Hosted Feature Layers, we've found that using the ArcGIS API for Python usually yields significantly better performance than ArcPy, especially if you apply your updates (or edits) in batches. What takes minutes with ArcPy might only take seconds with the ArcGIS API for Python.

[SteveJavins]

Good to know, thanks @PeterKnoop. I'm seeing more and more reasons to learn the API but haven't really used it yet.