topic Re: SearchCursor cannot open Hosted feature layer in ArcGIS API for Python Questions

SearchCursor cannot open Hosted feature layer

MikeMacRae — Thu, 24 Jun 2021 21:04:41 GMT

I am trying to use arcpy cursors on a hosted feature layer containing a geodatabase with multiple tables and feature classes. After digging around the internet, a few posts seem to allude that I can do this:

https://gis.stackexchange.com/questions/363932/how-can-i-use-updatecursor-in-agol-layers

https://community.esri.com/t5/arcgis-api-for-python-questions/updatecursor-with-a-hosted-table/td-p/781002

Whereby, it seems the process is to:

Access ArcGIS Online account
Search for the hosted feature layer in my content
Use the URL for the item as the first parameter in the arcpy cursor (i.e. arcpy.da.SearchCursor(item.url, fields))
Run Cursor

So, code looks something like this:

from arcgis.gis import GIS import arcpy gis = GIS("https://www.arcgis.com", "username", "password") hfl = gis.content.get('itemitd') url = hfl.tables[8] with arcpy.da.SearchCursor(url, '*') as cursor: for row in cursor: print(row)

Which returns the following error:

RuntimeError: cannot open 'https://services6.arcgis.com/yadda/arcgis/rest/services/nameofhostedfeaturelayer/FeatureServer/10'

I tried using a slightly different approach to grab the item, but is pretty much the same as the above:

hfl = gis.content.search(query="title:nameofhostedfeaturelayer, owner:username", max_items=1000) url = hfl[0].tables[8].url with arcpy.da.UpdateCursor(url , "*") as cursor: for row in cursor: row[0] = "hello" cursor.updateRow(row)

Which yields the same error.

Couple notes:

I own the data
It is shared to a few groups, but not publicly
I've allowed edits on the hosted feature layer and any other setting that may or may not allow the script to see and read the hosted feature layer
It appears I do have access to the hosted feature layer and I can perform other operations on it such as printing out items or values:

for v in hfl.values(): print(o) for i in hfl.items(): print(c)

Any suggestions why I am receiving that error?

Re: SearchCursor cannot open Hosted feature layer

DougBrowning — Thu, 24 Jun 2021 21:16:54 GMT

What I do is load the HFS in once to a FeatureSet then I can use it many times without more calls. I know not your question but it may help you.

specrichF = arcpy.FeatureSet()
specrichF.load(specrichURL)

with arcpy.da.SearchCursor(specrichF,......

Hope that helps

Re: SearchCursor cannot open Hosted feature layer

MikeMacRae — Thu, 24 Jun 2021 21:25:37 GMT

Thanks @DougBrowning I tested your suggestion (think that was a similar approach in one of the other links I posted above) and it returns the following error:

RuntimeError: RecordSetObject: Cannot open table for Load

So, there's something that is preventing the cursor from opening/accessing the table...

Re: SearchCursor cannot open Hosted feature layer

DougBrowning — Fri, 25 Jun 2021 00:20:35 GMT

I knew I should posted that if it is a table you need to use RecordSet

soilpitF = arcpy.RecordSet()

Re: SearchCursor cannot open Hosted feature layer

MikeMacRae — Fri, 25 Jun 2021 16:40:56 GMT

Thanks again @DougBrowning I think the coding you are providing is correct and you are right. I am trying this on a table but I also tested on a layer as well and it just doesn't seem to be accessing either of them with the cursor.

I did come across an ESRI support document that speaks to the errors I am receiving and how a map service can't access the attribute table, but according to the metadata on my hosted feature service, it is a feature layer, not a map service:

Re: SearchCursor cannot open Hosted feature layer

DougBrowning — Fri, 25 Jun 2021 17:10:47 GMT

I should have posted a sample URL too. Does it end in FeatureServer? Then you give it the number of the layer you want. You may be giving it the /MapServer URL instead.

url = "https://services1.arcgis.com/Hp6G80Pky0om7QvQ/ArcGIS/rest/services/MyServiceName/FeatureServer/0"

soilpitF = arcpy.RecordSet()
soilpitF.load(url)

Sorry I should have posted the complete code. Seems like you are trying to look it up instead? I would just give it the URL to the service.

You get this from the Item page of the service

Hope that works.

Re: SearchCursor cannot open Hosted feature layer

HenrikSchmidt — Thu, 07 Mar 2024 13:40:01 GMT

What finally made it work for me was to add the portal connection in ArcGIS Pro.

Re: SearchCursor cannot open Hosted feature layer

DougBrowning — Thu, 07 Mar 2024 14:40:58 GMT

Based on a notification updating this post.

In the end I found out that using RecordSet or FeatureSet actually just makes a pointer to the online data and does not load anything into a Python var or memory like I thought. Found out once the service grew in size my script starting taking 30-60 mins to run!

So I totally gave up on using it and instead specifically copy everything into memory like this

arcpy.conversion.FeatureClassToFeatureClass(plotsURL, "memory", "plots")
plotsF = "memory\\plots"

The script now runs in 1-3 minutes! Way better way to do it. Also note the new memory in Pro vs the older in_memory is also much faster. Changing this cut the script run time in half again. I have since converted all my scripts to copying to memory right away at the top of the script and it is working really well and crazy fast.

Also note I do the second line plotsF = "memory\\plots" just so that I can use var type ahead in PyScripter. You could write out "memory\\plots" each time but a var is way easier. Also note GP tools may not like the \\ part so you can instead use plotsF = os.path.join(memory,plots) and I am pretty sure that was working fine but I forget.

Sorry for the late update but hope it helps someone.