Recent Failure of Using ArcGIS Online Hosted Feature Layers as Inputs in arcpy

[B_NewGE]

Hello, I have a script that has ran without failure for the last 6 months but very recently started failing. The script includes lines of code that pass in a hosted feature layer from our ArcGIS Online instance into various geoprocessing tools. Here is an example:

well_points = "https://services3.arcgis.com/7FTAucs3gO3lUZyu/arcgis/rest/services/Well_Points_Input/FeatureServer/0"
well_points_sort = r'memory/well_points_fl_sort'

arcpy.management.Sort(in_dataset=well_points,out_dataset=well_points_sort,sort_field='WellName ASCENDING')

As of very recently, it started failing with these errors: 

ERROR 000732: Input Dataset: Dataset https://services3.arcgis.com/7FTAucs3gO3lUZyu/arcgis/rest/services/Well_Points_Input/FeatureServer/0 does not exist or is not supported
Failed to execute (Sort)

Here's another example with a different error:

well_master_points_hosted = "https://services3.arcgis.com/7FTAucs3gO3lUZyu/arcgis/rest/services/Well_Master_Points/FeatureServer/0"

insert_field_list = [f.name for f in arcpy.ListFields(well_master_points_hosted)]

icur = arcpy.da.InsertCursor(well_master_points_hosted, insert_field_list)
with arcpy.da.SearchCursor(mem_well_master_points, insert_field_list) as cursor:
    for row in cursor:
        icur.insertRow(row)
del icur

Error code: 400

Description: Bad syntax in request. Message: Invalid URL

As you can tell, I am using the full REST URL as the input to pass in my hosted feature layer. Can I no longer use the REST URL per the October release, and if so, how can I now correctly pass in our hosted feature layers? TIA!

[JakeSkinner]

Hi @B_NewGE, 

How are you authenticating with AGOL?  Whenever I'm using REST URLs with arcpy, I'll sign-in using the API and with arcpy.SignIntoPortal.  Ex:

# Connect to AGOL
gis = GIS('https://www.arcgis.com', username, password)
arcpy.SignIntoPortal('https://www.arcgis.com', username, password)

I've found this get's around the error about the URL not existing/being accessible.

[B_NewGE]

My organization uses SAML credentials and I was never successful in authenticating through username and password like you did above. I could try it again just to see if I can get it to work. I am using the python.exe that came with the ArcGIS Pro install as my interpreter. With that, this authentication has always worked previously: 

gis = GIS("home")

[D_Atkins]

I was curious about this, and was motivated to dig in when I saw @JakeSkinner response: I despise putting my credentials in a script if I don't have to! (And most days, we're just 'reading' data from a script, not trying to make edits or updates).

So just to confirm, from my end at least:

ArcPy doesn't seem to have any trouble reading from the Service URL; and we can Sort directly from the REST too, into a new Feature Class.  The short sample script below will show as much.  

But when I check your REST, i.e. (https://services3.arcgis.com/7FTAucs3gO3lUZyu/arcgis/rest/services/Well_Points_Input/FeatureServer/0), I see the message 'Token is Required'.  Is it possible that ownership or sharing permissions have changed recently, or the account used to invoke the script has changed?

import arcpy 
arcpy.env.addOutputsToMap = False

# ArcGIS US States REST Service:
hfc = 'https://services.arcgis.com/P3ePLMYs2RVChkJx/arcgis/rest/services/USA_Census_States/FeatureServer/0'

# Check our ability to 'Search' and Select from the Service:
count = 0
with arcpy.da.SearchCursor(hfc, ['*']) as sCursor:
    for row in sCursor:
        count+=1
print('We found', count, 'records.\n')

# Now lets try to sort by Area (SQMI), and list the results:
arcpy.management.Sort(hfc, "memory/HFC", 'SQMI')
with arcpy.da.SearchCursor("memory/HFC", ['STATE_NAME']) as sCursor:
    for row in sCursor:
        print(row[0])

[B_NewGE]

My organization has its own ArcGIS Online instance and none of our services are shared with the public, so an outside user would not be able to access our services. You are correct, the script can read the REST URL and export to a separate feature class, it's writing to the REST URL that is the problem. I have a couple scripts that requests data from a third party via their API, and writes to our layers in AGOL. The settings of the feature layer(s) have not changed at all, which is why I thought the October release may have changed parameters for calling REST URLs.

[B_NewGE · Accepted solution]

This could be a red herring but I believe I have fixed my issue. I changed two things specifically. Firstly, when creating a variable using the REST URL, I used double quotes instead of single quotes. Secondly, I changed the way I was writing to memory:

# original way I was writing to memory:
fl = r'memory/layer'

# new way I am writing to memory:
fl = 'memory/layer'

The difference seems arbitrary but I believe it made the difference. All of the scripts are running as they should now. I'll accept this as the solution but I'll update this thread if the issue reoccurs. Thanks all!