read and remove replicas from AGOL

[kapalczynski]

I have put together a bit of code from examples that looks at a service and loops through the list of replicas and then removes them.  I can get this to work on the example Service from ESRI but cannot get it to work from my data.

We are federated and are publishing the data from ArcPro and the result is a hosted Feature Layer in AGOL.

Basically what I am doing is using Field Maps to create a few downloadable sections for the filed to download and view.  This is what is creating the replicas.  

What I want to do is come around once a month or so and programmatically remove the replicas

If you navigate to the service that is commented out you will see a replica link but there are none there.

BUT if you go into this Feature Layer while signed into AGOL you will see there are 3 of them.

I guess my question is what URL do I use to get to my Hosted Feature Layer where I will be able to see the Replicas to unregister them?

# connect to a GIS
import arcgis
from arcgis.gis import GIS
import arcgis.features
import time

gis = GIS() # connect to www.arcgis.com anonymously. 
            # we will use a public sync enabled feature layer

duration = 1440 # thats minutes in 24 hours

# THIS URL WORKS GREAT
url = 'https://sampleserver6.arcgisonline.com/arcgis/rest/services/Sync/WildfireSync/FeatureServer/'

# TRYING TO GET TO THE FEATURE LAYER TO DO THE SAME
#url = 'https://services.arcgis.com/p5v98VHDX9Atv3l7/ArcGIS/rest/services/VDOT_Bridges_and_Culverts_Inspection/FeatureServer'


bridge_layer = arcgis.features.FeatureLayerCollection(url, gis)
replica_list = bridge_layer.replicas.get_list()
print ("There are " + str(len(replica_list)) + " replicas")
replica1 = bridge_layer.replicas.get(replica_list[0]['replicaID'])

time.localtime(replica1['creationDate']/1000) #dividing by 1000 to convert micro seconds to seconds
ten_min_earlier_epoch = time.time() - duration
ten_min_earlier_epoch

removal_list = []
for r in replica_list:
    temp_r = bridge_layer.replicas.get(r['replicaID'])
    temp_dict = {'replica_id': r['replicaID'],
                'creationDate':temp_r['creationDate']/1000}
    #check
    if temp_dict['creationDate'] < ten_min_earlier_epoch:
        removal_list.append(temp_dict)
        print(temp_dict)
        
# REMOVE THE REPLICAS
#for r in removal_list:
#    result = bridge_layer.replicas.unregister(r['replica_id'])
#    print(result)
    

[kapalczynski]

Alright this is where I am... weird stuff going on and confused.

Again I can run this on the example ESRI service but for some reason not on my hosted service.

Funny thing is if I add "REPLICAS" to the URL and type this into a web browser I can see the replicas that are on the service....

https://services.arcgis.com/p5v98VHDX9Atv3l7/arcgis/rest/services/VDOT_Bridges_and_Culverts_Inspecti...replicas?f=json&token=gdSsb-7696a1taTcjkRhiziBStFD13MPvhHVgDx4BPVF38hEoiB6KN0nxyKORuGo7wNgceLMtk5oI0lCcEGLjBv2hD5ygyaUtoFxYuEef8JAcFbasR6INZp_qi8AmKkLVdYiGfymOQZAhCNZy-EeDuBZfAvRBOMVk9pOi0yyDw0.

But I am building the string for Replicas later with the above variable 'replica_list' which is causing the error somehow...

Bad request error here:  replica_list = bridge_layer.replicas.get_list()

Note that bridge_layer variable is returning this:

[<FeatureLayer url:"https://services.arcgis.com/p5v98VHDX9Atv3l7/arcgis/rest/services/VDOT_Bridges_and_Culverts_Inspection/FeatureServer?f=json&token=gdSsb-7696a1taTcjkRhiziBStFD13MPvhHVgDx4BPVF38hEoiB6KN0nxyKORuGo7wNgceLMtk5oI0lCcEGLjBv2hD5ygyaUtoFxYuEef8JAcFbasR6INZp_qi8AmKkLVdYiGfymOQZAhCNZy-EeDuBZfAvRBOMVk9pOi0yyDw0./0">]

Thoughts?

# connect to a GIS
import arcgis
from arcgis.gis import GIS
import arcgis.features
import time
from arcgis.features import FeatureLayerCollection

gis = GIS("https://vdot.maps.arcgis.com", "username", "xxxxx")
token = gis._con.token
print ("")
print ("Token Value: ")
print (token)

duration = 1440 # thats minutes in 24 hours

url = 'https://services.arcgis.com/p5v98VHDX9Atv3l7/arcgis/rest/services/VDOT_Bridges_and_Culverts_Inspection/FeatureServer?f=json&token='
tokenvalue = token

newURL = str(url + token)
print (newURL)

bridge_layer = arcgis.features.FeatureLayerCollection(newURL, gis)
print (bridge_layer)
print ("--------------------------------")
type(bridge_layer)
layerList = bridge_layer.layers
print(layerList)
isSynced = bridge_layer.properties.syncEnabled
print ("--------------------------------")
print(isSynced)
print ("--------------------------------")

replica_list = bridge_layer.replicas.get_list()

print ("---------------------")
print ("There are " + str(len(replica_list)) + " replicas")
print ("---------------------")

replica1 = bridge_layer.replicas.get(replica_list[0]['replicaID'])

time.localtime(replica1['creationDate']/1000) #dividing by 1000 to convert micro seconds to seconds
ten_min_earlier_epoch = time.time() - duration
ten_min_earlier_epoch

removal_list = []
for r in replica_list:
    temp_r = bridge_layer.replicas.get(r['replicaID'])
    temp_dict = {'replica_id': r['replicaID'],
                'creationDate':temp_r['creationDate']/1000}
    #check
    if temp_dict['creationDate'] < ten_min_earlier_epoch:
        removal_list.append(temp_dict)
        print(temp_dict)
        

Result and ERROR

========= RESTART: C:\Users\xxxxx\Desktop\BridgeInspection\RemoveReplicas.py =======

Token Value:
gdSsb-7696a1taTcjkRhiziBStFD13MPvhHVgDx4BPVF38hEoiB6KN0nxyKORuGo7wNgceLMtk5oI0lCcEGLjBv2hD5ygyaUtoFxYuEef8JAcFbasR6INZp_qi8AmKkLVdYiGfymOQZAhCNZy-EeDuBZfAvRBOMVk9pOi0yyDw0.
--------------------------------
https://services.arcgis.com/p5v98VHDX9Atv3l7/arcgis/rest/services/VDOT_Bridges_and_Culverts_Inspection/FeatureServer?f=json&token=gdSsb-7696a1taTcjkRhiziBStFD13MPvhHVgDx4BPVF38hEoiB6KN0nxyKORuGo7wNgceLMtk5oI0lCcEGLjBv2hD5ygyaUtoFxYuEef8JAcFbasR6INZp_qi8AmKkLVdYiGfymOQZAhCNZy-EeDuBZfAvRBOMVk9pOi0yyDw0.
--------------------------------
<FeatureLayerCollection url:"https://services.arcgis.com/p5v98VHDX9Atv3l7/arcgis/rest/services/VDOT_Bridges_and_Culverts_Inspection/FeatureServer?f=json&token=gdSsb-7696a1taTcjkRhiziBStFD13MPvhHVgDx4BPVF38hEoiB6KN0nxyKORuGo7wNgceLMtk5oI0lCcEGLjBv2hD5ygyaUtoFxYuEef8JAcFbasR6INZp_qi8AmKkLVdYiGfymOQZAhCNZy-EeDuBZfAvRBOMVk9pOi0yyDw0.">
--------------------------------
[<FeatureLayer url:"https://services.arcgis.com/p5v98VHDX9Atv3l7/arcgis/rest/services/VDOT_Bridges_and_Culverts_Inspection/FeatureServer?f=json&token=gdSsb-7696a1taTcjkRhiziBStFD13MPvhHVgDx4BPVF38hEoiB6KN0nxyKORuGo7wNgceLMtk5oI0lCcEGLjBv2hD5ygyaUtoFxYuEef8JAcFbasR6INZp_qi8AmKkLVdYiGfymOQZAhCNZy-EeDuBZfAvRBOMVk9pOi0yyDw0./0">]
--------------------------------
True
--------------------------------
Traceback (most recent call last):
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 610, in _handle_response
data = resp.json()
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\models.py", line 900, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\xxxxxx\Desktop\Bridge Inspection\RemoveReplicas.py", line 45, in <module>
replica_list = bridge_layer.replicas.get_list()
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\features\managers.py", line 553, in get_list
return self._fs._replicas
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\features\layer.py", line 4323, in _replicas
return self._con.get(path=url, params=params)
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 506, in get
ignore_error_key=ignore_error_key,
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 613, in _handle_response
raise Exception(resp.text)
Exception: Bad Request

[kapalczynski]

Anyone have any thoughts out there?

[JoshuaFlickinger · Accepted solution]

Hi kap,

I used a portion of your code to automate my own replica unregistering.  I'm not sure why your code isn't working, but I don't think it's the URL.  Granted I did mine a smidge differently, but the URL looks fairly similar to the one you commented out.  If I had to guess, it might have to do with the token.  I get lost in the weeds a little bit when it comes to account security, so I use Jupyter Notebooks tied to ArcGIS Pro, where I'm signed in to Pro with the same account as my AGOL account.  You'll notice I don't need to pass credentials to 'GIS()' or a token value to my URL.  I think Pro handles all that for me.  Just a thought - I'm not sure.  At any rate, here is my code that worked:

import arcgis
from arcgis.gis import GIS
import arcgis.features
gis = GIS()
grove_layer = arcgis.gis.Item(gis, itemid="xxxxxxxxxxxxxxxxxxxxxxx")
grove_layer_flc = arcgis.features.FeatureLayerCollection(grove_layer.url, gis)
replica_list = grove_layer_flc.replicas.get_list()
for r in replica_list:
    replica = grove_layer_flc.replicas.get(r['replicaID'])
    replica_id = replica["replicaID"]
    print('Unregistering ' + replica_id)
    grove_layer_flc.replicas.unregister(replica_id)