topic Trying to Back Up Hosted Feature Service in Python Questions

Trying to Back Up Hosted Feature Service

MarkTukman — Mon, 25 Aug 2014 00:16:55 GMT

Does anyone have a good solution to backing up a hosted feature service using python, so a local backup can be run in task scheduler (preferably as a python script)? I tried to do this using Create Replica and the requests library, but it doesn't seem to be working. Does anyone know why the following python doesn't work?

import requests, json

payload={"geometry": '',

"geometryType": "esriGeometryEnvelope",

"inSR": '',

"layerQueries": '',

"layers": "0",

"replicaName": "read_only_rep",

"returnAttachments": 'true',

"returnAttachmentsDataByUrl": 'true',

"transportType": "esriTransportTypeURL",

"async": 'false',

"syncModel": "none",

"dataFormat": "filegdb",

"replicaOptions": '',

"f": "html"}

payload2 = json.dumps(payload)

r2 = requests.post('http://services2.arcgis.com/Pw6oQMuXLspbq6zz/arcgis/rest/services/SON_ROADS/FeatureServer/createReplica', data=payload2)

Re: Trying to Back Up Hosted Feature Service

XanderBakker — Mon, 25 Aug 2014 04:19:40 GMT

It seems there is something wrong with the transportType "esriTransportTypeURL". I changed the code to use urllib and urllib2 and it returned the following error:

{"error":{"code":400,"message":"","details":["'transportType' parameter is invalid"]}}

I changed the transportType to esriTransportTypeEmbedded and that did return a link to a ZIP file.

import json
import urllib
import urllib2

payload={"geometry": '',
    "geometryType": "esriGeometryEnvelope",
    "inSR": '',
    "layerQueries": '',
    "layers": "0",
    "replicaName": "read_only_rep",
    "returnAttachments": 'true',
    "returnAttachmentsDataByUrl": 'true',
    "transportType": "esriTransportTypeEmbedded",
    "async": 'false',
    "syncModel": "none",
    "dataFormat": "filegdb",
    "replicaOptions": '',
    "f": "json"}

url = 'http://services2.arcgis.com/Pw6oQMuXLspbq6zz/arcgis/rest/services/SON_ROADS/FeatureServer/createReplica'

data = urllib.urlencode(payload)
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
json_out = response.read()

print json_out

This printed:

{"transportType":"esriTransportTypeUrl","responseUrl":"http://services2.arcgis.com/Pw6oQMuXLspbq6zz/ArcGIS/rest/services/SON_ROADS/replicafiles/8c8720356d6e428882414e343f1dee9d.zip"}

The ZIP contains the fgdb.

Kind regards, Xander

Re: Trying to Back Up Hosted Feature Service

TrilliumLevine1 — Wed, 10 Sep 2014 14:12:16 GMT

Hi Xander,

I'm trying to make a script for replicating secured AGOL-hosted services to a local disk. I know I have to use a token for this, so I've tried integrating your code with the standard token code (see below), but it's not working for me. The token is printing, but then it's followed immediately by another message indicating that a token is needed (see screenshot). I know I must be missing something simple, but am not sure what it is...any ideas? Feedback is greatly appreciated!

import json, urllib, urllib2, os

USERNAME = "username"
PASSWORD = "pw"
REFER = "www.arcgis.com"

def urlopen(url, data=None):
    referer = "http://arcgis.com/arcgis/rest"
    req = urllib2.Request(url)
    req.add_header('Referer', referer)
    if data:
        response = urllib2.urlopen(req, data)
    else:
        response = urllib2.urlopen(req)
    return response

def gentoken(username, password, referer, expiration=60):
    #Re-usable function to get a token

    query_dict = {'username': username,
                  'password': password,
                  'expiration': str(expiration),
                  'client': 'referer',
                  'referer': referer,
                  'f': 'json'}

    query_string = urllib.urlencode(query_dict)
    tokenUrl = "https://www.arcgis.com/sharing/rest/generateToken"
    tokenResponse = urllib.urlopen(tokenUrl, urllib.urlencode(query_dict))
    token = json.loads(tokenResponse.read())
    if "token" not in token:
        print token['messages']
        exit()
    else:
        # Return the token to the function which called for it
        return token['token']

token = gentoken(USERNAME, PASSWORD, REFER)
print token


payload={"geometry": '',
    "geometryType": "esriGeometryEnvelope",
    "inSR": '',
    "layerQueries": '',
    "layers": "0",
    "replicaName": "read_only_rep",
    "returnAttachments": 'true',
    "returnAttachmentsDataByUrl": 'true',
    "transportType": "esriTransportTypeEmbedded",
    "async": 'false',
    "syncModel": "none",
    "dataFormat": "filegdb",
    "replicaOptions": '',
    "f": "json"}

url = 'http://services1.arcgis.com/0cr41EdkajvOA232/arcgis/rest/services/Forstmobil_WV_Daten_view/FeatureServer/createReplica'

data = urllib.urlencode(payload)
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
json_out = response.read()

print json_out

Re: Trying to Back Up Hosted Feature Service

DamianoMontrasio — Wed, 05 Nov 2014 13:28:26 GMT

Change row 57 by adding token to the request.

es:

url = 'http://services1.arcgis.com/0cr41EdkajvOA232/arcgis/rest/services/Forstmobil_WV_Daten_view/FeatureServer/createReplica?token=' + token

Damiano
GIS Soluzioni e Progetti | il Blog di One Team dedicato al GIS…