Python script to replicate an ArcGIS Online hosted feature service.

[JeffWard]

I am trying to replicate a hosted feature service in a local file geodatabase to do some local analysis.  I have found this code on GitHub that I have been trying to use.  I set it up according to the instructions and I get a KeyError on line 9 below:

crValues = {'f' : 'json',
  'layers' : '0',
  'returnAttachments' : 'true',
  'token' : token }
crData = urllib.urlencode(crValues)
crRequest = urllib2.Request(crUrl, crData)
crResponse = urllib2.urlopen(crRequest)
crJson = json.load(crResponse)
replicaUrl = crJson['URL']
urllib.urlretrieve(replicaUrl, 'myLayer.json')

Error screenshot:

I was able to run all of the commands in the IDLE interactive window successfully and check the results of each command.  There is no 'URL' key in the crJson dictionary.  Here are a list of the keys that are:

[u'supportsCalculate', u'editingInfo', u'typeIdField', u'supportsValidateSql', u'advancedQueryCapabilities', u'supportsCoordinatesQuantization', u'supportsRollbackOnFailureParameter', u'allowGeometryUpdates', u'globalIdField', u'supportsAdvancedQueries', u'id', u'relationships', u'drawingInfo', u'capabilities', u'indexes', u'currentVersion', u'geometryType', u'hasStaticData', u'type', u'useStandardizedQueries', u'supportedQueryFormats', u'isDataVersioned', u'supportsAttachmentsByUploadId', u'supportsApplyEditsWithGlobalIds', u'description', u'standardMaxRecordCount', u'defaultVisibility', u'extent', u'objectIdField', u'tileMaxRecordCount', u'htmlPopupType', u'types', u'hasM', u'displayField', u'name', u'templates', u'supportsStatistics', u'hasAttachments', u'fields', u'maxScale', u'copyrightText', u'hasZ', u'maxRecordCount', u'minScale', u'maxRecordCountFactor', u'editFieldsInfo', u'serviceItemId']

The code is over two years old, have there been changes to the way requests are handled?

Any thoughts?

Many thanks.

[FredSpataro]

Hi Jeff:

First what request are you making ie. what's the value of crUrl?

Second, i do think that methodology is a little outdated.... Do any of the solutions in this blog post help?

Quick Tips: Consuming Feature Services with Geoprocessing | ArcGIS Blog

Do you really care about creating a "replica" which keeps everything in sync for reloading offline editing or do you just need a copy?  The export item is the best way to download a copy: ArcGIS REST API

[JoshuaBixby]

I haven't had time to look at the GitHub page yet, but the error message is indicating the crJson dictionary does not have an entry for 'URL'.  I would look at the raw/returned JSON.  The code wants a key labeled 'URL', and it appears one isn't getting returned from your crRequest.

[JeffWard]

Yes, I listed the keys in the crJson dictionary in my original post and 'URL' wasn't among them.

[AndyOmmen]

Hi Jeff,

Check your token, if you aren't getting a URL string returned in your JSON object then chances are the token wasn't valid and your JSON object isn't getting built. I'm running the same code but am finding that during urlretreive method, I'm not getting JSON written to the file I'm specifying. Instead the file contains HTML that holds a div block showing restErrors with a value "Token Required".  This code was working a few weeks ago but now seems to break due to no token authentication. Maybe others are experiencing the same behavior.

Thanks

Andy

[JeffWard]

I think my token is valid, when I run the same code in IDLE interactively I print out the token and it gives a long string with an expiration if I remember correctly.  I also get a very large dictionary in my crJson variable, but 'URL' is not among the keys.

[FredSpataro · Accepted solution]

Hi Jeff,

I think it might be the url you're sending... you might be missing the "CreateReplica" portion on the end.  If I make a request without the CreateReplica I get back the overall FeatureService description JSON which has keys that match the output you listed.  So the value of crURL should be in the format:

http://services1.arcgis.com/{id}/arcgis/rest/services/{name}/FeatureServer/CreateReplica

Fred

[JeffWard]

Fred, I think you are right.  I just copied my url from my layer and didn't see I needed the CreateReplica.  I will have to look into this on Monday.  Thanks so much for your help.

[JeffWard]

You were right Fred.  I had failed to include the createReplica method at the end of my url.  I am now able to get the URL key from the json response.  But now I am running into another error that I will post in another question.

Thanks for your help.