Python script to replicate an ArcGIS Online hosted feature service.

3238
8
Jump to solution
03-24-2016 09:52 AM
JeffWard
Regular Contributor II

I am trying to replicate a hosted feature service in a local file geodatabase to do some local analysis.  I have found this code on GitHub that I have been trying to use.  I set it up according to the instructions and I get a KeyError on line 9 below:

crValues = {'f' : 'json',
  'layers' : '0',
  'returnAttachments' : 'true',
  'token' : token }
crData = urllib.urlencode(crValues)
crRequest = urllib2.Request(crUrl, crData)
crResponse = urllib2.urlopen(crRequest)
crJson = json.load(crResponse)
replicaUrl = crJson['URL']
urllib.urlretrieve(replicaUrl, 'myLayer.json')

Error screenshot:

I was able to run all of the commands in the IDLE interactive window successfully and check the results of each command.  There is no 'URL' key in the crJson dictionary.  Here are a list of the keys that are:

[u'supportsCalculate', u'editingInfo', u'typeIdField', u'supportsValidateSql', u'advancedQueryCapabilities', u'supportsCoordinatesQuantization', u'supportsRollbackOnFailureParameter', u'allowGeometryUpdates', u'globalIdField', u'supportsAdvancedQueries', u'id', u'relationships', u'drawingInfo', u'capabilities', u'indexes', u'currentVersion', u'geometryType', u'hasStaticData', u'type', u'useStandardizedQueries', u'supportedQueryFormats', u'isDataVersioned', u'supportsAttachmentsByUploadId', u'supportsApplyEditsWithGlobalIds', u'description', u'standardMaxRecordCount', u'defaultVisibility', u'extent', u'objectIdField', u'tileMaxRecordCount', u'htmlPopupType', u'types', u'hasM', u'displayField', u'name', u'templates', u'supportsStatistics', u'hasAttachments', u'fields', u'maxScale', u'copyrightText', u'hasZ', u'maxRecordCount', u'minScale', u'maxRecordCountFactor', u'editFieldsInfo', u'serviceItemId']

The code is over two years old, have there been changes to the way requests are handled?

Any thoughts?

Many thanks.

Jeff Ward
Summit County, Utah
0 Kudos
1 Solution

Accepted Solutions
FredSpataro
Occasional Contributor III

Hi Jeff,

I think it might be the url you're sending... you might be missing the "CreateReplica" portion on the end.  If I make a request without the CreateReplica I get back the overall FeatureService description JSON which has keys that match the output you listed.  So the value of crURL should be in the format:

http://services1.arcgis.com/{id}/arcgis/rest/services/{name}/FeatureServer/CreateReplica

Fred

View solution in original post

8 Replies
FredSpataro
Occasional Contributor III

Hi Jeff:

First what request are you making ie. what's the value of crUrl?

Second, i do think that methodology is a little outdated.... Do any of the solutions in this blog post help?

Quick Tips: Consuming Feature Services with Geoprocessing | ArcGIS Blog

Do you really care about creating a "replica" which keeps everything in sync for reloading offline editing or do you just need a copy?  The export item is the best way to download a copy: ArcGIS REST API

JoshuaBixby
MVP Esteemed Contributor

I haven't had time to look at the GitHub page yet, but the error message is indicating the crJson dictionary does not have an entry for 'URL'.  I would look at the raw/returned JSON.  The code wants a key labeled 'URL', and it appears one isn't getting returned from your crRequest.

0 Kudos
JeffWard
Regular Contributor II

Yes, I listed the keys in the crJson dictionary in my original post and 'URL' wasn't among them.

Jeff Ward
Summit County, Utah
0 Kudos
AndyOmmen
Esri Contributor

Hi Jeff,

Check your token, if you aren't getting a URL string returned in your JSON object then chances are the token wasn't valid and your JSON object isn't getting built. I'm running the same code but am finding that during urlretreive method, I'm not getting JSON written to the file I'm specifying. Instead the file contains HTML that holds a div block showing restErrors with a value "Token Required".  This code was working a few weeks ago but now seems to break due to no token authentication. Maybe others are experiencing the same behavior.

Thanks

Andy

0 Kudos
JeffWard
Regular Contributor II

I think my token is valid, when I run the same code in IDLE interactively I print out the token and it gives a long string with an expiration if I remember correctly.  I also get a very large dictionary in my crJson variable, but 'URL' is not among the keys.

Jeff Ward
Summit County, Utah
0 Kudos
FredSpataro
Occasional Contributor III

Hi Jeff,

I think it might be the url you're sending... you might be missing the "CreateReplica" portion on the end.  If I make a request without the CreateReplica I get back the overall FeatureService description JSON which has keys that match the output you listed.  So the value of crURL should be in the format:

http://services1.arcgis.com/{id}/arcgis/rest/services/{name}/FeatureServer/CreateReplica

Fred

JeffWard
Regular Contributor II

Fred, I think you are right.  I just copied my url from my layer and didn't see I needed the CreateReplica.  I will have to look into this on Monday.  Thanks so much for your help.

Jeff Ward
Summit County, Utah
0 Kudos
JeffWard
Regular Contributor II

You were right Fred.  I had failed to include the createReplica method at the end of my url.  I am now able to get the URL key from the json response.  But now I am running into another error that I will post in another question.

Thanks for your help.

Jeff Ward
Summit County, Utah
0 Kudos