Python script to replicate hosted feature service on ArcGIS Online including attachments.

5947
10
Jump to solution
03-28-2016 03:13 PM
JeffWard
Occasional Contributor III

I am trying to get this script to work for an application for one of our departments.  If you look at the comment by stuartalexandersanders in the GitHub link - that is the same error I am getting.  I'm hoping Alex Sanders​ is the same person and has fixed it?

gtUrl = 'https://www.arcgis.com/sharing/rest/generateToken'
gtValues = {'username' : 'MyUsername',
  'password' : 'MyPassword',
  'referer' : 'http://www.arcgis.com',
  'f' : 'json' }
gtData = urllib.urlencode(gtValues)
gtRequest = urllib2.Request(gtUrl, gtData)
gtResponse = urllib2.urlopen(gtRequest)
gtJson = json.load(gtResponse)
token = gtJson['token']

### Create Replica ###
### Update service url HERE ###
crUrl = 'http://services2.arcgis.com/MyServices/arcgis/rest/services/MyServicName/FeatureServer/createReplica'

That code works and I am able to get a token for the feature service.

This next block is where I am having my trouble -

crValues = {'f' : 'json',
  'layers' : '0',
  'returnAttachments' : 'true',
  'token' : token }
crData = urllib.urlencode(crValues)
crRequest = urllib2.Request(crUrl, crData)
crResponse = urllib2.urlopen(crRequest)
crJson = json.load(crResponse)
replicaUrl = crJson['URL']
urllib.urlretrieve(replicaUrl, 'myLayer.json')

The error I get is from the urlretrieve call.  myLayer.json is not a json file but an html file that says "Token Required".  The error text in python is:

Any ideas why this would happen?  Apparently it was working for Alex before the update to ArcGIS Online earlier this month.

Thanks,

Jeff

Jeff Ward
Summit County, Utah
0 Kudos
1 Solution

Accepted Solutions
KhaledHassen
Esri Contributor

Hi Jeff,

Not sure what the url looks like but if the token is the first parameter in the url which I think it is, you will have to use:

?token={your-token}. Pl. not the use of ? instead of &.

Khaled

View solution in original post

10 Replies
JeffWard
Occasional Contributor III

Fred Spataro​, got any ideas?  You asked me in my earlier question if I could just use the Export Item method, but I need the attachments and didn't see any parameters to include attachments with Export Item.

Jeff Ward
Summit County, Utah
0 Kudos
FredSpataro
Occasional Contributor III

Jeff,

The export item method 'should' bring back ALL the data from the entire feature service.  I haven't tried it with attachments but did have a service with multiple layers and tables and got back a file geodatabase will all the items... use exportFormat as file geodatabase and leave the layers parameter empty. 

KhaledHassen
Esri Contributor

In the last release we have in online, download the json file/sqlite from create replica API will require a token. This is to prevent unauthenticated users from download the create replica output by other users. Pl. when download the output from create or sync replica append a &token={your-token} to the json file url. Pl. let us know if this does not work.

Thanks

Khaled

JeffWard
Occasional Contributor III

I was actually thinking this as I was driving home last night.  So I need to get another token for the url to the json file (replicaUrl in my code)?

Jeff Ward
Summit County, Utah
0 Kudos
JakeSkinner
Esri Esteemed Contributor

You may want to try the following tool:

Download ArcGIS Online Feature Service or ArcGIS Server Feature/Map Service

It's setup as a GP tool so it's easier to configure.  Let me know if you run into any errors.  I'm seeing some issues trying to download older services that contain attachments.

JeffWard
Occasional Contributor III

Jake,

Thanks for the input.  It looks like you have done what I am trying to do, but I think the code you use for the attachments is the same code I am trying to use in my script (line 286 in your code is where I hit my problem).  So it is running into the same problem I am having.  I was able to run your tool successfully but it said there were no attachments when there are.  Khaled Hassen​ mentioned above that with the recent ArcGIS Online update there is another token required to get the json that contains the data.  I'm not sure how to go about submitting that token to get at the json, I'm a newbie at the REST API and web requests.

Any other thoughts?

Jeff Ward
Summit County, Utah
0 Kudos
KhaledHassen
Esri Contributor

What I meant in my previous message is that all what you need to do is to append the token to the url returned from create replica. So if the server returns url=http://services.arcgis.com/mytenent/../myreplica.json, you need to add &token=mytoken to the json url you are downloading. The token can be the same token you used in creating the replica if it is not expired. Otherwise you will need to acquire a new token based on your user identity.

Thanks

Khaled

JeffWard
Occasional Contributor III

Sorry Khaled Hassen​I should have read your reply more thoroughly.  I tacked the token on the end of my url using:

replicaUrl = crJson['URL'] + '&token=' + token
urllib.urlretrieve(replicaUrl, 'myLayer.json')

And again I get html code in myLayer.json, but this time it looks like this:

Am I not adding the token correctly?

Jeff Ward
Summit County, Utah
0 Kudos
KhaledHassen
Esri Contributor

Hi Jeff,

Not sure what the url looks like but if the token is the first parameter in the url which I think it is, you will have to use:

?token={your-token}. Pl. not the use of ? instead of &.

Khaled