topic Re: Python script to replicate hosted feature service on ArcGIS Online including attachments. in Python Questions

Python script to replicate hosted feature service on ArcGIS Online including attachments.

JeffWard — Sat, 11 Dec 2021 06:35:47 GMT

I am trying to get this script to work for an application for one of our departments. If you look at the comment by stuartalexandersanders in the GitHub link - that is the same error I am getting. I'm hoping Alex Sanders is the same person and has fixed it?

gtUrl = 'https://www.arcgis.com/sharing/rest/generateToken'
gtValues = {'username' : 'MyUsername',
  'password' : 'MyPassword',
  'referer' : 'http://www.arcgis.com',
  'f' : 'json' }
gtData = urllib.urlencode(gtValues)
gtRequest = urllib2.Request(gtUrl, gtData)
gtResponse = urllib2.urlopen(gtRequest)
gtJson = json.load(gtResponse)
token = gtJson['token']

### Create Replica ###
### Update service url HERE ###
crUrl = 'http://services2.arcgis.com/MyServices/arcgis/rest/services/MyServicName/FeatureServer/createReplica'

That code works and I am able to get a token for the feature service.

This next block is where I am having my trouble -

crValues = {'f' : 'json',
  'layers' : '0',
  'returnAttachments' : 'true',
  'token' : token }
crData = urllib.urlencode(crValues)
crRequest = urllib2.Request(crUrl, crData)
crResponse = urllib2.urlopen(crRequest)
crJson = json.load(crResponse)
replicaUrl = crJson['URL']
urllib.urlretrieve(replicaUrl, 'myLayer.json')

The error I get is from the urlretrieve call. myLayer.json is not a json file but an html file that says "Token Required". The error text in python is:

Any ideas why this would happen? Apparently it was working for Alex before the update to ArcGIS Online earlier this month.

Thanks,

Jeff

Re: Python script to replicate hosted feature service on ArcGIS Online including attachments.

JeffWard — Mon, 28 Mar 2016 22:16:40 GMT

Fred Spataro, got any ideas? You asked me in my earlier question if I could just use the Export Item method, but I need the attachments and didn't see any parameters to include attachments with Export Item.

Re: Python script to replicate hosted feature service on ArcGIS Online including attachments.

FredSpataro — Mon, 28 Mar 2016 23:22:20 GMT

Jeff,

The export item method 'should' bring back ALL the data from the entire feature service. I haven't tried it with attachments but did have a service with multiple layers and tables and got back a file geodatabase will all the items... use exportFormat as file geodatabase and leave the layers parameter empty.

Re: Python script to replicate hosted feature service on ArcGIS Online including attachments.

KhaledHassen — Tue, 29 Mar 2016 16:14:52 GMT

In the last release we have in online, download the json file/sqlite from create replica API will require a token. This is to prevent unauthenticated users from download the create replica output by other users. Pl. when download the output from create or sync replica append a &token={your-token} to the json file url. Pl. let us know if this does not work.

Thanks

Khaled

Re: Python script to replicate hosted feature service on ArcGIS Online including attachments.

JeffWard — Tue, 29 Mar 2016 19:25:30 GMT

I was actually thinking this as I was driving home last night. So I need to get another token for the url to the json file (replicaUrl in my code)?

Re: Python script to replicate hosted feature service on ArcGIS Online including attachments.

JakeSkinner — Tue, 29 Mar 2016 19:32:57 GMT

You may want to try the following tool:

Download ArcGIS Online Feature Service or ArcGIS Server Feature/Map Service

It's setup as a GP tool so it's easier to configure. Let me know if you run into any errors. I'm seeing some issues trying to download older services that contain attachments.

Re: Python script to replicate hosted feature service on ArcGIS Online including attachments.

JeffWard — Tue, 29 Mar 2016 22:33:22 GMT

Jake,

Thanks for the input. It looks like you have done what I am trying to do, but I think the code you use for the attachments is the same code I am trying to use in my script (line 286 in your code is where I hit my problem). So it is running into the same problem I am having. I was able to run your tool successfully but it said there were no attachments when there are. Khaled Hassen mentioned above that with the recent ArcGIS Online update there is another token required to get the json that contains the data. I'm not sure how to go about submitting that token to get at the json, I'm a newbie at the REST API and web requests.

Any other thoughts?

Re: Python script to replicate hosted feature service on ArcGIS Online including attachments.

KhaledHassen — Wed, 30 Mar 2016 03:38:10 GMT

What I meant in my previous message is that all what you need to do is to append the token to the url returned from create replica. So if the server returns url=http://services.arcgis.com/mytenent/../myreplica.json, you need to add &token=mytoken to the json url you are downloading. The token can be the same token you used in creating the replica if it is not expired. Otherwise you will need to acquire a new token based on your user identity.

Thanks

Khaled

Re: Python script to replicate hosted feature service on ArcGIS Online including attachments.

JeffWard — Sat, 11 Dec 2021 06:35:50 GMT

Sorry Khaled HassenI should have read your reply more thoroughly. I tacked the token on the end of my url using:

replicaUrl = crJson['URL'] + '&token=' + token
urllib.urlretrieve(replicaUrl, 'myLayer.json')

And again I get html code in myLayer.json, but this time it looks like this:

Am I not adding the token correctly?

Re: Python script to replicate hosted feature service on ArcGIS Online including attachments.

KhaledHassen — Thu, 31 Mar 2016 05:18:30 GMT

Hi Jeff,

Not sure what the url looks like but if the token is the first parameter in the url which I think it is, you will have to use:

?token={your-token}. Pl. not the use of ? instead of &.

Khaled

Re: Python script to replicate hosted feature service on ArcGIS Online including attachments.

AlexSanders — Fri, 01 Apr 2016 15:40:21 GMT

ESRI has pointed me to this script that pulls down a fgd from a AGO hosted service. It seems to be working for me developer-support/createReplicaAndDownload.py at master · Esri/developer-support · GitHub