nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Télécharger le service de fonctionnalités ArcGIS Online ou le service de fonctionnalités/cartes ArcGIS Server

JakeSkinner

<\/HEAD>

J'ai rencontré de nombreux utilisateurs qui ont demandé la possibilité de télécharger les services de fonctionnalités hébergés ArcGIS Online et les services de fonctionnalités/cartes ArcGIS Server. Voir l'outil ci-joint pour faire exactement cela. L'outil vous permettra également de télécharger les pièces jointes des services de fonctionnalités hébergés ArcGIS Online. J'espère que les utilisateurs trouveront cela utile et pratique.<\/P>

<\/P>

Mise à jour 10\/08\/17:<\/STRONG> L'outil téléchargera désormais les domaines de valeurs codées dans la géodatabase. De plus, consolidation des versions 2x et 3x en un seul outil. L'outil peut être exécuté depuis ArcGIS Pro ou ArcGIS Desktop.<\/P>Mise à jour 13\/11\/17: <\/STRONG>L'outil prend désormais en charge l'utilisation d'une clause where pour interroger le service.<\/P>Mise à jour 14\/03\/17:<\/STRONG> Mise à jour vers un script unique pour ArcMap\/Pro utilisant le module requests<\/P>Mise à jour 17\/01\/19:<\/STRONG> Merci à Adam Eversole<\/A> d'avoir signalé que la fonctionnalité de cet outil peut désormais être prise en charge par l'outil Feature Class to Feature Class<\/A> d'ArcGIS Pro :<\/P>

<\/P><\/BODY><\/HTML>

Trouver plus de publications taguées avec

Réponses

TiffanySelvidge1

Thanks Jake for making this. It is a big help.

I have it working with what I need but I have encountered a snag in that I need to encrypt the username and password. I have tried storing the username and password in a separate file and just referencing the parameters in my code (which uses your code). The referencing works but it does not hide the actual value of the parameters. I have tried using base64 encoding but now arcgisonline does not recognize my credentials. Do you know how I can encrypt the password so that the actual value cannot be seen but arcgisonline can still recognize it?

JakeSkinner

graynic‌ I was unable to reproduce this. Below is how I had the tool setup:

Are you outputting the data to a geodatabase?

ISP_graynic

jskinner-esristaff Yes, I would like to output it to a geodatabse. The end goal is to have it exported as a spreadsheet for Excel on a regular basis.

JakeSkinner

graynic‌ can you provide a screen shot of how you have the tool setup?

ISP_graynic

Jake Skinner‌ Here is how I set it up. Were you able to get it to run? If so, how many records showed up in your layer after it was exported?

JakeSkinner

graynic‌ yes, it ran and returned 953 records. Are you receiving an error when you run the tool, or is it just not returning the correct amount of records?

ISP_graynic

jskinner-esristaff Now I am getting the error below

se3d/

JakeSkinner

@Nicholas_Gray check to make sure you don't have 'Downloading tabular data' checked. This error usually indicates you are trying to a load a table into a feature class.

ISP_graynic

jskinner-esristaff‌ I don't have that checked but am still getting that message. I'm running it with the same settings you sent me in the screenshot.

Thanks for helping!

JakeSkinner

Tiffany Selvidge‌ can you provide an example of using the base64 encoding. I'm not an expert with this, but I just tried generating an AGOL token using this and it worked.

<SPAN class="keyword token">import</SPAN> base64<SPAN class="punctuation token">,</SPAN> urllib<SPAN class="punctuation token">,</SPAN> urllib2<SPAN class="punctuation token">,</SPAN> json

username <SPAN class="operator token">=</SPAN> <SPAN class="string token">"jskinner_CountySandbox"</SPAN>
password <SPAN class="operator token">=</SPAN> <SPAN class="string token">'agol1234'</SPAN><SPAN class="punctuation token">.</SPAN>encode<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'base64'</SPAN><SPAN class="punctuation token">)</SPAN>

tokenURL <SPAN class="operator token">=</SPAN> <SPAN class="string token"><SPAN>'</SPAN><A class="jive-link-external-small" href="https://community.esri.com/external-link.jspa?url=https%3A%2F%2Fwww.arcgis.com%2Fsharing%2Frest%2FgenerateToken" target="_blank">https://www.arcgis.com/sharing/rest/generateToken</A><SPAN>'</SPAN></SPAN>
params <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">{</SPAN><SPAN class="string token">'f'</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="string token">'pjson'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'username'</SPAN><SPAN class="punctuation token">:</SPAN> username<SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'password'</SPAN><SPAN class="punctuation token">:</SPAN> password<SPAN class="punctuation token">.</SPAN>decode<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'base64'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'referer'</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="string token"><SPAN>'</SPAN><A class="jive-link-external-small" href="https://community.esri.com/external-link.jspa?url=http%3A%2F%2Fwww.arcgis.com" target="_blank">http://www.arcgis.com</A><SPAN>'</SPAN></SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'expiration'</SPAN><SPAN class="punctuation token">:</SPAN> str<SPAN class="punctuation token">(</SPAN><SPAN class="number token">21600</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">}</SPAN>
req <SPAN class="operator token">=</SPAN> urllib2<SPAN class="punctuation token">.</SPAN>Request<SPAN class="punctuation token">(</SPAN>tokenURL<SPAN class="punctuation token">,</SPAN> urllib<SPAN class="punctuation token">.</SPAN>urlencode<SPAN class="punctuation token">(</SPAN>params<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
response <SPAN class="operator token">=</SPAN> urllib2<SPAN class="punctuation token">.</SPAN>urlopen<SPAN class="punctuation token">(</SPAN>req<SPAN class="punctuation token">)</SPAN>
data <SPAN class="operator token">=</SPAN> json<SPAN class="punctuation token">.</SPAN>load<SPAN class="punctuation token">(</SPAN>response<SPAN class="punctuation token">)</SPAN>
token <SPAN class="operator token">=</SPAN> data<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'token'</SPAN><SPAN class="punctuation token">]</SPAN>
<SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN>token<SPAN class="punctuation token">)</SPAN><SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

TiffanySelvidge1

Jake,

Thanks for your response.

I was not including 'password.decode('base64')'. I am now able to download the service. I am wondering though that since decode will cause the password to be read, is the base64 encoding any use? Or is it just me when I run the code and print the line that is able to see the plain text? I have tried running Fiddler to see what others might see but the code will not run while Fiddler is open.

TiffanySelvidge1

I think I answered my own question regarding reading the password. All anyone has to do is grab the code and run the line to acquire the password. That leads to my next question which is whether any encoding encrypting method can securely work since the password has to be sent to arcgisonline and agol has to be able to read the password to login. The only solution I can think of is that agol would have to be the one decrypting the password otherwise I am be encrypting and decrypting in the same exact code. Since I do not think agol decrypting is at all an option, for now I will stick with the base64 coding as at least the plain text cannot be read without having to run the line.

Thanks again for writing the script, Jake. It is exactly what was needed. I am downloading 14 layers and running them through another script and updating them into Portal on a daily basis. Manually downloading each day and then running the script would not have been pleasant. Being able to automate the entire task is much much better.

ISP_graynic

jskinner-esristaff‌ I was able to get the tool to run in Pro instead of ArcMap. Thank you for all your help!

JaniceBaird

Jake,

When I run the tool in ArcMap it works fine. I used copy as python snippet to bring it into a new python window for creating a script file to run it from. When running from a python script, I get an error (Output name cannot contain dots). I did some digging around in DownloadServicewithAttachments_2x.py and found that if I removed the ".1" from "esriDataSourcesGDB.SdeWorkspaceFactory.1", it works fine.

I made this change because this code: ##desc = arcpy.Describe(os.path.dirname(wrkspc))
##print(desc.workspaceFactoryProgID) returns "esriDataSourcesGDB.SdeWorkspaceFactory".

Can you explain to me what the difference is between the workspaceFactoryProgID with the ".1" and the one without the ".1"? Is there something in my code that runs the DownloadService that needs to change so that it works with the ".1"?

Thanks,

Janice.

TiffanySelvidge1

Jake,

I have been experimenting to modify the script in order to download only data edited within the past 2 days. I have python script that checks for all data within the past 2 days but because all data is downloaded at the start regardless of time, a bottleneck occurs with the initial download taking 2-4 minutes. This bottleneck is preventing the scripts from running during the day as the total time for all scripts is over an hour (16 layers).

I have tried:

x = iteration
y = minOID

where = 'EditDate+>%3D+CURRENT_TIMESTAMP+-1' + OID + '>' + str(y) + 'AND ' + OID + '<=' + str(x)

and

x = iteration
y = minOID

where = 'EditDate+like+%27%252017-10-02%25%27&' + OID + '>' + str(y) + 'AND ' + OID + '<=' + str(x)

but the modifications are ignored and all data is downloaded from the past month. I am placing the modifications within the 'if count < 1000:' and 'else:
newIteration = (math.ceil(iteration/1000.0) * 1000)' sections.

Do you happen to know of a method to achieve what I am trying to do?

pmckinney

jskinner-esristaff‌, can explain or provide a link that describes the logic in the PrintException(error) function? It appears it gets the name of the python script being run. I'm updating a custom tool that is powered by 4 scripts (modules), and want to include which script the error is originating from.

I also typically include an EnvironmentError and Exception error except statements. Does this function handle both situations, or would I need to refactor it to be able to handle both exception types?

Thanks,

Patrick

JakeSkinner

tselvidge‌ I updated the tool to now include a where clause parameter that will allow you to query the service.

JakeSkinner

Patrick McKinney‌ I forgot where I snagged this portion of code from. It should print the file and line of code that the error is originated from. Kind of losing me with the Environment and Exception error statements question. I typically use this function like the following:

<SPAN class="keyword token">try</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp; perform some action
<SPAN class="keyword token">except</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp; PrintException<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"Error performing action"</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

PankajJamwal1

Hi,

Does it download the subtypes too?

Thanks

JakeSkinner

This tool currently does not download subtypes.

TiffanySelvidge1

Thank you very much! I have downloaded and started using it. So far it is saving me an average of 3min for each layer; instead of taking ~1 1/2 hours for 16 areas it is now only taking ~16min. With the change in time I can now run the code for all the areas during the day instead of just nightly updates.

JohnBroecher

@Jake Skinner

Thanks for the script!

I noticed the globalid and OBJECTID are changed in the downloaded copy. How do I make this work when I want to download data with related tables that use the globalid field as the key?

JakeSkinner

John.Broecher‌ if you are working with a hosted feature service, the easiest approach will be to enable attachments and add an attachment to the feature. When downloading the service, choose the option to Get Attachments. This will create a new field called "GlobalID_Str" in the feature class that contains the original GlobalIDs.

JohnBroecher

Tried enabling attachments on the parent feature service. It gives me a permission denied error now. It was successfully downloading just the parent table when I unchecked get attachments.

Executing: DownloadService true false http://services.arcgis.com/jWPBXspaQsJStWX8/arcgis/rest/services/service_0ba0b35ba4b6436cbe2e9d36eaccc71a/FeatureServer/0 # true false John.Broecher ***** "Database Connections\espr1sql14 prod04 GFSurveys GFSurveys_Admin.sde\GFSurveys.GFSURVEYS_ADMIN.testWeedSpray" true #
Start Time: Wed Nov 29 07:54:06 2017
Running script DownloadService...
Generating Token
Copying features with ObjectIDs from 10 to 659
Retrieving Attachments
Failed script DownloadService...
Traceback (most recent call last):
File "M:\GFP\Wildlife\Region3\Broecher\ArcGIS Tools\Download Service 11-27-2017\Download Service.py", line 22, in <module>
arcpy.GetParameterAsText(8), arcpy.GetParameterAsText(9), arcpy.GetParameterAsText(10))
File "M:\GFP\Wildlife\Region3\Broecher\ArcGIS Tools\Download Service 11-27-2017\DownloadServicewithAttachments_2x.py", line 534, in downloadservice2x
urllib.urlretrieve(replicaUrl, cwd + os.sep + 'myLayer.json')
File "C:\Python27\ArcGIS10.5\Lib\urllib.py", line 98, in urlretrieve
return opener.retrieve(url, filename, reporthook, data)
File "C:\Python27\ArcGIS10.5\Lib\urllib.py", line 249, in retrieve
tfp = open(filename, 'wb')
IOError: [Errno 13] Permission denied: '\\myLayer.json'
Failed to execute (DownloadService).
Failed at Wed Nov 29 07:54:32 2017 (Elapsed Time: 25.83 seconds)

JakeSkinner

John.Broecher‌ can you share the service to a Group and invite my user account (jskinner_CountySandbox) to the Group. I can take a look at the service. Also, can you check to make sure 'Sync' privileges have been enabled on the service?

JohnBroecher

I can't seem to share that service with you since its not in my content. I shared another one that gets the same error. This one has 3 related tables, and one of the related tables has attachments. I am trying to figure out how to download dozens of feature services that are made in Survey123 and published from desktop for collector. A lot have related tables and attachments. I have been using a script that uses the downloadSurvey123 library and it works OK but it can only see services published from S123, and sometimes it can't see a S123 because for an unknown reason the sharing in AGO doesn't carry over to S123.

JohnBroecher

I think I was assuming a related table to be treated the same as an attachment. It doesn't seem to work that way. I can get each related table to download only if I use its index in the URL and select tabular data. Still there is a global ID problem. I enabled attachments but you are saying I actually have to have an attachment in every table to make this work? How would I do that without trashing the surveys? Is there an entirely different approach that you would recommend for me? The related table with index 3 actually does have an attachment but it gives the permission denied error still.

Executing: DownloadService true false http://services.arcgis.com/jWPBXspaQsJStWX8/arcgis/rest/services/service_9e43947ea8c54e23aec9c3295a7f37a6/FeatureServer/3 # false true John.Broecher ***** "Database Connections\espr1sql14 prod04 GFSurveys GFSurveys_Admin.sde\GFSurveys.GFSURVEYS_ADMIN.testMOCREEL3" true #
Start Time: Thu Nov 30 08:29:25 2017
Running script DownloadService...
Generating Token
Copying features with ObjectIDs from 0 to 2
Retrieving geodatabase domains
Adding domain cvd_Species to geodatabase
The value being added to the CodedValueDomain already exists
Applying domain cvd_Species to field Species
Retrieving geodatabase domains
Adding domain cvd_TagType to geodatabase
The value being added to the CodedValueDomain already exists
Applying domain cvd_TagType to field TagType
Retrieving geodatabase domains
Adding domain cvd_TagColor to geodatabase
The value being added to the CodedValueDomain already exists
Applying domain cvd_TagColor to field TagColor
Retrieving Attachments
Failed script DownloadService...
Traceback (most recent call last):
File "M:\GFP\Wildlife\Region3\Broecher\ArcGIS Tools\Download Service 11-27-2017\Download Service.py", line 22, in <module>
arcpy.GetParameterAsText(8), arcpy.GetParameterAsText(9), arcpy.GetParameterAsText(10))
File "M:\GFP\Wildlife\Region3\Broecher\ArcGIS Tools\Download Service 11-27-2017\DownloadServicewithAttachments_2x.py", line 534, in downloadservice2x
urllib.urlretrieve(replicaUrl, cwd + os.sep + 'myLayer.json')
File "C:\Python27\ArcGIS10.5\Lib\urllib.py", line 98, in urlretrieve
return opener.retrieve(url, filename, reporthook, data)
File "C:\Python27\ArcGIS10.5\Lib\urllib.py", line 249, in retrieve
tfp = open(filename, 'wb')
IOError: [Errno 13] Permission denied: '\\myLayer.json'
Failed to execute (DownloadService).
Failed at Thu Nov 30 08:32:12 2017 (Elapsed Time: 2 minutes 46 seconds)

dougbrowning

My post is buried back from Jul 3 but have you tried this method? It has worked perfectly for me since July.

This way is equivalent to doing a Export Data from the Item page in AGO (the one with all the settings and such).

**So you may want to do the Export manually first to see if this works on your data.

First you need the Item ID of the HFS - get this from the top of the Item page in AGO. You want the last part after id=

URL looks like http://xyz.maps.arcgis.com/home/item.html?id=46954aa9bfb24c53bba320f363af8550

Then I slightly modified the ArcRest sample script called create_replica_portal_item.py (in other words some of this is not my code)

ArcREST/create_replica_portal_item.py at master · Esri/ArcREST · GitHub

LOOK for the caps on what to change Note the post messed up all the indents - i tried to fix but may be off

import arcrest
from arcresthelper import securityhandlerhelper
from arcresthelper import common

import os, time

def trace():

"""
trace finds the line, the filename
and error message and returns it
to the user
"""
import traceback, inspect, sys
tb = sys.exc_info()[2]
tbinfo = traceback.format_tb(tb)[0]
filename = inspect.getfile(inspect.currentframe())
# script name + line number
line = tbinfo.split(", ")[1]
# Get Python syntax error
#
synerror = traceback.format_exc().splitlines()[-1]
return line, filename, synerror

if __name__ == "__main__":

proxy_port = None
proxy_url = None

securityinfo = {}
securityinfo['security_type'] = 'Portal'#LDAP, NTLM, OAuth, Portal, PKI
securityinfo['username'] = "PUT USERNAME HERE" #<UserName>
securityinfo['password'] = "PUT PASSWORD HERE" #<Password>
securityinfo['org_url'] = "http://www.arcgis.com"
securityinfo['proxy_url'] = proxy_url
securityinfo['proxy_port'] = proxy_port
securityinfo['referer_url'] = None
securityinfo['token_url'] = None
securityinfo['certificatefile'] = None
securityinfo['keyfile'] = None
securityinfo['client_id'] = None
securityinfo['secret_id'] = None

# where to save the zip files
savePath = r"PUT OUTPUT DIR HERE"

itemId = "PUT ITEM ID HERE"
#sample itemId = "5a9ecdf30f53466bad14f5dbbedb1014"

try:

shh = securityhandlerhelper.securityhandlerhelper(securityinfo=securityinfo)
if shh.valid == False:
print shh.message
else:
admin = arcrest.manageorg.Administration(securityHandler=shh.securityhandler)

item = admin.content.getItem(itemId)
user = admin.content.users.user(username=item.owner)

# use to set jsut some layers. NOT using this
## exportParameters = {"layers":[
## {
## "id": 0,
## "where": "OBJECTID = 1"
## }]
## }

# this version waits for the download so no pause needed
res = user.exportItem(title=HFSname,
itemId=itemId,
exportFormat="File Geodatabase",
exportParameters=None,
wait=True)

exportItemId = res.id
exportItem = admin.content.getItem(exportItemId)
itemDataPath = exportItem.itemData(f=None, savePath=savePath)
# Delete from AGO list to remove clutter
exportItem.userItem.deleteItem()

# rename file so it makes sense
#taken out for ease

except (common.ArcRestHelperError),e:

print "error in function: %s" % e[0]['function']
print "error on line: %s" % e[0]['line']
print "error in file name: %s" % e[0]['filename']
print "with error message: %s" % e[0]['synerror']
if 'arcpyError' in e[0]:
print "with arcpy message: %s" % e[0]['arcpyError']

except:
line, filename, synerror = trace()
print "error on line: %s" % line
print "error in file name: %s" % filename
print "with error message: %s" % synerror

Let me know how it goes.

TCPlanningDepartment

Greetings Jake,
I have been trying to download my hosted feature service and continually get the attached error. The features are downloading but the attachments are not. Please advise.

JakeSkinner

troydawson‌ can you share the service you are trying to download to a Group and invite my user account (jskinner_CountySandbox)?

Rejoindre la conversation

Les membres connectés peuvent publier, suivre les mises à jour, et plus encore. Nouveau ici ? Inscrivez-vous gratuitement.

Liens rapides

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.