J'ai rencontré de nombreux utilisateurs qui ont demandé la possibilité de télécharger les services de fonctionnalités hébergés ArcGIS Online et les services de fonctionnalités/cartes ArcGIS Server. Voir l'outil ci-joint pour faire exactement cela. L'outil vous permettra également de télécharger les pièces jointes des services de fonctionnalités hébergés ArcGIS Online. J'espère que les utilisateurs trouveront cela utile et pratique.<\/P>
<\/P>
Mise à jour 10\/08\/17:<\/STRONG> L'outil téléchargera désormais les domaines de valeurs codées dans la géodatabase. De plus, consolidation des versions 2x et 3x en un seul outil. L'outil peut être exécuté depuis ArcGIS Pro ou ArcGIS Desktop.<\/P>Mise à jour 13\/11\/17: <\/STRONG>L'outil prend désormais en charge l'utilisation d'une clause where pour interroger le service.<\/P>Mise à jour 14\/03\/17:<\/STRONG> Mise à jour vers un script unique pour ArcMap\/Pro utilisant le module requests<\/P>Mise à jour 17\/01\/19:<\/STRONG> Merci à Adam Eversole<\/A> d'avoir signalé que la fonctionnalité de cet outil peut désormais être prise en charge par l'outil Feature Class to Feature Class<\/A> d'ArcGIS Pro :<\/P><\/P><\/BODY><\/HTML>
<\/P><\/BODY><\/HTML>
John Broecher even though you receive an error when trying to download a related table, you should still see all of the field populated correctly. Here are the attributes from the table with an index of 3:
Here is the downloaded result (the tool did error):
You'll see that the field "GlobalID_Str" matches that of the 'parentglobalid' field. You can use the GlobalID_Str field to recreate the relationship class.
Just sent the request to add you to the group with the feature class. There are several teams deployed currently updating this layer offline via collectors. I need to get a backup of this data downloaded right away. Looking forward to hearing from you.
Thanks in advance
troydawson I just made a minor change to the tool. Download this new version and it should work.
Greetings Jake,
Amazing stuff there...That worked; however I seem to be having another issue. For every feature there are 2 attachments but based on the file size, only one attachment is not being downloaded properly (see attached). I would need to have both attachments downloaded. Please advise.
troydawson are you able to view the attachment in ArcMap when you identify the feature?
yes, both attachments are listed in Arcmap; however only one can be viewed. The other attachment gives an error when you try to open (see below).
Troy Dawson not sure what is going on with this service. I was able to reproduce your issue. I did not receive an error, but the attachment simply states it could not be displayed. I've shared a service (PropDamage) to the Service Download group for you to test. This service should work successfully.
As a workaround you can always export the feature service to a File Geodatabase. This will be stored in your Organization that you can then download.
Hi Jake,
I am not sure if your tool will help me with the following service, could you, please, assist me in obtaining a local dataset of the service?
Data Type: ArcGIS Map ServiceConnection: InternetServer: http://www.geoservices.isprambiente.it/arcgis/servicesName: Alluvioni/Aree_pericolosita_idraulicaOperations Allowed: Map (Display), Query (Identify), Data (Find)Map Service Type: Not Cached
I might be doing something wrong.
Best,
Gijs
Gijs van den Dool looks like you are missing 'rest' in the Server URL. Try each URL for each layer in this service:
http://www.geoservices.isprambiente.it/arcgis/rest/services/Alluvioni/Aree_pericolosita_idraulica/MapServer/0 http://www.geoservices.isprambiente.it/arcgis/rest/services/Alluvioni/Aree_pericolosita_idraulica/MapServer/1 http://www.geoservices.isprambiente.it/arcgis/rest/services/Alluvioni/Aree_pericolosita_idraulica/MapServer/2<SPAN class="line-numbers-rows"><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN></SPAN>
Thanks Jake,
That worked partial, the tool seems to stop after a while. The first layer hangs on: Copying features with ObjectIDs from 20800.0 to 20850.0
(I changed the records returned to 50 because that timed out earlier)
Any idea why that is happening?
If I display the polygon layer from my gdb I see that a tile (large area) is missing... could it be that the service timed out and that I have to restart (modifying your code) from the last ID? I think I noticed this before, but am wondering why this is happening. Your insight will be helpful.
I added two parameters to your code:
intCountIt = 50.0intStartRun = 749000
and changed lines [83-84] to
iteration = intStartRun #int(data['objectIds'][-1])minOID = intStartRun #int(data['objectIds'][0]) - 1
and line 94 to >> if count < intCountIt:
and lines [128-129]
newIteration = (math.ceil(iteration/intCountIt) * intCountIt) x = minOID + intCountIt
But would not like to run subsets of data blocks to get the full layer... too many things can go wrong.
Gijs1973 I can't say for sure what is going on with the service. I received the same error as you did. It seems there is an issue with a record(s) that have OBJECTIDs between 20,000 and 21,000. When trying to query the REST service using a where query of OBJECTID > 20000 AND OBJECTID <= 21000 it fails. I cannot say for sure why this occurs.
I will try tomorrow again (based in Paris, France) to get the missing
records. Unfortunately, the data provider is not responding to emails. I
keep you posted,
Does anybody know why I would be getting this error?
Traceback (most recent call last): File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 1254, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 1107, in request self._send_request(method, url, body, headers) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 1152, in _send_request self.endheaders(body) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 1103, in endheaders self._send_output(message_body) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 934, in _send_output self.send(msg) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 877, in send self.connect() File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 1261, in connect server_hostname=server_hostname) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\ssl.py", line 385, in wrap_socket _context=self) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\ssl.py", line 760, in __init__ self.do_handshake() File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\ssl.py", line 996, in do_handshake self._sslobj.do_handshake() File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\ssl.py", line 641, in do_handshake self._sslobj.do_handshake()ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:720)
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "C:\Users\pohlinger\Desktop\WWI Download\Download Service_New\Download Service.py", line 16, in <module> arcpy.GetParameterAsText(7), arcpy.GetParameterAsText(8), arcpy.GetParameterAsText(9)) File "C:\Users\pohlinger\Desktop\WWI Download\Download Service_New\DownloadServicewithAttachments_3x.py", line 225, in downloadservice3x response = urllib.request.urlopen(req) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 163, in urlopen return opener.open(url, data, timeout) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 466, in open response = self._open(req, data) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 484, in _open '_open', req) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 444, in _call_chain result = func(*args) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 1297, in https_open context=self._context, check_hostname=self._check_hostname) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 1256, in do_open raise URLError(err)urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:720)>Failed to execute (DownloadService).
I should have also noted, this is the feature service I am trying to download: https://gis.co.brown.wi.us:6443/arcgis/rest/services/ParcelPolygons/MapServer/1
I see there a SSL: CERTIFICATE_VERIFY_FAILED error appears a few times. Not sure if this is the culprit, but it could be. What happens if you try to download it using HTTP instead of HTTPS?
Philip Ohlinger not sure why you are receiving a certificate error, when it's a valid certificate. I think it's the port (6443) in the URL that is causing this to fail. I figured out how to fix this for the ArcGIS Desktop version of this tool. You can comment out line 195 and add the following:
<SPAN class="keyword token">try</SPAN><SPAN class="punctuation token">:</SPAN> response <SPAN class="operator token">=</SPAN> urllib2<SPAN class="punctuation token">.</SPAN>urlopen<SPAN class="punctuation token">(</SPAN>req<SPAN class="punctuation token">)</SPAN> <SPAN class="keyword token">except</SPAN><SPAN class="punctuation token">:</SPAN> gcontext <SPAN class="operator token">=</SPAN> ssl<SPAN class="punctuation token">.</SPAN>SSLContext<SPAN class="punctuation token">(</SPAN>ssl<SPAN class="punctuation token">.</SPAN>PROTOCOL_TLSv1<SPAN class="punctuation token">)</SPAN> response <SPAN class="operator token">=</SPAN> urllib2<SPAN class="punctuation token">.</SPAN>urlopen<SPAN class="punctuation token">(</SPAN>req<SPAN class="punctuation token">,</SPAN> context<SPAN class="operator token">=</SPAN>gcontext<SPAN class="punctuation token">)</SPAN><SPAN class="line-numbers-rows"><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN></SPAN>
For ArcGIS Pro you will need to replace line 227 with the following:
ctx <SPAN class="operator token">=</SPAN> ssl<SPAN class="punctuation token">.</SPAN>create_default_context<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN> ctx<SPAN class="punctuation token">.</SPAN>check_hostname <SPAN class="operator token">=</SPAN> <SPAN class="token boolean">False</SPAN> ctx<SPAN class="punctuation token">.</SPAN>verify_mode <SPAN class="operator token">=</SPAN> ssl<SPAN class="punctuation token">.</SPAN>CERT_NONE response <SPAN class="operator token">=</SPAN> urllib<SPAN class="punctuation token">.</SPAN>request<SPAN class="punctuation token">.</SPAN>urlopen<SPAN class="punctuation token">(</SPAN>req<SPAN class="punctuation token">,</SPAN> context<SPAN class="operator token">=</SPAN>ctx<SPAN class="punctuation token">)</SPAN><SPAN class="line-numbers-rows"><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN></SPAN>
You will also need to import the ssl module at the top of either script. Ex:
<SPAN class="keyword token">import</SPAN> arcpy<SPAN class="punctuation token">,</SPAN> urllib<SPAN class="punctuation token">,</SPAN> urllib2<SPAN class="punctuation token">,</SPAN> json<SPAN class="punctuation token">,</SPAN> os<SPAN class="punctuation token">,</SPAN> math<SPAN class="punctuation token">,</SPAN> sys<SPAN class="punctuation token">,</SPAN> linecache<SPAN class="punctuation token">,</SPAN> ssl <SPAN class="line-numbers-rows"><SPAN></SPAN><SPAN></SPAN></SPAN>
Jake,
I changed your code slightly, setting the start at the failing object id and increments of 5 objects (it's slow) and again the program stopped at range of object ids. I have narrowed it down to a small range of ID's where the website returns:
{ "error": { "code": 500, "message": "Error performing query operation", "details": [] } }
(object id 16792)
http://www.geoservices.isprambiente.it/arcgis/rest/services/Alluvioni/Aree_pericolosita_idraulica/MapServer/1
Perhaps you could include a error handling for this exception; I don't know why there is an error in the query.
Does this tool work on any ArcGIS Online hosted feature service regardless of it's shared properties? I am having success but only on a feature service that is shared publicly. If I point the tool to something shared only inside the org or only within a group in the org it doesn't work. In all cases I am providing the appropriate ArcGIS Online credentials. I receive a similar error as folks above:
I am using a slightly older version of the tool (about 1 year) as I have not yet created a script tool from the latest downloadable version.
I am assuming the script tool is not already created as a part of the latest download correct? I see the toolbox, however it is empty. Thanks.
An update, I was partially successful running the tool however one of my service layers simply won't export while another nested layer in the same service does. Essentially layer ID #0 runs error free but nothing is exported while layer ID #1, again from the same service, exports as expected with no issues.
For layer ID #0 (the one that doesn't export) the text "...copying features with object id's _ through _ ..." is missing from messages:
Also concerning, I pointed the tool to a feature layer that I know has attachments the tool exported the layer fine but had a warning "no attachments present". It output the feature class itself with not attachment table.
That all being said, I would like to try the latest version of the tool but when I download it and view via Catalog the toolbox is empty. I am running 10.3. I don't see any mention of the toolbox version this is created in. Could there be an issue where running 10.3 can't see say a 10.5 toolbox? Thanks.
jschuckert_InterwestGIS, yes I believe the reason you cannot see the script within the toolbox is because this was created using ArcGIS Desktop 10.5.1. Are you able to upgrade to 10.5.1?
Thank you for the reply. I was finally able to get on a 10.5 machine and run the latest tool with no issues. I tried pointing the tool at several ArcGIS Online layers, some shared publicly some not and some having attachment data others not, in all cases it worked very well. Thank you for developing and sharing such a useful tool!
Steven Castongia it looks like you are using the 2x version for ArcGIS Desktop. You can copy the DownloadServicewithAttachments_2x.py to the same directory as your other script. You can then use the following statement in your current script:
<SPAN class="keyword token">import</SPAN> DownloadServicewithAttachments_2x DownloadServicewithAttachments_2x<SPAN class="punctuation token">.</SPAN>downloadservice2x<SPAN class="punctuation token">(</SPAN>arcpy<SPAN class="punctuation token">.</SPAN>GetParameterAsText<SPAN class="punctuation token">(</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>GetParameterAsText<SPAN class="punctuation token">(</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>GetParameterAsText<SPAN class="punctuation token">(</SPAN><SPAN class="number token">2</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>GetParameterAsText<SPAN class="punctuation token">(</SPAN><SPAN class="number token">3</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>GetParameterAsText<SPAN class="punctuation token">(</SPAN><SPAN class="number token">4</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>GetParameterAsText<SPAN class="punctuation token">(</SPAN><SPAN class="number token">5</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>GetParameterAsText<SPAN class="punctuation token">(</SPAN><SPAN class="number token">6</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>GetParameterAsText<SPAN class="punctuation token">(</SPAN><SPAN class="number token">7</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>GetParameterAsText<SPAN class="punctuation token">(</SPAN><SPAN class="number token">8</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>GetParameterAsText<SPAN class="punctuation token">(</SPAN><SPAN class="number token">9</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>GetParameterAsText<SPAN class="punctuation token">(</SPAN><SPAN class="number token">10</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="line-numbers-rows"><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN></SPAN>
You will replace all of the arcpy.GetParameterAsText variables. Ex:
<SPAN class="keyword token">import</SPAN> DownloadServicewithAttachments_2x DownloadServicewithAttachments_2x<SPAN class="punctuation token">.</SPAN>downloadservice2x<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'true'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">''</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token"><SPAN>'</SPAN><A class="jive-link-external-small" href="https://community.esri.com/external-link.jspa?url=https%3A%2F%2Fservices.arcgis.com%2F9N1857LB1QVyzq54%2Farcgis%2Frest%2Fservices%2FITSAsBuilts%2FFeatureServer%2F0" target="_blank">https://services.arcgis.com/9N1857LB1QVyzq54/arcgis/rest/services/ITSAsBuilts/FeatureServer/0</A><SPAN>'</SPAN></SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">''</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'true'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">''</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'scastongiaCity'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'password'</SPAN><SPAN class="punctuation token">,</SPAN> r<SPAN class="string token">'S:\Public\CityGIS\ArcSDE\connections\development\cdot_dev.sde\ITSAsBuiltsCamera'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">''</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">''</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="line-numbers-rows"><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN></SPAN>
What Jake said. I guess you would like to pass the services as a parameter,
be careful with the string formatting. I would make the tool a function,
which you call from your script.
Good luck, the tool is very adaptable, you will work it out.
Thanks, Jake! I have been trying to use your example and that fixed that error message. The problem I am having now is that it is not executing anything. It returns "exit code" but it didn't actually export the hosted feature service.
Steven
Steven,
By when do you have this fixed... I could have a look tomorrow (if you can
pass me your code)... I'm not at a desk right now.
G
Hi Gijs,
I have no set timeframe on getting this done but it is something I want to implement so that I can copy our hosted feature service data out of the cloud. Below is my script. The first print statement "made it this far" will display while running the script but the last print statement "cameras copied" will not, which tells me that the tool is not executing correctly. Thanks for your help!
import arcpyimport osimport urllib2import urllibimport jsonimport mathimport sysimport linecacheimport DownloadServicewithAttachments_2xarcpy.env.overwriteOutput = 1 # overwrite output is trueprint "made it this far"#ITSAsBuilts/CameraDownloadServicewithAttachments_2x.downloadservice2x("true", "", "https://services.arcgis.com/9N1857LB1QVyzq54/arcgis/rest/services/ITSAsBuilts/FeatureServer/0", "", "true", "", "AGOLUsername", "password", r'd:\temp\temp.gdb\ITSAsBuilts_Camera', "", "")print "cameras copied"
import arcpyimport osimport urllib2import urllibimport jsonimport mathimport sysimport linecacheimport DownloadServicewithAttachments_2x
arcpy.env.overwriteOutput = 1 # overwrite output is true
print "made it this far"#ITSAsBuilts/CameraDownloadServicewithAttachments_2x.downloadservice2x("true", "", "https://services.arcgis.com/9N1857LB1QVyzq54/arcgis/rest/services/ITSAsBuilts/FeatureServer/0", "", "true", "", "AGOLUsername", "password", r'd:\temp\temp.gdb\ITSAsBuilts_Camera', "", "")
print "cameras copied"
I will have a look tomorrow, if Jake doesn't beat me for a solution, have a
nice evening.
I tried your script and it executes until line 200 in the *_2x.py script; I placed a break on the exit points.
I don't have access to your service, but when I use this line:
DownloadServicewithAttachments_2x.downloadservice2x("true", "", "http://www.geoservices.isprambiente.it/arcgis/rest/services/Alluvioni/Aree_pericolosita_idraulica/MapServer/0", "", "true", "", "", "", r'C:\Default.gdb\test_script', "", "")
I get results.
Please check your access credentials; you could try first to run the tool from ArcGIS, linking the py script to the toolbox, to test your connection, and the geodatabase must exists, otherwise the tool can't write the features out.
Hope this helps,
Thank you for your help! I think you are correct, there is a problem during the credential verification that is preventing the data from being copied. That particular hosted feature layer is only shared with a group and only that group can edit it so that is probably the issue. I swapped it out with a hosted feature layer that is shared with the public and it ran correctly.
Thanks again for your help! I greatly appreciate it!
Steven Castongia, GISP
GIS Supervisor
scastongia@charlottenc.gov<mailto:scastongia@charlottenc.gov>
704-336-3816
Hi Steven,
You're welcome, it was a quick test, and your script helped me to run my
extraction outside of the ArcGIS main interface
Best regards,
Les membres connectés peuvent publier, suivre les mises à jour, et plus encore. Nouveau ici ? Inscrivez-vous gratuitement.
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.
