Latest Contributions by RichardFisher

Thank you Josh and for the Arcade tip, I agree this is the most common solution as is having a table with the correct field types so this is not an issue and one most likely to be the fix for this. I find it interesting that the rest query does work for this and still wonder if there is a way even if not for this solution to allow widgets to consume such a query and resulting json as the data source rather than the table directly as the source, kind of a way to pre-process the data the widget would see.  ... View more

Sure - There is more there because it exports and puts away several file geodatabase files with the download that creates folders elsewhere but the general flow for one of them is here. It was working for over a year and now, fails on one which may be due to a change my customer made in layer assignment still trying to sort that out and the changes I had to make to arcgis.gis for some reason..? ArcPro upgrades not sure.. but it's running on Python 3.6.10 64bit and works for one Hosted Feature... hope this helps with some context. #import arcgis.gis from arcgis.gis import GIS as g # Snipped from calling python file to file doing the detail export work start = timer() print ("Export Hosted Feature") from AGOL_aaaaaaaaa import export_xxxxxxxx_from_agol export_xxxxxxxx_from_agol() end = timer() print ("aaaaaaaaaaaaa export in AGOL Done " + str((end - start)/60) ) # Time in minutes # ---- python general flow of export on one of the hosted features called by the 'main' python snippet above import time import datetime import os import zipfile #import arcgis.gis from arcgis.gis import GIS as g from zipfile import ZipFile from datetime import datetime import pathlib from timeit import default_timer as timer from datetime import datetime, date from dateutil.relativedelta import relativedelta import csv, sys # --------------------------------- vars ----------------------------------------------- # Where the GDB will be extracted to after extraction and then download of that extraction outputFolder=r"\\xxxxxxxxx\yyyyyyyyyy\zzzzzz" # Path to credentials file for Prod credentialsFile=r"C:\\aaa\\bbb\\cccc\\yourcreds.csv" # Gather site url and login credentials cred = '' # log into AGOL object gis = '' # name of AGOL temporary GDB saved from Hosted feature class GDBname = "xxxxxxxxx_fgdb" # item_id is the service_ID of the hosted Prod CRUB feature layer to download item_id='aaaaaaaaaaaaaaaaaaaaaaaaaaa' #Parameters for editor tracking try and preserve pub_params = {"editorTrackingInfo" : {"enableEditorTracking":True, "preserveEditUsersAndTimestamps":True}} # --------------------------- called functions ------------------------------ # open(file, mode, buffering, encoding, errors, newline, closefd, opener) def read_credentials(): with open(credentialsFile, newline='') as f: reader = csv.reader(f) for row in reader: print("row[0] :" + str(row[0])) return row def export_xxxxxxxx_from_agol(): print ("AGOL_ADA_Export_CURB_Prod.py mndot.maps.arcgis.com") # Gather site url and login credentials cred = read_credentials() # log into AGOL object gis = g(cred[0], cred[1], cred[2]) print ("GDBname : " + GDBname ) # search AGOL for the item to export via the item id AGOLitem = gis.content.get(item_id) print ("Exporting Hosted ccccccccc Feature Layer to temp AGOL file..." + str(GDBname)) AGOLitem.export(GDBname,'File Geodatabase', parameters=pub_params, wait=True) #add 20 seconds delay to allow export to complete time.sleep(60) print ("Export Hosted xxxxxxxxx Feature Layer to temp AGOL file Done ..." + str(GDBname)) return GDBname # --------------------------- called functions ------------------------------ def search_for_agol_fgdb(monthFolder): """This function searches for the name of the AGOL file geodatabase exported from the hosted feature class""" try: print ("Try targeted search for :" + str(GDBname) + ".zip") # Gather site url and login credentials cred = read_credentials() # log into AGOL object gis = g(cred[0], cred[1], cred[2]) print ("GDBname : " + GDBname ) search_fgb = gis.content.search(query=(str(GDBname) + ".zip"), item_type = 'File Geodatabase',sort_field="created" ,sort_order="desc") fgb_item_id = search_fgb[0].id fgb = gis.content.get(fgb_item_id) print ("Finished search. Starting download to folder..:" + monthFolder) fgb.download(save_path=monthFolder) #download file gdb from ArcGIS Online to your computer return fgb print ("Finished targeted search") except Exception as e: print("type error: " + str(e)) print ("Was not able to find exact match to download :" + (str(GDBname) + ".zip") ) return "Error : " + str(e) + " : No data found" finally: print ("Finished search.") # backup one month and year if needed for folder name to store downloads def create_month_directory(): mydate = datetime.now() backdate = mydate - relativedelta(months=1) # create the output folder pathlib.Path(outputFolder + '\\' + str(backdate.strftime("%b")) + '_' + str(backdate.strftime("%Y"))).mkdir(exist_ok=True) monthFolder = (outputFolder + '\\' + str(backdate.strftime("%b")) + '_' + str(backdate.strftime("%Y"))) print("monthFolder : " + str(monthFolder)) return monthFolder ... View more

I'm on 2.5.2 pro and trying the upgrade  I am not using the loop at all I just have one export line with params and a wait statement following .     AGOLitem.export(GDBname,'File Geodatabase', parameters=pub_params, wait=True) I have one large fgdb download that works and one that does not getting errors about indices which I have not found an answer to yet. File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\__init__.py", line 7340, in export     if status['status'] == 'failed': TypeError: string indices must be integers The other strange thing is that where previous I could import arcgis.gis that no longer worked and had to switch to the format from arcgis.gis import GIS as g to make it work. I am currently upgrading to see if it helps or hurts. I will also try to rebuild the indexes to see if that helps. ... View more

As you say it has been a while and that is no longer an issue, my recollection and notes indicated that we had ad DBA add GLOBALID and re-index and changed or collapsed the multipoint to a single point and things improved. I believe the map currently is coming out of AGOL since then so not exactly the same any more. Hope that helps. ... View more

Hello Xabier - The compressed file is 1,052,933 KB. So there were two issues as you can see, the one updating the records with the script login is the most painful as it alters the content and is not just a harmless download of data as a back up. Currently if I don't use the look it works well except the parameter to not update the 'creator' and the 'edited by' fields are ignored and updated by the login running the script. If I include the loop it also finishes with a complete download but fails on the bool issue testing the zip file which I believe but can't prove is back in the arcpro py File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\connection.py", line 816, in get    params['f'] = 'json' TypeError: 'bool' object does not support item assignment ... View more

Removing the loop seems to solve the issue and keeps the correct data downloaded. There is a new issue that I will enter in a new thread - The manual or via python download of AGOL Hosted feature to a fgdb changes the creator and edited by fields to be the account doing the download - I did take some input from a post regarding params for publishing that would preserver tracking data and when I try it, the export still runs but does not preserve the data. ... View more

I did change the code to load the bool check of is a proper zip file and it ran through with the result finishing and creating the valid zip file of the download but with an error still Reason for exception 'bool' object does not support item assignment I took this to mean it was still failing in the arcpro connection.py on line 816 trying to check for a Boolean json. I have not tried this yet but possibly I am missing a parameter of 'f' and setting it to fgd ?? Hard to find the documentation on this as it relates to fgb.dowload. As of now it just takes in the file name, but in the manual web interface you do have to choose a shapefile or fgdb or etc.... fgb.download(save_path=outputFolder) If I find an answer I will update this memo....  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\connection.py", line 816, in get    params['f'] = 'json' TypeError: 'bool' object does not support item assignment ... View more

While borrowing from many of these comments and script ideas you all have posted, I have an error in python right after the download from AGOL to the local zip file. Where it seems the test for valid zip is not done fast enough to prevent one extra loop from trying the download, or it is some kind of timeout as this runs for an hour. This is the part that finishes creating the correct zip file download but errors off right at the end in the ArcGIS Pro connection.py and request.py.  I believe that I can put in a try catch and work around this, but would prefer to understand the issue and have it work with out that tactic.  '''while statement runs until a valid zipped file is created''' zipfullpath=os.path.join(outputFolder,GDBname+".zip") while zipfile.is_zipfile(zipfullpath)==False: print("looping: " + str(datetime.now())) fgb.download(save_path=outputFolder) zf = ZipFile(os.path.join(outputFolder,GDBname+ ".zip")) looping: 2019-07-29 13:26:10.354947 Traceback (most recent call last): File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\connection.py", line 849, in get resp = opener.open(url) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 532, in open response = meth(req, response) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 642, in http_response 'http', request, response, code, msg, hdrs) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 570, in error return self._call_chain(*args) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 504, in _call_chain result = func(*args) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 650, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) error.HTTPError: HTTP Error 498: 498 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "c:\Users\xxx\.vscode\extensions\ms-python.python-2019.6.24221\pythonFiles\ptvsd_launcher.py", line 43, in <module> main(ptvsdArgs) File "c:\Users\xxx\.vscode\extensions\ms-python.python-2019.6.24221\pythonFiles\lib\python\ptvsd\__main__.py", line 434, in main run() File "c:\Users\xxx\.vscode\extensions\ms-python.python-2019.6.24221\pythonFiles\lib\python\ptvsd\__main__.py", line 312, in run_file run_path(target, run_name='__main__') File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "c:\wip\sandbox\agol_py\ArcGIS API AGOL Download Feature Layer as GDB_Python3.py", line 45, in <module> download(save_path=outputFolder) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\__init__.py", line 7032, in download out_folder=save_path, try_json=False, force_bytes=False) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\connection.py", line 924, in get return self.get(newpath, ssl, try_json, is_retry=True) File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\connection.py", line 816, in get params['f'] = 'json' TypeError: 'bool' object does not support item assignment At the bottom it also seems that in connection.py the bool test is not valid around line 815 if try_json: params['f'] = 'json' ... View more

We will be trying to do so with FME software. ... View more

I have the same issue: The same Collector application deployed to #PostgreSQL on Linux fails to collect the metadata in iOS or Android phones. The version tied to Oracle 11g works fine. Both are using the same Phones and R1 devices. Is there a version of PostgreSQL that is involved? We are using Collector apps downloaded this month so should be good there. This was one of the only postings I was able to find that was related to this matter, so maybe not a big issue for most, or many are not making use of the metadata or the R1? ... View more

I have the same issue and in addition, there is a 'queryDomains' rest services link that wants a "Layers:" input only, yet no layer value such as 2, or layers=all,2 or similar formats work only format error returned. Basically need the REST api to return the value and not just the code in some cases. ... View more

Update: Denise thank you; I have made good use of the MobileCacheTool taking the MapCache local database on the Trimble to shapefiles and images. I am doing so again now to solve an issue with Syncing, it would still be useful to be able to 'reset' the synchronization state back so that it could be tried again in the cases where the sync appears to work and finishes successfully yet data is not added to the database, One can not retry as some 'bit' gets set which prevents trying again unless you 'touch' each record (add comment etc) and try again or export with the previously mentioned tool. ... View more

I agree - What I was wanting to do (not sure if one can) is for fields that are 'edited' in the background from the GPS device, to have them 'hidden' in the input edit page so the human user didn't have to see the 'clutter' or change the values. I was concerned that if I marked them both not visible and not editable that the GPS unit (R1) would not be able to populate the PDOP, X,Y, and other values even if they are included in the MXD level and just 'hidden' in the application JSON. Not sure how that all works with the settings at each level of the technology each influencing the layer above to a final result. ... View more