ArcGIS Pro v 3.0.0 and changes to python?

erica_poisson — Thu, 22 Sep 2022 18:17:08 GMT

Hi -

One of my users contacted me because a python script he is running has started giving him some issues after our organization pushed Pro version 3.0.0 to our workstations. His script (below) is exporting a feature layer to a CSV, which is then being pulled into R for further analysis. Previously, the CSV would export and any null values in the attribute table would just be blank/empty. Now, they are all coded to "None" which has thrown off his processes in R. He added logic into the R script to deal with these new "None" values, however I was curious if there was some kind of change at Pro version 3.0.0 which could have caused this. His Py script had been updated to Python 3 awhile ago and was running fine. We are uncertain what the problem could be, but the issues started happening right around when we upgraded ArcGIS Pro.

I also notice that if I go to the hosted feature layer in ArcGIS Online and export to a CSV from the Item Details page, null values are empty and not coded as "None". Same if I view the feature layer in AGOL or Pro.

Any ideas on what could be causing this would be appreciated.

Here is the Python script:

#!C:\\Program Files\\ArcGIS\\Pro\\bin\\Python\\envs\\arcgispro-py3 python # coding: utf-8 # In[60]: import arcgis from arcgis.gis import GIS from arcgis.gis import * import os import getpass portal_url = rhttps://mass-eoeea.maps.arcgis.com/ # ArcGIS user credentials to authenticate against the portal username = 'USERNAME' password = 'PASSWORD' gis = GIS(portal_url, username, password) # ID of the feature service you want to export featureServiceID = "0d081b35bbe2466290926f0f09faffe5" # Get username username = getpass.getuser() # Output format Shapefile | CSV | File Geodatabase out_format = 'CSV' # Local folder where the data will be downloaded (include slash at the end) download_folder = "C:\\Users\\" + username + PATH # Address of your ArcGIS portal portal_url = rhttps://mass-eoeea.maps.arcgis.com/ download_file = download_folder + 'download.zip' # ID of Field Data item_id = featureServiceID filename = "stage_turbidity" + "." + out_format outpath = download_folder + filename # Get content and save to csv my_item = gis.content.get(item_id) my_layer = my_item.tables[0].query().sdf my_layer.to_csv(outpath, index = False) #Download photos download_folder_photos = download_folder + "Photos\\" #Query to get list of object ids in layer featureLayer = my_item.tables[0] featureObjectIds = featureLayer.query(where='1=1', return_ids_only=True) #Loop through features in layer emptyAttachments = 0 for j in range(len(featureObjectIds['objectIds'])): currentObjectId = featureObjectIds['objectIds'][j] currentObjectIdAttachments = featureLayer.attachments.get_list(oid=currentObjectId) if len(currentObjectIdAttachments) > 0: #Loop through feature attachments and download to appropriate folder for k in range(len(currentObjectIdAttachments)): attachmentId = currentObjectIdAttachments[k]['id'] attachmentName = currentObjectIdAttachments[k]['name'] attachmentParentId = currentObjectIdAttachments[k]['parentGlobalId'] FileName = '{}'.format(attachmentName) AttachmentPath = '{}\\{}'.format(download_folder_photos, FileName) FinalFileName = '{}.{}'.format(attachmentParentId,"jpg") FinalAttachmentPath = '{}\\{}'.format(download_folder_photos, FinalFileName) if not os.path.isfile(FinalAttachmentPath): featureLayer.attachments.download(oid=currentObjectId, attachment_id=attachmentId, save_path=download_folder_photos) os.rename(AttachmentPath, FinalAttachmentPath) print("Finished downloading csv file.") # Run batch file to process csv outputs and import into Database file = "C:\\Users\\" + username + PATH.bat os.system('"' + file + '"') print ("Stage/Turbdiity Data Processed with R Script" )

Thank you,

Re: ArcGIS Pro v 3.0.0 and changes to python?

VinceE — Fri, 18 Nov 2022 01:02:40 GMT

I don't have much to contribute, other than that I noticed similar issues in the way ArcGIS Pro seems to handle/flag/calculate <Null>/None in an attribute table at the 3.0 version level.

When using the Field Calculator and Python as the parser, calculating a numeric or a text field back to "None" (without the quotes, of course) yields a new warning message that was not present in previous versions of the software (for this particular calculation back to "<Null>", anyway). Operation still works, but it seems to indicate to me that either: A) NULLs are being handled/read/recorded differently, or B) this is totally unrelated to your problem and it's just a friendly warning message they now provide for... some reason.

Here is my posted question about this, in case the answer provided buy @Robert_LeClair is of any help to you:

Solved: Re: Calculating "None" Using Field Calculator Yiel... - Esri Community

topic ArcGIS Pro v 3.0.0 and changes to python? in ArcGIS Pro Questions

ArcGIS Pro v 3.0.0 and changes to python?

Re: ArcGIS Pro v 3.0.0 and changes to python?