Latest Contributions by LindsayRaabe_FPCWA

I have the below script which does the following: Logs into ArcGIS Online Reads a feature service URL from a csv Uses that feature service URL to save a local copy of the data as a feature class Recalculates the relevant Area field in the dataset Disables the editor tracking on the feature service Uses UpdateCursor to match any updated records to the original in the feature service and updates the area value in the feature service Renables editor tracking Repeat for the next feature service in the csv.  This script works when run in the interpretter, and has been running on our server when trigged by Task Scheduler overnight for some weeks. All of a sudden, now it doesn't.  I'm getting told that the feature service URL is not a valid parameter for ExportFeatures, and also not compatible with Update Cursor (according to Copilot).  I managed to find a way around the ExportFeatures issue by using Feature Layer to create a local copy.  What I can't figure out is why this was working with no issues, now isn't despite no software updates being done on the server, or data configuration changes occurring. Has ArcGIS Online restricted the use of the feature service URL as a data source in arcpy? import arcpy import time import os import csv import tempfile from arcgis.gis import GIS from arcgis.features import FeatureLayerCollection # Start Timer startTime = time.time() try: import certifi import urllib3 import ssl import warnings from urllib3.exceptions import InsecureRequestWarning # Overwrite Output arcpy.env.overwriteOutput = True # Create a default SSL context with certificate verification ssl_context = ssl.create_default_context(cafile=certifi.where()) http = urllib3.PoolManager(ssl_context=ssl_context) # Make a request to verify the setup response = http.request('GET', 'https://org-wa.maps.arcgis.com') print("http response: " + str(response.status)) # Suppress only the single InsecureRequestWarning from urllib3 if necessary warnings.simplefilter('ignore', InsecureRequestWarning) # Create GIS object print("Connecting to AGOL") client_id = '#####' client_secret = '#####' gis = GIS("https://org-wa.maps.arcgis.com", client_id=client_id, client_secret=client_secret) print("Logged in as: " + gis.properties.user.username) # Create a temporary directory temp_dir = tempfile.mkdtemp() # Define the path for the temporary File Geodatabase temp_gdb = os.path.join(temp_dir, "tempAreaHa.gdb") # Create the File Geodatabase arcpy.CreateFileGDB_management(temp_dir, "tempAreaHa.gdb") # Set the workspace to the temporary File Geodatabase arcpy.env.workspace = temp_gdb def remove_last_part(url): # Split the URL by '/' and join all parts except the last one return '/'.join(url.split('/')[:-1]) def extract_fc_name(url): # Extract the feature class name from the URL and clean it path_parts = url.split('/') fc_name = path_parts[-3] return fc_name # Read the CSV file containing feature service URLs with open(r'\\gis-prod\orggis\Update\Feature Service Update Scripts\FeatureServiceAreaUpdates.csv', mode='r') as file: csv_reader = csv.reader(file) next(csv_reader) # Skip header row if present for row in csv_reader: featureservice = row[0] fc_name = extract_fc_name(featureservice) with arcpy.EnvManager(preserveGlobalIds=True): temp_Areas = arcpy.conversion.ExportFeatures( in_features=featureservice, out_features=arcpy.env.workspace + "\\" + fc_name, where_clause="", use_field_alias_as_name="NOT_USE_ALIAS", field_mapping=f'AreaHa "Area (ha)" true true false 0 Double 0 0,First,#,{fc_name},AreaHa,-1,-1;GlobalID "GlobalID" false false true 38 GlobalID 0 0,First,#,{fc_name},GlobalID,-1,-1', sort_field=None ) print(f"Features exported for {fc_name}") # Determine which area field exists fields = [f.name for f in arcpy.ListFields(temp_Areas)] area_field = None for candidate in ["AreaHa", "Area", "Hectares"]: if candidate in fields: area_field = candidate break if area_field: print(f"Calculating area for field: {area_field}") arcpy.management.CalculateGeometryAttributes( in_features=temp_Areas, geometry_property=f"{area_field} AREA", length_unit="", area_unit="HECTARES", coordinate_system='PROJCS["GDA2020_Australian_Albers",GEOGCS["GDA2020",DATUM["GDA2020",SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Albers"],PARAMETER["False_Easting",0.0],PARAMETER["False_Northing",0.0],PARAMETER["Central_Meridian",132.0],PARAMETER["Standard_Parallel_1",-18.0],PARAMETER["Standard_Parallel_2",-36.0],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]', coordinate_format="SAME_AS_INPUT" ) print(f"Areas calculated for {fc_name}") else: raise ValueError(f"No matching area field found in {fc_name}. Expected one of: 'AreaHa', 'Area', or 'Hectares'.") # Remove the last part of the URL modified_url = remove_last_part(featureservice) # Access the feature layer collection flc = FeatureLayerCollection(modified_url, gis) # Disable editor tracking disable_params = { "editorTrackingInfo": { "enableEditorTracking": False } } flc.manager.update_definition(disable_params) print(f"Editor tracking disabled for {fc_name}") # Create a dictionary from the exported table sourceFieldsList = ['GlobalID', area_field] valueDict = {r[0]: r[1] for r in arcpy.da.SearchCursor(temp_Areas, sourceFieldsList)} # Update the feature service using the dictionary updateFieldsList = ['GlobalID', area_field] print(f"\nUpdating {area_field} field in {fc_name}...\n") with arcpy.da.UpdateCursor(featureservice, updateFieldsList) as updateRows: for updateRow in updateRows: keyValue = updateRow[0].upper() if keyValue in valueDict: if valueDict[keyValue] != updateRow[1]: for n in range(1, len(sourceFieldsList)): print(f"Updating {area_field} for {updateRow[0]} from {updateRow[1]} to {valueDict[keyValue]}") updateRow[n] = valueDict[keyValue] updateRows.updateRow(updateRow) del valueDict print(f"\n{area_field} field updated successfully for {fc_name}") # Re-enable editor tracking enable_params = { "editorTrackingInfo": { "enableEditorTracking": True, "enableOwnershipAccessControl": True, "allowOthersToUpdate": True, "allowOthersToDelete": True } } flc.manager.update_definition(enable_params) print(f"Editor tracking re-enabled for {fc_name}\n") endTime = time.time() print(f"\nScript completed in {endTime - startTime} seconds.") except Exception as e: if "The Product License has not been initialized" in str(e): e = "\n\n" + r"ArcGIS Pro license not available." if "cannot open 'https" in str(e): e = "\n\n" + r"Login to ArcGIS Online failed - check ArcGIS Pro on the FPC-GIS-01 server is logged in using an Admin account." if "Exception: Failed to execute. Parameters are not valid." in str(e): e = "\n\n" + r"Either login to ArcGIS Online failed - check ArcGIS Pro on the FPC-GIS-01 server is logged in using an Admin account, or Parameters need fixing if changes have been made in ArcGIS Online to the input layers." from email.mime.multipart import MIMEMultipart from email.mime.text import MIMEText import smtplib, os fromaddr = "GISNotifications@org.wa.gov.au" toaddr = "gis@org.wa.gov.au" msg = MIMEMultipart() msg['From'] = fromaddr msg['To'] = toaddr msg['Subject'] = "Updating AreaHa field process failed (" + os.path.basename(__file__) + ")" # Enter email body text below body = "There has been an error in the script that checks and updates the AreaHa field in listed feature services. \n\nPlease check the script to troubleshoot any problems. \n\nException: " + str(e) # DON'T include URL's or links to the web in the body of the email or it might get blocked by the scam filters msg.attach(MIMEText(body, 'plain')) server = smtplib.SMTP('org-wa-gov-au.mail.protection.outlook.com') #No login required so this section is commented out #server.login("youremailusername", "password") server.sendmail(fromaddr, toaddr, str(msg)) print ("\nScript failed - email notification sent to gis@org.wa.gov.au") print ("\nException: " + str(e)) ... View more

IDEA: In line with the option in Enterprise to include data classification schemas in the organisation settings, add the option to classify the sensitivity of data in ArcGIS Online.  Use Case: When a feature service is loaded into an ArcGIS Pro map, the layer with the highest level of data classification will inform the overall sensitivity rating applied to a layout that would then be honoured as part of the export process (e.g. export to PDF) as per this related idea: Include Document Classification Functions in ArcGIS Pro's Export Layout to PDF  ... View more

Problem: In Microsoft Word, Excel and Powerpoint, plus Adobe PDF's, our organisation has enabled Add-ins that assign a document classification to every document/email. By default, each document receives the classification "Official" and we can then adjust it down or up to suit the required level of sensitivity. It's been asked that we start applying these categories to our exported PDF maps from ArcGIS Pro. From my research, it seems at present this can only be done doc by doc manually after exporting has happened in Adobe. This may be fine for the occasion map export, but we leverage Map Series to export dozens or hundreds of maps at a time and manually assigning a sensitivity category to each document is not an efficient use of time.  IDEA: Include "Document Classification" that leverages the "Microsoft Purview Information Protection" to the Export Map/Layout pane under the Security settings.  In Microsoft Word: In Adobe:   ... View more