Latest Contributions by DonMorrison1

I'm quite sure a new process is not started to run a toolbox tool, so yes, python is keeping things in memory - specially those created at the toolbox/tool class level and even in the getParameterInfo method. I've found number of places where locks are left dangling (even reported some with no resolution) and can't find a way around them other than to restart ArcGIS Pro (or restart the python kernel if running in the IDE) - good luck with this one! ... View more

Hi - I created a process for cloning a Survey 123 'template', but it is probably even more hacky - but it does the job for us so far. I don't know if it could handle your complexity. Our survey is used by a large number of organizations that we have a relationship with.  Each organization gets it's own survey and the survey is customized for that org (underlying web map, thumbnail, title, etc), based on the template.  There are a couple of steps involved.  I'll include the code as-is so it might take a bit to decipher (variables prefixed with 'dp_' contain data from a local configuration database that describes the participating organizations). 1. Whenever the template changes, we run an export to the local file system import os from row.deploy.util.app_info import AppInfo import row.db.organization import row.deploy.util.common from zipfile import ZipFile import shutil import row.consolidation.logger logger = row.consolidation.logger.get('row_log') def export(template_folder, template_title, dp_form_pid): logger.info ("Logging to %s" % (row.consolidation.logger.LOG_FILE)) logger.info("Exporting survey 123 form %s from %s to Git repository" % (template_title, template_folder)) # Find the form's AGOL item gis_item = __get_gis_item (template_folder, template_title, 'Form') # Get the zip file attached to the item and unzip it into the GIT repository logger.info("Downloading spec files") zip_file_name = gis_item.get_data(False) git_dir = os.path.join(AppInfo.path_code_base, 'config', 'templates', dp_form_pid) if os.path.exists(git_dir): shutil.rmtree (git_dir, ignore_errors=True) if os.path.exists(git_dir): os.rmdir(git_dir) os.makedirs(git_dir) logger.info("Survey spec files unzipping to %s" % (git_dir)) with ZipFile(zip_file_name, 'r') as zip_obj: # Extract all the contents of zip file in different directory zip_obj.extractall(git_dir) # Delete the original zip file os.remove(zip_file_name) def __get_gis_item (folder_id, title, type_): for gis_item in AppInfo.gis_agol.users.me.items(folder=folder_id): if gis_item.title == title and gis_item.type == type_: logger.info ("%s: Found survey 123 %s '%s'. GIS ID=%s" % (folder_id, type_, title, gis_item.id)) return gis_item raise Exception ("%s: Can not find survey 123 %s %s" % (folder_id, type_, title))   2.  When we on-board a new organization we created an ArcGIS Online folder for the org, create the web map/data source in that folder, run the 'create' function   3.  When we want to propagate template change to the org-specific versions we run 'update' for each org import os import arcpy from row.deploy.util.app_info import AppInfo import row.deploy.util.common import row.deploy.util.deployment_plan from zipfile import ZipFile import zipfile import shutil import re import json import row.consolidation.logger logger = row.consolidation.logger.get('row_log') def create (dp, dp_form): # Clean up any orphans for gis_item in AppInfo.gis_agol.users.me.items(folder=dp['org_id']): if (gis_item.title == dp_form['title'] or __get_short_title(gis_item.title) == dp_form['title']) and gis_item.type == 'Form': logger.info ("%s: Form '%s' already exists. Deleting it now" % (dp['org_id'], dp_form['title'])) delete (gis_item) # Create the new form with no content information logger.info ("%s: Creating form: '%s'" % (dp['org_id'], __get_full_title(dp_form))) item_properties={ 'title': __get_full_title(dp_form), 'type': 'Form', 'typeKeywords': ["Form", "Survey123", "Survey123 Connect", "xForm"] } # Create the Survey 123 form gis_form = AppInfo.gis_agol.content.add(item_properties=item_properties, folder=dp['org_id']) row.db.persistent_id_map.add_entry (gis_form.id, dp_form['persistent_id'], dp['org_id']) update (dp, dp_form, gis_form) return gis_form def update (dp, dp_form, gis_form): logger.info ("%s: Updating form: '%s'" % (dp['org_id'], __get_full_title(dp_form))) # Make a copy of the template files located in the git directory git_dir = os.path.join(AppInfo.path_code_base, 'config', 'templates', 'SRVY_001') temp_dir = os.path.join(arcpy.env.scratchFolder, dp['org_id'] + '_s123') if os.path.exists(temp_dir): shutil.rmtree (temp_dir, ignore_errors=True) shutil.copytree (git_dir, temp_dir) # Get the item ID of this org's feature class dp_fs = row.deploy.util.deployment_plan.get_pfs (dp, dp_form['xref_pfs_pid']) gis_pfs = AppInfo.gis_agol.content.get(row.db.persistent_id_map.lookup_gis_item_id (dp['org_id'], dp_fs['persistent_id'])) # Get the names of the spec files from the forminfo.json file fn = os.path.join(temp_dir, 'esriinfo', 'forminfo.json') json_ = __read_json_file (dp['org_id'], fn) template_title = json_['name'] # Edit the submission URL in the webform fn = os.path.join(temp_dir, 'esriinfo', template_title + '.webform') text = __read_text_file (dp['org_id'], fn) text = re.sub(r'action=\\"https://www.arcgis.com/sharing/rest/content/items/[a-z0-9]{32}\\"', 'action=\\"https://www.arcgis.com/sharing/rest/content/items/%s\\"' % (gis_pfs.id), text) __write_text_file (dp['org_id'], fn, text) # Edit the submission URL and header label in the xml form fn = os.path.join(temp_dir, 'esriinfo', template_title + '.xml') text = __read_text_file (dp['org_id'], fn) text = text.replace('<h:title>', '<h:title>%s ' % dp['org_id'], 1) text = re.sub(r'action="https://www.arcgis.com/sharing/rest/content/items/[a-z0-9]{32}"', 'action="https://www.arcgis.com/sharing/rest/content/items/%s"' % (gis_pfs.id), text) __write_text_file (dp['org_id'], fn, text) # Edit the lat,long in the .info file fn = os.path.join(temp_dir, 'esriinfo', template_title + '.info') lat, lng = __calculate_midpoint (dp, dp_fs) json_ = __read_json_file (dp['org_id'], fn) json_['displayInfo']['map']['home']['latitude'] = lat json_['displayInfo']['map']['home']['longitude'] = lng __write_json_file (dp['org_id'], fn, json_) # Zip up the files and upload into the item zip_file_name = os.path.join(temp_dir, 'ROW.zip') logger.debug("%s: Zipping survey spec files to %s" % (dp['org_id'], zip_file_name)) owd = os.getcwd() os.chdir(temp_dir) with ZipFile(zip_file_name, 'w', zipfile.ZIP_DEFLATED) as zip_obj: zip_obj.write('esriinfo') for root, dirs, files in os.walk('esriinfo'): for file in files: zip_obj.write(os.path.join(root, file)) os.chdir(owd) # Upload the modified zip file gis_form.update(None, data=zip_file_name) # Update the item metadata item_spec = {} item_spec['title'] = __get_full_title(dp_form) item_spec['description'] = dp_form['description'] item_spec['tags'] = dp['tags'] item_spec['snippet'] = dp['summary'] item_spec['ownerFolder'] = gis_form['ownerFolder'] url = AppInfo.agol_connect_host_name + '/sharing/rest/content/users/' + AppInfo.agol_connect_user_name + '/' + gis_form['ownerFolder'] + '/items/' + gis_form.id + '/update' resp = row.deploy.util.common.submit_http_req (url, item_spec, 'AGOL') AppInfo.gis_agol.content.get(resp['id']) # Reset the relationship with the underlying feature layer for related_item in gis_form.related_items('Survey2Service', 'forward'): gis_form.delete_relationship(related_item, 'Survey2Service') gis_form.add_relationship(gis_pfs, 'Survey2Service') # Upload thumbnail row.deploy.util.common.upload_thumbnail (gis_form, dp_form['thumbnail']) def delete (gis_form): try: if gis_form is not None: logger.info ("Deleting form: '" + gis_form.title + "'") gis_form_id = gis_form.id gis_form.delete() row.db.persistent_id_map.remove_entry (gis_form_id) else: logger.info ("Could not find form: '" + str(gis_form) + "'") except Exception: logger.info ("Could not delete form: '" + str(gis_form) + "'") def __calculate_midpoint (dp, dp_fs): dp_ags_rest = row.deploy.util.deployment_plan.get_ags_rest (dp, dp_fs['xref_ags_rest_id']) ags_rest_layer_url = dp_ags_rest['url'] + '/' + str(0) url = ags_rest_layer_url.replace(AppInfo.ags_service_rest_base, AppInfo.ags_service_rest_base_internal) + '/query' params = {'returnExtentOnly' : 'true', 'returnCountOnly': 'true', 'where': '1=1', 'returnGeometry': 'false', 'spatialRel': 'esriSpatialRelIntersects', 'outSR': '4326'} data = row.deploy.util.common.submit_http_req (url, params, 'AGS_CONSUMER', False) x_min, y_min, x_max, y_max = [data['extent']['xmin'], data['extent']['ymin'], data['extent']['xmax'], data['extent']['ymax']] if x_min != 'NaN' and y_min != 'NaN' and x_max != 'NaN' and y_max != 'NaN': lng = (x_min + x_max) / 2 lat = (y_min + y_max) / 2 else: lat = 38.8627768334364 lng = -96.6298169140754 return lat, lng def __read_text_file (org_id, fn): logger.debug ("%s: Opening text file for edit: '%s'" % (org_id, fn)) with open(fn) as text_file: return text_file.read() def __write_text_file (org_id, fn, text): logger.debug ("%s: Updating text: '%s'" % (org_id, fn)) with open(fn, 'w') as text_file: text_file.write(text) def __read_json_file (org_id, fn): logger.debug ("%s: Opening text file for edit: '%s'" % (org_id, fn)) with open(fn) as json_file: return json.load(json_file) def __write_json_file (org_id, fn, json_): logger.debug ("%s: Updating json: '%s'" % (org_id, fn)) with open(fn, 'w') as json_file: json.dump(json_, json_file, indent=4) def __get_full_title (dp_form): return dp_form['title'] + ' ' + dp_form['version'] def __get_short_title (full_title): return full_title.rsplit(' ', 1) [0]       ... View more

If it is the consolidation process that is changing the string to something incorrect (I've seen this happen with directory paths but not urls), then I've been able to get around problems like that by breaking up the string literal. I'm not sure if this is relevant to your situation...... Here is an example of how I did it # To get around a problem with consolidation we break the template directory name apart # to trick the consolidator so it doesn't try to change the path (for shared projects the # consolidator puts the files in the right place but generates the wrong path literals) template_rptx = os.path.dirname(__file__) + os.path.sep + 'report' + '_' + 'templates' + os.path.sep + 'CCAA_Report.rptx'   ... View more

Can you please post your code snippet to show how you are using defining the URL and using it to access the data store? ... View more

You should only pass 2 parameters to the init function - 'self'  is sort of implied, which I find confusing coming from a C++ background. Anyway - here is how I would do it which I think is a bit clearer. It didn't look like you were using the FID field so I got rid of it - and I now see how you calculate the assessment.  Using a class to calculate the assessment seems like overkill, but you must have a reason to use it .....   cursor = arcpy.da.UpdateCursor(fc, ["Landuse","Value", "Tax"]) for land_use, value, tax in cursor: myparcel = parcelclass.parcel(land_use, value) mytax = myparcel.assessment() cursor.updateRow([land_use, value, mytax])   ... View more

It would be best if you format your code so we can see the indentation and explain what you mean by being "stuck".  Without putting a lot of thought into it (since it seems the objective is for you to figure it out) I would try something along these lines, in terms of writing out the calculated tax value :   cursor = arcpy.da.UpdateCursor(fc, ["FID","Landuse","Value", "Tax"]) for row in cursor: myparcel = parcelclass.parcel(row[1],row[2],row[3]) # This doesn't look right mytax = myparcel.assessment() # This call appears to only return the rate, not the assessment cursor.updateRow(row[1],row[2],row[3], mytax)   ... View more

Other than running slow does it work OK?  If line 19 is trying to print the number of features selected on line 18, then your code looks wrong to me.  I would expect 'selectedFeatures[2]' would give you the correct count.  On the performance question, you could try to explicitly delete the layer object that is getting created every iteration 'arcpy.Delete_management(selectedFeatures[0])' . I doubt it will help but it is worth a try. ... View more

Right - it seems to be the equivalent.  Can you do something similar with your data and get it to work in the ArcGIS Pro python window? That might indicate a problem in your python environment. ... View more

This works for me when I run it in the ArcGIS Pro python window aprx = arcpy.mp.ArcGISProject('CURRENT') active_map = aprx.activeMap rfc = r'E:\CW_Hub\temp\CW_Boundary_2009\CW_Boundary_2009.shp' grplyrfile = r'E:\CW_Hub\temp\Group layer.lyrx' grplyr = arcpy.mp.LayerFile(grplyrfile) osrm_group = active_map.addLayer(grplyr)[0] osrm_lyr = active_map.addDataFromPath(rfc) active_map.addLayerToGroup(osrm_group, osrm_lyr) active_map.removeLayer(osrm_lyr) ... View more

Try using "!shape.area!" in your expression instead of "![SHAPE.STArea()]!". I'm not sure why it is prepending the field with the schema name but perhaps "area" is a reserved name and you should try something else like "my_area". ... View more

I'm not surprised by this based on my experience with publishing Python toolboxes.  The  tool's "updateParameters" function, which provides the ability to react to parameter value changes, is never called in the published version of the tool, so the parameters are basically static, based on what the tool returns from the "getParameterInfo" function (which is called only when the tool is initialized).  I've never used dependent parameters so I'm not sure how that is affected.  I was trying to find the ESRI help page that discusses this limitation but can't seem to find it right now.  ... View more

Did you try schema_type="NO_TEST" and the arcpy.Append_management call?  This allows appends when schemas don't match, however I suspect it will still fail if you have a value in one of those two fields that does not fit in the target column. But it might be worth a try. ... View more

I would try to delete the table using the Delete Geoprocessing tool in ArcGIS Pro to see if you can establish that  it is related to something in your python code.  Another possibility is something in the database itself has it locked. Our systems use MS SQL Server and I would guess there are things you could do in the MS SQL Server Manager that would lock up a table. I've never had a problem with the arcpy.Exists call leaving the object locked.  ... View more