Latest Contributions by GerryGabrisch

‎01-08-2022

Yes, adding the make feature layer and turning off the logging vastly improved code preformance.

‎01-08-2022

That's what I was looking for, thanks.

‎01-07-2022

Turning off geoprocessing logging is always a good idea to keep your CITRIX profile small. As far as code execution goes I am running this as a stand-alone Python script in WingIDE. ArcGIS Pro is not actively running.

‎01-07-2022

Should I delete the feature layer for each selection after it has executed the code for that selection or just keep reassigning the variable each time the code makes a new feature layer?

‎01-07-2022

Thanks, I am running the code in a Python IDE and not in a Python window in ArcGIS Pro. I am assuming there is no geoprocessing logging happening. Is this correct or is geoprocessing logging a basic function in arcpy?

‎01-07-2022

This python script selects groups of data on specific days, processes those data, and selects another group of data to process. The code executes quickly when first executed. As the code chugs along it gets slower and slower. There are no nested cursors, no nested loops - just selections and data processing. Why is code execution getting slower over time? Any tips or tricks for preventing this type of slowdown is appreciated. import sys import traceback import os import shutil import arcpy import shapefile try: arcpy.env.overwriteOutput = True #The input shapefile... in_fc = r"C:\GIS\telemetrytags\projected_data\telemetryUTM.shp" #An empty directory to store the geometric average outputs out_dir = r'C:\GIS\telemetrytags\individual_fish_individual_days' #For the first cursor only these fields are needed... fields = ['TagID', 'Date'] #A list to hold all of the tag ids... tag_ids = [] #A list to hold the survey dates ping_dates = [] #Iterate the shapefile and get a list of all unique dates and a list of all unique tags. with arcpy.da.SearchCursor(in_fc, fields) as cursor: for row in cursor: if row[0] not in tag_ids: tag_ids.append(row[0]) if row[1] not in ping_dates: ping_dates.append(row[1]) #Do you need to delete row now with python 3? Doing so did not speed up the code which lags over time... del row #This is the field list for populating the rest of the attribute table of the mean center files... fields = ['Origin','Stock','Sex','FL','Destinatio', 'Mortality_'] #Build the query to select individual tags on individual days... for tag in tag_ids: for ping in ping_dates: #Build the query... query = '\"TagID\" = ' + str(int(tag)) + ' And \"Date\" = timestamp ' +'\'' + str(ping).split(' ')[0] +'\'' print(query) #Select all those tags on that particular day... this_fish, ping_count = arcpy.management.SelectLayerByAttribute(in_fc, selection_type="NEW_SELECTION", where_clause=query) print('ping return count = ', ping_count) #If a fish was not found on a particular day then ignore that selection... if ping_count == 0: pass #If there is only one ping then just rename the original file and move it to the out_dir. if int(ping_count) == 1: out_fc = out_dir + "\\" + str(int(tag)) + '_' + str(ping).split(' ')[0] + '.shp' arcpy.CopyFeatures_management(this_fish, out_fc) #Clean up that table for the final merging of shapefiles... arcpy.DeleteField_management(out_fc, ['MEAS','Pos_ft','CumPos_ft','Pos_RM','CumPos_RM','Time','Channel','Antenna','Power','Latitude','Longitude','NEAR_FID','NEAR_DIST','NEAR_X','NEAR_Y','Data_Inclu']) #More that one ping on a given day needs to be averaged... if int(ping_count) > 1: #Build the output file and path for the averaged data... out_fc = out_dir + "\\" + str(int(tag)) + '_' + str(ping).split(' ')[0] + '.shp' date = str(ping).split(' ')[0] #Create the mean center file based on the geospatial locations... arcpy.stats.MeanCenter(this_fish, out_fc, Weight_Field='Power') #The mean center operation output does not preserve attributes. Create them now... arcpy.AddField_management(out_fc, 'TagID', "SHORT") arcpy.AddField_management(out_fc, 'Date_', "TEXT", field_length='12') arcpy.AddField_management(out_fc, 'Origin', "TEXT", field_length='3') arcpy.AddField_management(out_fc, 'Stock', "TEXT", field_length='20') arcpy.AddField_management(out_fc, 'Sex', "TEXT", field_length='1') arcpy.AddField_management(out_fc, 'FL', "SHORT") arcpy.AddField_management(out_fc, 'Destinatio', "TEXT", field_length='10') arcpy.AddField_management(out_fc, 'Mortality_', "TEXT", field_length='20') #These values are all the same between records, get the values to populate the out_fc... with arcpy.da.SearchCursor(this_fish, fields) as cursor: for row in cursor: Origin = row[0] Stock = row[1] Sex = row[2] FL = int(row[3]) Destinatio = row[4] Mortality_ = row[5] del row arcpy.CalculateField_management(out_fc, 'TagID', int(tag), "PYTHON3") arcpy.CalculateField_management(out_fc, 'Date_', "'"+ date +"'", "PYTHON3") arcpy.CalculateField_management(out_fc, 'Origin', "'"+Origin+"'", "PYTHON3") arcpy.CalculateField_management(out_fc, 'Stock', "'"+Stock+"'", "PYTHON3") arcpy.CalculateField_management(out_fc, 'Sex', "'"+Sex+"'", "PYTHON3") arcpy.CalculateField_management(out_fc, 'FL', FL, "PYTHON3") arcpy.CalculateField_management(out_fc, 'Destinatio', "'"+Destinatio+"'", "PYTHON3") arcpy.CalculateField_management(out_fc, 'Mortality_', "'"+Mortality_+"'", "PYTHON3") except Exception: tb = sys.exc_info()[2] tbinfo = traceback.format_tb(tb)[0] pymsg = "PYTHON ERRORS:\nTraceback info:\n" + tbinfo + "\nError Info:\n" + str(sys.exc_info()[1]) msgs = "ArcPy ERRORS:\n" + arcpy.GetMessages(2) + "\n" arcpy.AddError(pymsg) arcpy.AddError(msgs)

‎01-06-2022

I have a ArcGIS Pro v2.8 stand-alone Python script. This script create a new shapefile via the mean center tool. I use python to add a new field to this mean center output as a type short. After viewing the resulting shapefile in ArcGIS Pro I see that the attribute table is showing my TagID attribute but ArcGIS Pro is referring to the attribute as a long! I can confirm that QGIS 3.10 shows the TagID as an integer. I can confirm that I can populate the attribute table with a numeric value in both QGIS and ArcGIS Pro manually. After I try to populate the new attribute with a calculate field I get the following error. ERROR: ArcPy ERRORS: ERROR 160176: The index passed was not within the valid range. Failed to execute (CalculateField). Here is a link to the tool error...160176 error Here is the relevant code: arcpy.AddField_management(out_fc, 'TagID', "SHORT") tagid = '"TagID"' value = str(int(tag)) arcpy.CalculateField_management(out_fc, tagid, value, "PYTHON3") Thinking that an AddIndex might save the problem I execute the mean center tool then do an Add Index. This gets me to the CalculateField but passing that crashes arcpy. Why is ArcGIS Pro showing the attribute type as Long? Why am I getting this error? How can I fix the problem?

‎01-06-2022

I am trying to execute a calculate field in arcpy from ArcGIS Pro 2.7. Here is the relevant code. #a confirmed existing shapefile .... out_fc = r'C:\GIS\telemetrytags\individual_fish_individual_days\112_2019-04-12.shp' # a confirmed and working add field.... arcpy.AddField_management(out_fc, 'TagID', "SHORT") #The existing field name... tagid = "TagID" #The value to populate TagID with...in this case the value of tag is a float (with a value of 112.0) before I #convert it to an int and then string. value = str(int(tag)) #code fails here with the message below. arcpy.CalculateFields_management(out_fc, tagid, value, "PYTHON3") Error Info: Failed to execute. Parameters are not valid. ERROR 000800: The value is not a member of Python 3 | Arcade. ERROR 000728: Field 112 does not exist within table Failed to execute (CalculateFields). 112 is the value I am trying to populate field TagID with and not the field name so I do not understand what is up with that error. Another oddity - when I add my shapefile to ArcGIS pro and check the TagID field it returns that the field is a type Long with is not what I am doing with the calculate field. Why is this 'the most simple of calculations' failing in arcpy for ArcGIS Pro 2.7

‎03-06-2017

Allan, Yes, my IT department shut down open access to that FTP site. I moved the code and toolbox tool to Github. The link above is now valid.

‎09-07-2016

That tool that I created will create a perpendicular line but the "perpendicular line" is perpendicular to an imaginary line that runs from the start node to the end node so some care is required to get what you want. If you want to create perps at specific places why not create a new line file where each line is tangent to your stream course at the intersect of the point locations, buffer the points some reasonable distance, clip the tangent lines so your tangents are centered on the points, then run the perpendicular lines tool creating new perps at the midpoint of your tangent line.

‎09-07-2016

The output values will be in unit of measure of the projected data. If your raster is using UTM zone X, the volumes are areas output by the tool will be in (assumed) m^3 or m^2. Failure to convert the cm z values to meters will result in calculation errors.

‎09-07-2016

Dan, no, the original output to ASCII from ArcMap was corrupted somehow. As a test I could take that original exported ASCII and convert it back to a GRID and I would get the error. Restarting the desktop seems to have had a magical effect on the output ASCII and I am not getting errors anymore. FYI fw.write(" ".join(newlinelist)+"\n"), I already had that. Thanks again for your help.

‎09-06-2016

I am testing a Python script where an ESRI shapefile is converted to a raster (based on FID). The resulting raster is converted to ASCII for further processing. Interestingly the tool worked great then started giving errors. The source of the error is that the output ASCII file is now inserting extra spaces in the header of the ASCII file where earlier in the day it was not. Here is an example of the problematic ASCII output. ncols 17 nrows 41 xllcorner 1193433.4604796 yllcorner 659603.16649559 cellsize 10 NODATA_value -9999 The column of numbers have extra spaces resulting in a left justified column of data when only one space should exist between the names (ncols, nrows...) and the data (17, 41). What would cause this behavior? Am I stuck with using regular expressions to purge the extra white spaces? try: print "Split polygon by percent area from the top down..." import sys, traceback, os import arcpy arcpy.env.overwriteOutput = True ######################################################################################## #The Input Feature Class or projected data to Divide into Percents... inFC = r"Z:\GISpublic\GerryG\PythonScripts\SplitPolygonByPercent\testData\testdata.shp" #The rasterized version of the polygon... outRas = r"Z:\GISpublic\GerryG\PythonScripts\SplitPolygonByPercent\outRas" #The cell size of the outRas-smaller is better but takes longer to process.. outCellSize = 10 #Useer defined precent by area divisions...will split a polygons into 60%, 20%, and 20% divisions = [60,25,15] #ASCII version of the raster data.... outASCII = r"Z:\GISpublic\GerryG\PythonScripts\SplitPolygonByPercent\testdata.asc" #ASCII version of the split by area data splitASCII = r"Z:\GISpublic\GerryG\PythonScripts\SplitPolygonByPercent\Split.asc" #The Output Raster.... FinalRaster = r"Z:\GISpublic\GerryG\PythonScripts\SplitPolygonByPercent\PercentArea" ########################################################################################## #Holds the total area of each percentage.... areaPercents = [] #Get the polygon area... rows = arcpy.SearchCursor(inFC) shapeName = arcpy.Describe(inFC).shapeFieldName for row in rows: feat = row.getValue(shapeName) polygonArea = feat.area #Calculate the areas for each percent division for item in divisions: areaPercents.append(polygonArea* item/100.0) print "Converting polygon to ESRI GRID..." arcpy.PolygonToRaster_conversion(inFC, "FID",outRas,"", "", outCellSize) print "Converting ESRI GRID to ASCII..." arcpy.RasterToASCII_conversion(outRas, outASCII) print "Create the new ascii file" fw = open(splitASCII, 'a') #Keep track of the line number to ensure the header gets written correctly to #the new .asc file linecounter = 1 cell_area_counter = 0 thispercentcounter = 0 with open(outASCII, 'r') as f: for line in f: #The first 6 lines are header-just copy that over to the new asc file. if linecounter <=6: fw.write(line) #The remaining lines have data...convert them to percent values.. else: #Strip those pesky blank spaces at the start and ends of the line line = line.strip() linelist = line.split(" ") newlinelist = [] for item in linelist: if item =="-9999": newlinelist.append(item) else: #thispercentcounter +=1 cell_area_counter = cell_area_counter + (outCellSize*outCellSize) if cell_area_counter <areaPercents[thispercentcounter]: newlinelist.append(str(divisions[thispercentcounter])) else: thispercentcounter +=1 cell_area_counter = 0 print thispercentcounter newlinelist.append(str(divisions[thispercentcounter])) fw.write(" ".join(newlinelist)+"\n") linecounter+=1 print "Creating output raster..." arcpy.ASCIIToRaster_conversion(splitASCII,FinalRaster, "INTEGER") #Take out the trash.... os.remove(outASCII) os.remove(splitASCII) arcpy.Delete_management(outRas) print "Done" except: print "error" tb = sys.exc_info()[2] tbinfo = traceback.format_tb(tb)[0] pymsg = "PYTHON ERRORS:\nTraceback info:\n" + tbinfo + "\nError Info:\n" + str(sys.exc_info()[1]) print pymsg + "\n"

‎08-03-2016

Thanks for the tool. It worked like a charm!

‎01-04-2016

Dan, I can get the extent of all the elements in an MXD and I can use those values and the Python Image Library to crop the exported image, but for some unknown reason the clipped output has a few pixels trimmed off of the left and bottom edges of the image. I am not sure of the cause without more investigation. See the example below.

Online Status	Offline
Date Last Visited	‎09-23-2025 10:15 AM

My Ideas

Latest Contributions by GerryGabrisch

Re: Why does this code take longer to execute over time?

Re: Why does this code take longer to execute over time?

Re: Why does this code take longer to execute over time?

Re: Why does this code take longer to execute over time?

Re: Why does this code take longer to execute over time?

Why does this code take longer to execute over time?

Add field returns type long not type short

Calculate Field in ArcGIS Pro 2.7 Python Stand-Alone Fails

Re: Construct perpendicular lines

Re: Create a perpendicular line to a stream segment and plot elevation profile

Re: Units for Surface Volume tool

Re: Raster to ASCII inserts extra spaces in header?!

Raster to ASCII inserts extra spaces in header?!

Re: Calculating Bearing of a Polyline ArcGIS 10

Re: Arcpy.mapping Get the extent of the map elements.

Please stop sending me direct mail

User cannot see a directory in Catalog but it is v...

Re: Cannot select or edit elements in Layout view

Re: Creating KML files with Attribute Data

Hey ESRI, your hillshade python code example is wr...