Latest Contributions by RichardFairhurst

‎06-09-2015

I am not sure I understand what I am seeing regarding the difference between clip and selection. I don't really understand the analysis taking place with this code, so I am not sure what is best to suggest. It also seems like you are getting the lengths of the n2 sequence, not the n sequence, and I am not sure why. In any case, if every road must ultimately be intersected by every envelope, please try the Intersection tool with an output of Lines. Output to a file geodatbase feature class, not a shapefile (stop using shapefiles, I beg you), so that the length of the roads is automatically provided for you. Keep track of the time it takes to process to make sure it completes in much less time than 288 minutes (I expect it will take roughly 5 to 10 minutes). Then open the feature class table view, select one of the sequence IDs you already know the length for and get the statistics of the length field (right click, statistics). If it is the same length (you should ignore any difference below 1/10th of a meter or at most 1/100th of a meter, since file geodatabases provide higher precision shapes than shapefiles), we can run intersect before any of the loops we have been altering and can use that output for all sequence lengths in the final step using the dictionary summary I mentioned. If I am right, this will eliminate days of processing to complete your entire dataset.

‎06-09-2015

Jamal: I see you still don't bother to read the most basic things about anything you try and use the forum to have other people read the help for you. Making the faulty assumption that a Pro project operates identically to an MXD map shows you have not bothered to look at even the most basic information about Pro.

‎06-09-2015

I don't use clip much for my work, so I cannot be sure what result occurs if you did all of the envelops against all of the sequence polygons at once for one of the 360 envelope layers, but that is worth a try. The set up of the dictionary for the output can use the One To Many approach to compile a summary of lengths directly for all clipped shapes based on the sequence IDs rather than a list of lists. So for example it can be structured like: envelopeFieldsList = ["Sequence", "Shape@"] envelopeDict = {} with arcpy.da.SearchCursor(envelopeFC, envelopeFieldsList) as envelopes: for envelope in envelopes: relatekey = envelope[0] geometry = envelope[1] if relatekey in envelopeDict: envelopeDict[relatekey] += geometry.length else: envelopeDict[relatekey] = geometry.length I would expect that you would have many envelopes clipping the same sequence polygon, so the code above would probably have to be modified to keep track of an envelope ID field for each summarized value in addition to the Sequence number I would need to know more about the output of the multiple envelope clip from your data before proposing how to do that. Of course you should time the multiple envelope clip to be sure that the batch process saves time by avoiding the clip set up steps of the individualized process. Figure out how many objects in both layers are being clipped and multiply that by the 2 minutes you have measured for a single envelope to compare to the time spent doing the batch process. If there is a significant time savings than that approach is worthwhile, but if it is negligible than the clip operation needs to be reexamined to find out if there are alternative ways to reach the final goal. For example, Intersect may work and may (or may not) be faster. Also look at the help here. It may work faster to do clip geometry operations directly on geometry objects using geometry envelopes rather than using the clip tool. There is a good chance this would save some time, since it would not waste time creating a feature class schema or outputting a feature class to disk or memory, since you can don't seem to need that and could handle the field schema through integrating dictionary data. Anyway, code optimization usually is a result of the process we are going through, You have to start with what you know and what works and then isolate speed bottlenecks and develop alternatives until further optimization becomes impossible or too costly for the potential benefit. It is normally a learning experience. Necessity is the mother of invention.

‎06-09-2015

I tried to see if it would work with the few changes I made, but by changing from a one to many dictionary to a one to one dictionary I need to make more changes. This code should access the FromPrevTr and FromPrevDi fields. if n2 in sequenceDict: postTime = sequenceDict[n2][0] totTime = prevTime + float(postTime) print totTime dataList.append(str(totTime)) postDist = sequenceDict[n2][1] # print postDist totDist = prevDist + float(postDist) # print totDist dataList.append(str(totDist)) What is your clip layer? Before I completely rebuild this section I need to better understand what is going on. I would load cliplayer into a dictionary and include its sequence and Shape@ fields once at the beginning of the script for fast access if it has many records. The geometry for the n2 sequence record can be fed directly to the clip process based on the first example in this help post (ignore the array, it can come from the shape@ field) For using the in_memory workspace you should get a better understanding (which I also need). Start by looking at the help here. You use in_memory as a path and you still need a slash and a feature class name after it for output. It is more like a geodatabase (which you should convert to anyway, since shapefile performance sucks). So it would be more like changing line 173 to: clipRoads = r"in_memory\" + 'rdsClip_' + blockEID + '_' + scaleEID + '_' + bufEID + '_' + str(n) You would not add .shp to the end of the feature class name, since it is not outputting a shapefile. If you use the in_memory workspace and do not actually need the output of the clip permanently, you should get rid of the individualized feature class names and should delete the feature class with arcpy.Delete_management() as soon as you are done with it to keep the memory clean. Why do you use CopyFeatures to a geometry object and not a cursor to read through the geometry lengths? Duplicating the feature again is a waste of time as far as I can see. Anyway, clipping one shape at a time will be a slow process, but it can be sped up. You should print a time stamp and time difference before and after the actual clip to find out how much time is being eaten by each clip operation. Only by batch processing many objects in one clip are you likely to speed that up significantly, since the set up time for each clip is probably what is killing your time. You should still notice a speed improvement in the section of code we have revised. In fact you should start adding print statements to show time spent in each key section if you are trying to find the most time consuming operations in your code. You also need to evaluate what is the maximum time that this process can take for it to actually be useful to you. I have had to abandon several approaches I spent considerable time developing when the time spent doing them did not pay off sufficiently. I usually can rethink it eventually to hit my performance tolerance and get a close enough result that I can live with. If your process cannot perform acceptably because of an operation you cannot control (the clip), you may want to consider alternative processes that may be less precise, but that may produce a more worthwhile output in light of the performance improvements they offer.

‎06-08-2015

Are you sure that the sequence field is a string field? I just made that assumption, but it probably is wrong. If sequence is a numeric field then the key is wrong at lines 133 and 134. The code would have to be: if n2 in sequenceDict: values = list(sequenceDict[n2]) Also, if you want to process the dictionary sorted, you don't sort the dictionary, since it can't be sorted. You sort the list of dictionary keys instead. I don't recommend using an OrderedDictionary, since it isn't necessary for this process. So line 110 would be (assuming the sequence key is numeric and not string): for key in sorted(sequenceDict.keys()): This won't have any noticeable effect the speed of the code, but it will output the data in the order that you probably want.

‎06-08-2015

I have fixed the items pointed out by Blake. I also corrected the code in line 22 and 24 to reverse them. I had them backwards (or else I had forgotten to put the word "not" in line 21). Since the depot visits table only has two records then you could replace lines 13 through 24 with (using the same indentation as line 13): qLast = "= 2" expression2 = arcpy.AddFieldDelimiters("depotVisits", "VisitType")+ qLast dopotVisitsDict = {r[0]:(r[1:]) for r in arcpy.da.SearchCursor(sourceFC, sourceFieldsList, expression2)} Then you would also have to replace line 77 with (using the same indentation as line 77): row3 = list(depotVisitsDict[2])

‎06-07-2015

I have created a Python toolbox tool that converts the set of unique key values found in multiple fields that relate two layers/tables into a set of related sequential numbers in a CASE_ID field populated in both layers/tables. This allows a user to create a standard join or relate between the two layers/tables on the CASE_ID field that is equivalent to creating a multi-field join or relate. This tool was inspired by the ArcInfo Workstation's FREQUENCY command which could optionally add a numeric case field to the source and output that would maintain a single field relationship when the frequency was based on more than one case field. That capability was lost in the Desktop Frequency tool. However, the tool I have created is actually more flexible than what Workstation provided, since it can be applied to any pair of layers/tables, even when the Frequency tool had nothing to do with how they were created or related. The two zipped python toolboxes attached are designed for ArcGIS 10.3 (Field Match Tools.pyt.zip) and ArcGIS 10.2 (Field Match Tools v10.2.pyt.zip) . The toolboxes should be placed in the "%APPDATA%\ESRI\Desktop10.[3]\ArcToolbox\My Toolboxes" folder to show up in your My Toolbox under ArcCatalog (modify the items in brackets to fit your Desktop version). I use python lists and itertools to get my sorted unique list of multiple field key values and to generate the sequential numbers associated with each key, but I convert the list into a dictionary prior to running the updateCursor so that I gain the speed or dictionary random access matching when I am writing the Case ID number values back to the data sources. Dictionary key matching is much faster than trying to locate matching items in a list. Here is the interface: The validation works to make sure that the two input data sources are not the same and that the fields used in the case field list are actually in both sources. The user can choose as many fields as they want to make up their unique multiple field case value keys. The field names do not have to be the same in both data sources. The position of the fields in the list will control the sort priority of the Case Fields (highest priority = top field) and the Sort Order column controls whether the values in each field are sorted in Ascending or Descending order. The sort order of the Case field values controls the Case ID number sequencing. The arrangement of the fields can be different from the field arrangement actually used in the sources. There are three options for creating sequential numbers in the CASE_ID field output. The first operates like a standard Join, where all unique key values in the Primary table are numbered, but only matching key values in the Secondary table are numbered. All unmatched values in the Secondary table are given a CASE_ID of -1. The second option is an union, where the sequential numbers are based on the complete set of key values in the combination of the Primary and Secondary layers/tables. The third option is an intersection, where only key values found in both layers/tables receive positive sequential numbers. All unmatched values of either table not found in the other table received a CASE_ID of -1. The 10.3 version tool works the best and has the best features, However, I have provided a 10.2 version, but in order for the tool to work under the limitations of 10.2, the tool has fewer capabilities and a somewhat less intuitive interface. The 10.2 version of the tool does not support controlling the sort order of the fields, so their sequential number values are always based on the use of an ascending order for the field values. To match the fields in the two layers/tables you must click on fields in two lists, but those fields only appear in the separate field string text box when you click somewhere outside of the field lists. The tool sidebar help provides more detail on how to use it. Here is my code for the 10.3 version of the tool: import arcpy class Toolbox(object): def __init__(self): """Define the toolbox (the name of the toolbox is the name of the .pyt file).""" self.label = "Field Match Tools" self.alias = "FieldMatchTools" # List of tool classes associated with this toolbox self.tools = [MultiToSingleFieldKey, InsertSelectedFeaturesOrRows] class MultiToSingleFieldKey(object): def __init__(self): """Define the tool (tool name is the name of the class).""" self.label = "Multiple Field Key To Single Field Key" self.description = "" self.canRunInBackground = False def getParameterInfo(self): """Define parameter definitions""" # First parameter param0 = arcpy.Parameter( displayName="Input Primary Table", name="in_prim_table", datatype="DETable", parameterType="Required", direction="Input") # Second parameter param1 = arcpy.Parameter( displayName="Input Secondary Table", name="in_sec_table", datatype="DETable", parameterType="Required", direction="Input") # Third parameter param2 = arcpy.Parameter( displayName="Case Fields", name="case_fields", datatype="GPValueTable", parameterType="Required", direction="Input") param2.columns = [['GPString', 'Primary Case Field'], ['GPString', 'Secondary Case Field'], ['GPString', 'Sort Order']] param2.filters[0].type="ValueList" param2.filters[0].list = ["X"] param2.filters[1].type="ValueList" param2.filters[1].list=["x"] param2.filters[2].type="ValueList" param2.filters[2].list=["Ascending", "Descending"] param2.parameterDependencies = [param0.name] # Fourth parameter param3 = arcpy.Parameter( displayName="Case ID Field Name", name="in_Case_ID_field", datatype="GPString", parameterType="Required", direction="Input") param3.value = "CASE_ID" # Fifth parameter param4 = arcpy.Parameter( displayName="The created unique Case ID numbers form this Primary/Secondary relationship:", name="case_key_combo_type", datatype="GPString", parameterType="Required", direction="Input") param4.filter.type = "valueList" param4.filter.list = ["Left Join","Full Join","Inner Join"] param4.value = "Left Join" newField = arcpy.Field() newField.name = param3.value newField.type = "LONG" newField.precision = 10 newField.aliasName = param3.value newField.isNullable = "NULLABLE" # Sixth parameter param5 = arcpy.Parameter( displayName="Output Primary Table", name="out_prim_table", datatype="DETable", parameterType="Derived", direction="Output") param5.parameterDependencies = [param0.name] param5.schema.clone = True param5.schema.additionalFields = [newField] # Seventh parameter param6 = arcpy.Parameter( displayName="Output Secondary Table", name="out_sec_table", datatype="DETable", parameterType="Derived", direction="Output") param6.parameterDependencies = [param1.name] param6.schema.clone = True param6.schema.additionalFields = [newField] # Eighth parameter param7 = arcpy.Parameter( displayName="String comparisons are Case:", name="case_sensitive_combo_type", datatype="GPString", parameterType="Required", direction="Input") param7.filter.type = "valueList" param7.filter.list = ["Insensitive","Sensitive"] param7.value = "Insensitive" # Ninth parameter param8 = arcpy.Parameter( displayName="String ends trimmed of whitespace:", name="whitespace_combo_type", datatype="GPString", parameterType="Required", direction="Input") param8.filter.type = "valueList" param8.filter.list = ["Both", "Left", "Right", "None"] param8.value = "Both" params = [param0, param1, param2, param3, param4, param5, param6, param7, param8] return params def isLicensed(self): """Set whether tool is licensed to execute.""" return True def updateParameters(self, parameters): """Modify the values and properties of parameters before internal validation is performed. This method is called whenever a parameter has been changed.""" if parameters[0].value: # Return primary table tbl = parameters[0].value desc = arcpy.Describe(tbl) fields = desc.fields l=[] for f in fields: if f.type in ["String", "Text", "Short", "Long", "Float", "Single", "Double", "Integer","OID", "GUID"]: l.append(f.name) parameters[2].filters[0].list = l if parameters[1].value: # Return secondary table tbl = parameters[1].value desc = arcpy.Describe(tbl) fields = desc.fields l=[] for f in fields: if f.type in ["String", "Text", "Short", "Long", "Float", "Single", "Double", "Integer","OID", "GUID"]: l.append(f.name) parameters[2].filters[1].list = l if parameters[2].value != None: mylist = parameters[2].value for i, e in list(enumerate(mylist)): if mylist[i][2] != "Descending": mylist[i][2] = "Ascending" parameters[2].value = mylist return def updateMessages(self, parameters): """Modify the messages created by internal validation for each tool parameter. This method is called after internal validation.""" if parameters[3].value and parameters[0].value and parameters[1].value: desc = arcpy.Describe(parameters[0].value) fields = desc.fields in_primary = False is_primary_error = False is_primary_uneditable = False for f in fields: if parameters[3].value.upper() == f.name.upper(): in_primary = True if f.type != "Integer": is_primary_error = True elif not f.editable: is_primary_uneditable = False desc2 = arcpy.Describe(parameters[1].value) fields2 = desc2.fields in_secondary = False is_secondary_error = False is_secondary_uneditable = False for f2 in fields2: if parameters[3].value.upper() == f2.name.upper(): in_secondary = True if f2.type != "Integer": is_secondary_error = True elif not f2.editable: is_secondary_uneditable = False newField = arcpy.Field() newField.name = parameters[3].value newField.type = "LONG" newField.precision = 10 newField.aliasName = parameters[3].value newField.isNullable = "NULLABLE" fields1 = [] fields2 = [] order = [] for item in parameters[2].value: fields1.append(item[0].upper()) fields2.append(item[1].upper()) order.append(item[2]) if str(parameters[0].value).upper() == str(parameters[1].value).upper(): parameters[1].setErrorMessage("The Input Secondary Table {0} cannot be the same as the Input Primary Table {1} ".format(parameters[1].value, parameters[0].value)) else: parameters[1].clearMessage() if in_primary and in_secondary: if is_primary_error and is_secondary_error: parameters[3].setErrorMessage("{0} exists and is not a Long Integer field in both the Input Primary and Secondary Tables".format(parameters[3].value.upper())) elif is_primary_error: parameters[3].setErrorMessage("{0} exists and is not a Long Integer field in the Input Primary Table".format(parameters[3].value.upper())) elif is_secondary_error: parameters[3].setErrorMessage("{0} exists and is not a Long Integer field in the Input Secondary Table".format(parameters[3].value.upper())) elif parameters[3].value.upper() in fields1 and parameters[3].value.upper() in fields2: parameters[3].setErrorMessage("{0} is used as a Case Field for both the Input Primary and Secondary Tables".format(parameters[3].value.upper())) elif parameters[3].value.upper() in fields1: parameters[3].setErrorMessage("{0} is used as a Case Field for the Input Primary Table".format(parameters[3].value.upper())) elif parameters[3].value.upper() in fields2: parameters[3].setErrorMessage("{0} is used as a Case Field for the Input Secondary Table".format(parameters[3].value.upper())) elif is_primary_uneditable and is_secondary_uneditable: parameters[3].setErrorMessage("{0} exists and is not editable in both the Input Primary and Secondary Tables".format(parameters[3].value.upper())) elif is_primary_uneditable: parameters[3].setErrorMessage("{0} exists and is not editable in the Input Primary Table".format(parameters[3].value.upper())) elif is_secondary_uneditable: parameters[3].setErrorMessage("{0} exists and is not editable in the Input Secondary Table".format(parameters[3].value.upper())) else: parameters[3].setWarningMessage("{0} will be overwritten in both the Input Primary and Secondary Tables".format(parameters[3].value.upper())) elif in_primary: parameters[6].schema.additionalFields = [newField] if is_primary_error: parameters[3].setErrorMessage("{0} exists and is not a Long Integer field in the Input Primary Table".format(parameters[3].value.upper())) elif is_primary_uneditable: parameters[3].setErrorMessage("{0} exists and is not editable in the Input Primary Table".format(parameters[3].value.upper())) else: parameters[3].setWarningMessage("{0} will be overwritten in the Input Primary Table".format(parameters[3].value.upper())) elif in_secondary: parameters[5].schema.additionalFields = [newField] if is_secondary_error: parameters[3].setErrorMessage("{0} exists and is not a Long Integer field in the Input Secondary Table".format(parameters[3].value.upper())) elif is_secondary_uneditable: parameters[3].setErrorMessage("{0} exists and is not editable in the Input Secondary Table".format(parameters[3].value.upper())) else: parameters[3].setWarningMessage("{0} will be overwritten in the Input Secondary Table".format(parameters[3].value.upper())) else: parameters[5].schema.additionalFields = [newField] parameters[6].schema.additionalFields = [newField] parameters[3].clearMessage() return def stringCaseTrim(self, parameters, value): tempstr = None if parameters[7].value.upper() == 'Sensitive'.upper(): tempstr = value else: tempstr = value.upper() if parameters[8].value.upper() == 'None'.upper(): return tempstr if parameters[8].value.upper() == 'Left'.upper(): return tempstr.lstrip() if parameters[8].value.upper() == 'Right'.upper(): return tempstr.rstrip() else: return tempstr.strip() def execute(self, parameters, messages): """The source code of the tool.""" try: desc = arcpy.Describe(parameters[0].value) fields = desc.fields in_primary = False for f in fields: if parameters[3].value.upper() == f.name.upper(): in_primary = True if not in_primary: arcpy.AddField_management(parameters[0].value, parameters[3].value.upper(), "Long", 10) arcpy.AddMessage("Added a Case ID field to the Input Primary Table") desc2 = arcpy.Describe(parameters[1].value) fields2 = desc2.fields in_secondary = False for f2 in fields2: if parameters[3].value.upper() == f2.name.upper(): in_secondary = True if not in_secondary: arcpy.AddField_management(parameters[1].value, parameters[3].value.upper(), "Long", 10) arcpy.AddMessage("Added a Case ID field to the Input Secondary Table") tbl1 = parameters[0].value tbl2 = parameters[1].value fields1 = [] fields2 = [] order = [] for item in parameters[2].value: fields1.append(item[0]) fields2.append(item[1]) order.append(item[2]) arcpy.AddMessage("Primary Case Fields are {0}".format(str(fields1))) arcpy.AddMessage("Secondary Case Fields are {0}".format(str(fields2))) arcpy.AddMessage("Sort Orders are {0}".format(str(order))) import itertools k = [] arcpy.AddMessage("Strings Comparisons Are {0}".format(parameters[7].value)) k = list(tuple([self.stringCaseTrim(parameters, value) if str(type(value)) in ("<class 'str'>", "<type 'unicode'>") else value for value in r]) for r in arcpy.da.SearchCursor(tbl1, fields1)) arcpy.AddMessage("Case Values have been read from the Input Primary Table") if parameters[4].value == "Full Join": j = [] j = list(tuple([self.stringCaseTrim(parameters, value) if str(type(value)) in ("<class 'str'>", "<type 'unicode'>") else value for value in r]) for r in arcpy.da.SearchCursor(tbl2, fields2)) k = k + j j = None arcpy.AddMessage("Case Values have been appended from the Input Secondary Table") from operator import itemgetter for i, e in reversed(list(enumerate(order))): if order[i] == "Descending": k.sort(key=itemgetter(i), reverse=True) else: k.sort(key=itemgetter(i)) k = list(k for k,_ in itertools.groupby(k)) if parameters[4].value == "Inner Join": j = list(tuple([self.stringCaseTrim(parameters, value) if str(type(value)) in ("<class 'str'>", "<type 'unicode'>") else value for value in r]) for r in arcpy.da.SearchCursor(tbl2, fields2)) arcpy.AddMessage("Case Values have been read from the Input Secondary Table") l = [] for item in k: if tuple(item) in j: l.append(item) j = None k = l l = None arcpy.AddMessage("Case Values have been matched to the Input Secondary Table") arcpy.AddMessage("A list of sorted and unique Case Values has been created") dict = {} fields1.append(parameters[3].value) fields2.append(parameters[3].value) for i in xrange(len(k)): dict[tuple(k[i])] = i + 1 k = None arcpy.AddMessage("A dictionary of unique Case Value keys with Case ID number values has been created") with arcpy.da.UpdateCursor(tbl1, fields1) as cursor: for row in cursor: caseinsensitive = tuple([self.stringCaseTrim(parameters, value) if str(type(value)) in ("<class 'str'>", "<type 'unicode'>") else value for value in row[0:len(fields2)-1]]) if caseinsensitive in dict: row[len(fields1)-1] = dict[caseinsensitive] else: row[len(fields2)-1] = -1 cursor.updateRow(row) del cursor arcpy.AddMessage("{0} values have been updated for Input Primary Table".format(parameters[3].value)) with arcpy.da.UpdateCursor(tbl2, fields2) as cursor2: for row2 in cursor2: caseinsensitive = tuple([self.stringCaseTrim(parameters, value) if str(type(value)) in ("<class 'str'>", "<type 'unicode'>") else value for value in row2[0:len(fields2)-1]]) if caseinsensitive in dict: row2[len(fields2)-1] = dict[caseinsensitive] else: row2[len(fields2)-1] = -1 cursor2.updateRow(row2) del cursor2 arcpy.AddMessage("{0} values have been updated for Input Secondary Table".format(parameters[3].value)) except Exception as e: messages.addErrorMessage(e.message) return class InsertSelectedFeaturesOrRows(object): def __init__(self): """Define the tool (tool name is the name of the class).""" self.label = "Insert Selected Features or Rows" self.description = "" self.canRunInBackground = False def getParameterInfo(self): """Define parameter definitions""" # First parameter param0 = arcpy.Parameter( displayName="Source Layer or Table View", name="source_layer_or_table_view", datatype="GPTableView", parameterType="Required", direction="Input") # Second parameter param1 = arcpy.Parameter( displayName="Target Layer or Table View", name="target_layer_or_table_view", datatype="GPTableView", parameterType="Required", direction="Input") # Third parameter param2 = arcpy.Parameter( displayName="Number of Copies to Insert", name="number_of_copies_to_insert", datatype="GPLong", parameterType="Required", direction="Input") param2.value = 1 # Fourth parameter param3 = arcpy.Parameter( displayName="Derived Layer or Table View", name="derived_table", datatype="GPTableView", parameterType="Derived", direction="Output") param3.parameterDependencies = [param1.name] param3.schema.clone = True params = [param0, param1, param2, param3] return params def isLicensed(self): """Set whether tool is licensed to execute.""" return True def updateParameters(self, parameters): """Modify the values and properties of parameters before internal validation is performed. This method is called whenever a parameter has been changed.""" return def updateMessages(self, parameters): """Modify the messages created by internal validation for each tool parameter. This method is called after internal validation.""" if parameters[1].value: insertFC = parameters[1].value strInsertFC = str(insertFC) if parameters[0].value and '<geoprocessing Layer object' in strInsertFC: FC = parameters[0].value strFC = str(FC) if not '<geoprocessing Layer object' in strFC: print("Input FC must be a layer if output is a layer") parameters[0].setErrorMessage("Input must be a feature layer if the Output is a feature layer!") else: dscFCLyr = arcpy.Describe(FC) dscinsertFCLyr = arcpy.Describe(insertFC) # add the SHAPE@ field if the shapetypes match if dscFCLyr.featureclass.shapetype != dscinsertFCLyr.featureclass.shapetype: print("Input and Output have different geometry types! Geometry must match!") parameters[0].setErrorMessage("Input and Output do not have the same geometry") if dscFCLyr.featureclass.spatialReference.name != dscinsertFCLyr.featureclass.spatialReference.name: print("Input and Output have different Spatial References! Spatial References must match!") parameters[0].setErrorMessage("Input and Output do not have the same Spatial References! Spatial References must match!") if parameters[2].value <= 0: parameters[2].setErrorMessage("The Number of Row Copies must be 1 or greater") return def execute(self, parameters, messages): """The source code of the tool.""" try: mxd = arcpy.mapping.MapDocument(r"CURRENT") df = arcpy.mapping.ListDataFrames(mxd)[0] FC = parameters[0].value insertFC = parameters[1].value strFC = str(FC) strInsertFC = str(insertFC) FCLyr = None insertFCLyr = None for lyr in arcpy.mapping.ListLayers(mxd, "", df): # Try to match to Layer if '<geoprocessing Layer object' in strFC: if lyr.name.upper() == FC.name.upper(): FCLyr = lyr if '<geoprocessing Layer object' in strInsertFC: if lyr.name.upper() == insertFC.name.upper(): insertFCLyr = lyr if FCLyr == None or insertFCLyr == None: # Try to match to table if no layer found if FCLyr == None: tables = arcpy.mapping.ListTableViews(mxd, "", df) for table in tables: if table.name.upper() == strFC.upper(): FCLyr = table break if insertFCLyr == None: tables = arcpy.mapping.ListTableViews(mxd, "", df) for table in tables: if table.name.upper() == strInsertFC.upper(): insertFCLyr = table break # If both layers/tables are found then process fields and insert cursor if FCLyr != None and insertFCLyr != None: dsc = arcpy.Describe(FCLyr) selection_set = dsc.FIDSet # only process layers/tables if there is a selection in the FCLyr if len(selection_set) > 0: print("{} has {} {}{} selected".format(FCLyr.name, len(selection_set.split(';')), 'feature' if '<geoprocessing Layer object' in strInsertFC else 'table row', '' if len(selection_set.split(';')) == 1 else 's')) arcpy.AddMessage("{} has {} {}{} selected".format(FCLyr.name, len(selection_set.split(';')), 'feature' if '<geoprocessing Layer object' in strInsertFC else 'table row', '' if len(selection_set.split(';')) == 1 else 's')) FCfields = arcpy.ListFields(FCLyr) insertFCfields = arcpy.ListFields(insertFCLyr) # Create a field list of fields you want to manipulate and not just copy # All of these fields must be in the insertFC manualFields = [] matchedFields = [] for manualField in manualFields: matchedFields.append(manualField.upper()) for FCfield in FCfields: for insertFCfield in insertFCfields: if (FCfield.name.upper() == insertFCfield.name.upper() and FCfield.type == insertFCfield.type and FCfield.type <> 'Geometry' and insertFCfield.editable == True and not (FCfield.name.upper() in matchedFields)): matchedFields.append(FCfield.name) break elif (FCfield.type == 'Geometry' and FCfield.type == insertFCfield.type): matchedFields.append("SHAPE@") break elif insertFCfield.type == "OID": oid_name = insertFCfield.name if len(matchedFields) > 0: # Print the matched fields list print("The matched fields are: {}".format(matchedFields)) arcpy.AddMessage("The matched fields are: {}".format(matchedFields)) copies = parameters[2].value print("Making {} {} of each {}".format(copies, 'copy' if copies == 1 else 'copies', 'feature' if '<geoprocessing Layer object' in strInsertFC else 'table row')) arcpy.AddMessage("Making {} {} of each {}".format(copies, 'copy' if copies == 1 else 'copies', 'feature' if '<geoprocessing Layer object' in strInsertFC else 'table row')) oid_list = [] # arcpy.AddMessage(oid_name) dscInsert = arcpy.Describe(insertFCLyr) if '<geoprocessing Layer object' in strInsertFC: oid_name = arcpy.AddFieldDelimiters(dscInsert.dataElement, oid_name) else: oid_name = arcpy.AddFieldDelimiters(dscInsert, oid_name) rowInserter = arcpy.da.InsertCursor(insertFCLyr, matchedFields) print("The output workspace is {}".format(insertFCLyr.workspacePath)) arcpy.AddMessage("The output workspace is {}".format(insertFCLyr.workspacePath)) if '<geoprocessing Layer object' in strInsertFC: versioned = dscInsert.featureclass.isVersioned else: versioned = dscInsert.table.isVersioned if versioned: print("The output workspace is versioned") arcpy.AddMessage("The output workspace is versioned") with arcpy.da.Editor(insertFCLyr.workspacePath) as edit: with arcpy.da.SearchCursor(FCLyr, matchedFields) as rows: for row in rows: for i in range(copies): oid_list.append(rowInserter.insertRow(row)) else: print("The output workspace is not versioned") arcpy.AddMessage("The output workspace is not versioned") with arcpy.da.SearchCursor(FCLyr, matchedFields) as rows: for row in rows: for i in range(copies): oid_list.append(rowInserter.insertRow(row)) del row del rows del rowInserter if len(oid_list) == 1: whereclause = oid_name + ' = ' + str(oid_list[0]) elif len(oid_list) > 1: whereclause = oid_name + ' IN (' + ','.join(map(str, oid_list)) + ')' if len(oid_list) > 0: # arcpy.AddMessage(whereclause) # Switch feature selection arcpy.SelectLayerByAttribute_management(FCLyr, "CLEAR_SELECTION", "") arcpy.SelectLayerByAttribute_management(insertFCLyr, "NEW_SELECTION", whereclause) print("Successfully inserted {} {}{} into {}".format(len(oid_list), 'feature' if '<geoprocessing Layer object' in strInsertFC else 'table row', '' if len(selection_set.split(';')) == 1 else 's', insertFCLyr.name)) arcpy.AddMessage("Successfully inserted {} {}{} into {}".format(len(oid_list), 'feature' if '<geoprocessing Layer object' in strInsertFC else 'table row', '' if len(selection_set.split(';')) == 1 else 's', insertFCLyr.name)) else: print("Input and Output have no matching fields") arcpy.AddMessage("Input and Output have no matching fields") else: print("There are no features selected") arcpy.AddMessage("There are no features selected") # report if a layer/table cannot be found if FCLyr == None: print("There is no layer or table named '{}' in the map".format(FC)) arcpy.AddMessage("There is no layer or table named '" + FC + "'") if insertFCLyr == None: print("There is no layer or table named '{}' in the map".format(insertFC)) arcpy.AddMessage("There is no layer or table named '{}' in the map".format(insertFC)) arcpy.RefreshActiveView() return except Exception as e: # If an error occurred, print line number and error message import traceback, sys tb = sys.exc_info()[2] print("Line %i" % tb.tb_lineno) arcpy.AddMessage("Line %i" % tb.tb_lineno) print(e.message) arcpy.AddMessage(e.message)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

‎06-07-2015

I am gathering some of the posts and code I have written over the years that I regularly want to reference and reposting them under my Blog to hopefully make them easier for me to find. I generally have difficulty finding these posts, because they were responses to questions by other users and the original post title they can be found under often is poorly related to the post I wrote. Here is how to calculate the MMonotonicity of a feature. MMonotonicity describes the trend of the M values as you traverse the polyline's vertices. This is seen in the Identify Route Locations tool in the Measure Values section, but is more useful as a field value to allow you to select routes where measures do not continuously increase. Anything other than a 1 (Strictly Increasing) indicates a complex route design that may cause problems. The MMonotonicy values and their domain translations are: 1 = Strictly Increasing 2 = All Measures are Level 3 = Increasing with Levels 4 = Strictly Decreasing 5 = Increasing and Decreasing 6 = Decreasing with Levels 7 = Increasing and Decreasing with Levels 8 = All Measures are NaN 9 = Increasing with NaN 10 = Measures with Levels and NaN only 11 = Increasing with Levels and NaN 12 = Decreasing with NaN 13 = Increasing and Decreasing with NaN 14 = Decreasing with Levels and NaN 15 = Increasing and Decreasing with Levels and NaN Here is the calculation: Parser: Python Show Codeblock: Checked Pre-Logic Script Code: import numpy def MMonotonicity(feat): partnum = 0 # Count the number of points in the current multipart feature partcount = feat.partCount pntcount = 0 mmonotonicity = 0 lastM = -100000000 # Enter while loop for each part in the feature (if a singlepart feature # this will occur only once) # while partnum < partcount: part = feat.getPart(partnum) pnt = part.next() # Enter while loop for each vertex # while pnt: pntcount += 1 if lastM < pnt.M and lastM != -100000000: mmonotonicity = mmonotonicity | 1 if lastM == pnt.M and lastM != -100000000: mmonotonicity = mmonotonicity | 2 if lastM > pnt.M and lastM != -100000000: mmonotonicity = mmonotonicity | 4 if numpy.isnan(pnt.M): mmonotonicity = mmonotonicity | 8 lastM = pnt.M pnt = part.next() # If pnt is null, either the part is finished or there is an # interior ring # if not pnt: pnt = part.next() partnum += 1 return mmonotonicity Expression: MMonotonicity( !Shape!)

‎06-07-2015

Python will not only modify the Unique Values listed in the layer, it will reset the symbology of every unique value in the list to random simple colors. Python mapping is very limited in what it can do with symbols. Therefore the picture you showed will never retain the symbol colors you showed if you use Python. You would have to use .Net ArcObjects to maintain the rest of the symbols in the layer after removing one of the values from the list. I am afraid you are wasting your time with Python if you require the symbols for the other Unique Values to remain unchanged.

‎06-06-2015

The address attributes are all being filled in using reverse geocoding and getting a single correct position on the Centerline is essential to that operation. It probably doesn't use any buffering option and requires snapping so that it never uses the wrong street for the address or the wrong position on the centerline. As a result the tool will never derive more than one address for any point you place (unless a user intentionally clicks on an intersection point for multiple roads). There is no way that the tool would ever recognize the lot line geometry to derive correct normal angles relative to the actual lot you are targeting without a complete rewrite of the tool. The tool actually doesn't register that there is any lot line feature class at all. Using a simple tolerance or buffer around a point on a lot line is also a bad approach, since it will often return too many possible roads and address numbers (especially for corner lots and curved roads) or the tolerance will be too small and miss too many centerlines as road widths vary. A cross street of a local road with an Arterial Highway would typically assign the address to the local road for any driveways near that corner, even if the driveway orientation was toward the Arterial Highway, since the local centerline will be closer to the lot line for some distance along both lot frontages. Having done a lot of geometry programming the current centerline snapping option is exponentially easier to implement than any option that would allow for buffering or the use of actual lot lines to determine the normal angle to the closest centerline position. So I fully understand why the programmer did not even attempt to handle the complexities that arise from a tool designed the way you want. For these reasons, I doubt there are any ways to alter the behavior of the existing tool.

‎06-05-2015

At 10.0 and above you have to use Python to do geometry calculations with the Field Calculator. The calculation to extract the Start point coordinates is: !Shape.FirstPoint.X! (or Y, Z, or M) The End point coordinates is similar: !Shape.LastPoint.X! (or Y, Z or M) See my responses in this thread for more geometry options.

‎06-04-2015

You are correct that the UpdateCursor would be replaced by the w.writerow code, but that has nothing to do with the look up process. Anyway, dictionaries need to build the list object that you later feed to the w.writerow portion of your code. Additionally using a key that looks up the next key in the dicitionary works perfectly well and embedding cursors that point to the same data source gives even worse performance that embedded cursors accessing separate datasources. Here is a stab at converting the code. Without knowing your data I assumed the Sequence field is a string field and that Sequence is a Unique key without duplicates in the data source. If Sequence is not a Unique key and can refer to multiple rows you will need to let me know so I can implement it as a list of lists value. I assumed a one to many relationship for the depot visits. Also, I am unfamiliar with what expression2 creates and how it is used in the cursor expression: expression2 = arcpy.AddFieldDelimiters("depotVisits", "VisitType")+ qLast for row3 in arcpy.da.SearchCursor("depotVisits", ("FromPrevTravelTime", "FromPrevDistance"), expression2): wouldn't that convert to this invalid invalid cursor set up (ignoring whatever delimiters apply)? for row3 in arcpy.da.SearchCursor("depotVisits", ("FromPrevTravelTime", "FromPrevDistance"), "depotVisits", "VisitType"=2): So I am ignoring the depotVisits and assuming it works with VisitType=2 until you explain it to me. The code is untested, so expect some debugging effort to make sure everything is indented correctly. I free pasted and then modified a fair amount of the code from multiple sources so I could have left some mismatched syntax in the code. Let me know of any errors that are not obvious omissions or misspellings. The code below should replace lines 137 through 198 in your original code posting. The relative indentation of the code should make line 1 below indented at the same level as line 137 in your original code. sourceFC = None #select corresponding route layer file and grab sublayer: DepotVisits print "extracting time & distance to Depot" routeLyr = arcpy.mapping.Layer(routeWkspc + "TrapRoute_" + blockEID + "_" + scaleEID + ".lyr") if routeLyr.isGroupLayer: for sublyrs in routeLyr: # print sublyrs.name if sublyrs.name == 'Depot Visits': arcpy.MakeFeatureLayer_management(sublyrs, "depotVisits") sourceFC = "depotVisits" depotVisitsDict = {} if sourceFC == "depotVisits": sourceFieldsList = ["VisitType", "FromPrevTravelTime", "FromPrevDistance"] # Set up a One to Many Dictionary with arcpy.da.SearchCursor(sourceFC, sourceFieldsList) as depots: for depot in depots: relatekey = depot[0] if relatekey in depotVisitsDict: depotVisitsDict[relatekey].append(depot[1:]) else: depotVisitsDict[relatekey] = [depot[1:]] sourceFC = eLyrPath sourceFieldsList = ["Sequence", "FromPrevTr", "FromPrevDi", "POINT_X", "POINT_Y", "BUFF_DIST"] # Use list comprehension to build a dictionary from a da SearchCursor sequenceDict = {r[0]:(r[1:]) for r in arcpy.da.SearchCursor(sourceFC, sourceFieldsList)} for key in sequenceDict: dataList = [] dataList.append(str(blockEID)) dataList.append(str(scaleEID)) dataList.append(str(bufEID)) dataList.append(str(key)) dataList.append(str(list(sequenceDict[key])[0])) dataList.append(str(list(sequenceDict[key])[1])) dataList.append(str(list(sequenceDict[key])[2])) dataList.append(str(list(sequenceDict[key])[3])) dataList.append(str(list(sequenceDict[key])[4])) n = int(key) sN = "=" + str(n) n2 = n +1 sel = "=" + str(n2) prevTime = float(list(sequenceDict[key])[0]) prevDist = float(list(sequenceDict[key])[1]) # print sN postTime = "" # use a nested search to get values for total time and total distance # there are some cases where the below expression will be false in that # event we will need to cacluclate the distance to the depot or just leave # the values for prevTime & prevDist as place holders, otherwise I end up with # road length being written in the wrong field if str(n2) in sequenceDict: values = list(sequenceDict[str(n2)]) for row2 in values: postTime = row2[0] totTime = prevTime + float(postTime) # print totTime dataList.append(str(totTime)) postDist = row2[1] # print postDist totDist = prevDist + float(postDist) # print totDist dataList.append(str(totDist)) # if above for loop yeilds no results, calculate postTime and postDist differently # grab DepotVisits layer - if VisitType = "End" then grab values for "FromPrevTravelTime" & "FromPrevDistance" if postTime == "": #select corresponding route layer file and grab sublayer: DepotVisits print "extracting time & distance to Depot" if 2 in depotVisitsDict: for row3 in depotVisitsDict[2]: postTime2 = row3[0] totTime2 = prevTime + float(postTime2) # print ("Depot Visit Time: {0}, {1}, {2}".format(prevTime, postTime2, totTime2)) dataList.append(str(totTime2)) postDist2 = row3[1] totDist2 = prevDist + float(postDist2) # print ("Depot Visit Distance: {0}, {1}, {2}".format(prevDist, postDist2, totDist2)) dataList.append(str(totDist2))

‎06-04-2015

You cannot choose fields that are different fields types between the two tables. So for example, you cannot join a String field in one table to a Numeric Field in another table. Numeric fields may only join to Numeric fields, String fields may only join to String Fields and Date Fields may only join to Date fields. When working in the Desktop with the menu items I mentioned, the dialog only presents fields in the Join table that are the appropriate type to match the field you chose from the Target layer. In Model Builder I assume there are ways to bypass the safeguards that a Desktop session provides and put in two fields to the Join tool that cannot be joined together. Make sure the fields in both tables are the same type. If they are not you can create a new text field in both tables (or in just one table if the other already has it as a Text field) and calculate the data into that new field (all data can convert to Text). In my original post the user would have created a Summary output with the Name field as a Case field and a Summary of MAX for the Acres field. The Join would be from the Name field in the original table to the Name field in the Summary Statistics output.

‎06-04-2015

Use a standard ArcMap join on a field. There are several ways to set it up. For layers I often right click the layer in the Table of Contents, choose Joins and Relates, and choose Join to bring up the Join dialog. In the Join dialog I choose the field from my layer that is related to another table, then I choose that other table and the field in that other table that matches. Then I press OK and the fields from he Join table appear in the table view of my layer. When you are done with the join you right click the layer, choose Joins and Relates, choose Remove Join(s), and choose Remove All. From within a Table View a Join can be set up by expanding the Table Options button (ArcMap 10.0 and above), and then using the same menu options that are available from the Layers context menu. The Geoprocessing tool equivalent is the Join tool and the Remove Join tool (for when you are done with the Join).

‎06-04-2015

The Nested Cursors are absolutely without doubt the source of the problem. In memory processing with dictionaries instead of iterative SQL statements will improve the speed 100 to 1000 fold with a two level nested cursor. The slow speed is due to all of the queries you are doing, especially if the query fields are not indexed. The dictionary method will perform extremely fast regardless of whether fields have an index or not. I pursued my solution for years to avoid the speed bottleneck you are experiencing with nested cursors and have no doubt that avoiding SQL and using in memory dictionaries for all record matching will make this process complete in a matter of minutes. Dictionaries do random access matching, while SQL queries are linear against the entire table. Therefore, even with an in memory table the SQL will never perform as fast as a dictionary. The problem grows exponentially worse as the records in the two (or three) sources increase with the nested cursor approach. The speed loss with dictionaries is hardly impacted at all by even doubling all of the records in all of the data sources. Read this article to discover why dictionaries are so much better than linear processing collections (which cursors and SQL operations on tables basically are) for lookup/record matching processes. Dictionaries are fast because: "The insert/delete/lookup time of items in the dictionary is amortized constant time - O(1) - which means no matter how big the dictionary gets, the time it takes to find something remains relatively constant."

Online Status	Offline
Date Last Visited	3 weeks ago

My Ideas

Latest Contributions by RichardFairhurst

Re: Script running slow, how to speed up?

Re: “Drag\drop feature class in ArcGIS Pro,

Re: Script running slow, how to speed up?

Re: Script running slow, how to speed up?

Re: Script running slow, how to speed up?

Re: Script running slow, how to speed up?

Multiple Field Key to Single Field Key Tool - Relate Two Layers Based on More than One Field

MMonotoniticy Calculation - How to Tell if Your Route is Simple or Complex

Re: How to remove a specific unique value with arcpy

Re: Offset Address Points From Centerline - Address Data Management Template

Re: Start and End Points of a line (Field Calculator)

Re: Script running slow, how to speed up?

Re: Select field value A based on largest polygon with field value B

Re: Select field value A based on largest polygon with field value B

Re: Script running slow, how to speed up?

Add Dynamic, Epoch-Aware Spatial Reference Support

Add a Bearing WKID System Parallel to Spatial Refe...

What Does ArcGIS Pro 4.0 Really Need To Be For Dev...

Re: Turbo Charging Data Manipulation with Python C...

Re: Turbo Charging Data Manipulation with Python C...

Addressing