Latest Contributions by RichardFairhurst

I have not found a good way to do this without assigning unique IDs to the undissolved polygons and creating a duplicate of all of those features during each dissolve cycle and comparing changes in the undissolved feature class to the last copy.  Then you can use the ID to do a full comparison of the feature set to detect the creation of new features, the deletion of old features, changes to the areas or perimeters of each undissolved feature indicating geometry changes, and changes to any attributes that would affect the dissolve.  Since features can overlap (bad topology, intentional overlaps) you cannot just add the areas of the undissolved feature class and compare it to the area of the dissolved features to decide if they are different or the same.  Any processing that involves geometry manipulation using cursors is too slow to do any other meaningful comparisons between undissolved and dissolved shapes.  You are better off just redissolving if that is your only option. ... View more

The Bearing Distance to Line tool can read a table to generate a line feature class from a table with x, y, distance and bearing fields.  This tool is available at all license levels.  You can then use the Feature Vertices to Point tool to create a point feature class of the end points of the lines if you have an Advanced license.  If you don't have an Advanced license you can calculate the line end point X and Y coordinates using the geometry calculator into a pair of fields and then export a table from the table view of the Lines and then use the Make XY Event Layer to generate a point layer from those fields and export the features from the layer in the Table of Contents to create a permanent point feature class. ... View more

The process you are trying to do is called conflation and the best tool for doing this is the Detect Feature Changes tool in the Data Comparison Toolset of the Data Management Toolbox.  This tool requires an Advanced license and was first available in ArcMap 10.2, so if you have that license and a version of ArcMap that supports it you should definitely use this tool, since there is nothing better.  The tool can compare line feature classes in all the ways you have described and more using tolerance settings that you specify and will compare any matching attributes that you want.  The tool can also create a separate table that shows all OID relationships between the two feature classes with all records needed to show all line to line matches for one-to-one, one-to-many, many-to-one, and many-to-many matches.  Once all relationships are created by the tool you can do standard selections based on the attributes and relates to the table so you can use the field calculator to update your lines to indicate whether or not they are matched.  No tool can do this kind of matching without some false positive and false negative matches, but this tool does everything needed to get the best results that will allow you to validate the results the quickest. ... View more

Chaim: Thanks for letting me know how this post helped you.  When you start loading a large number or records and fields to a dictionary, python can run out of memory.  I would expect running the code in an ArcMap session would be more likely to trigger the error than running it in Idle.  Having a lot of other applications open can sometimes contribute to the memory issue.  I also have found that if you have code that needs to load several large dictionaries that Python only releases the memory of a dictionary you no longer need if you loaded and processed the dictionary within a def method and not in the main program.  Once the def method completes and returns control to the main program the dictionary memory loaded by the method is released. ... View more

I want to add that in general I rarely abandon data at a fine grade level in favor of maintaining it only at an aggregated level, especially if my organization has access to or is responsible for authoritative data at the fine grade level.  Aggregation is a one way street.  You can easily and quickly generalize and aggregate from fine grade data anytime you want and achieve different levels of understandings of your data if it has a well defined relational structure, but it is impossible to recreate fine grade data directly from generalized or aggregated data once the relationships have been collapsed into a single simplified form. The trajectory of my organization is toward expanding our access to ever finer levels of detail in our authoritative data and improving the spatial representation of that data so that we can integrate it into ever more detailed analysis through on the fly or daily aggregation via scripts.. As a consequence in the last two years the data I work with has gone from a few thousand records into the millions.  However, by setting up a good relational design and learning how to exploit it through joins, relates and scripts, now that I'm past the initial design phase the amount of time I spend maintaining the data has dropped while at the same time the number of outputs I can derive from it keeps expanding. ... View more

You need to create a real data relationship, otherwise you won't ever understand how to solve your own problems.  Randy Burton and I can create scripts that can handle your request because we understand how data relationships work.  It is at the foundation of everything our scripts are doing.  So before you ask for a script that populates a Mutual field (and then continue to ask for more fields as your problem continues to evolve) create a new field for the key and foreign key relationship between your tables and find out how joins and relates apply to your problem. If you set up a standard data relationship you can manually do what the script  is doing and validate your work even better than any script will.  A script will not catch data entry errors as Blake has said, but joins and relates combined with your domain knowledge will expose them. I deal with million+ records to million+ record matching and updates daily through a script, but I would never think of completely abandoning a real data relationship that supports joins and relates for a purely script based solution.  I am currently validating 600,000 records covering 20 years of data entry using standard joins and relates.  In half an hour I was able to identify a class of outdated and/or erroneous references affecting 50,000 records and correct them with a simple field calculation using joins and relates.  Other errors require more research, but the joins and relates make it easy to identify and isolate the small number of affected records that require that research from the million+ records, because I understand the joins and relates I have established. There are too many complexities, human errors, and tracking issues with dynamic land data to rely only on a script, especially when human made boundaries are involved that are subject to continuous change due to subdivisions, property ownership transfers and political reasons.  I simply cannot stress the importance enough of learning the fundamentals of Joins and Relates as they apply to your problem.  ... View more

Fatin: You are failing to exploit any relational database capabilities with your set up.  Why not write the Community Name of FC2 into the Mutual field of FC1 instead of a random portion of it?  If the Mutual Field of FC1 has the same value as the Community Name field of FC2 you can use joins and relates, which would make your data much more useful than anything you have done so far.  Also the code has found the connection between the Community Name of FC1 and the Community Name of FC2 already and writing the Community Name of FC2 to the Mutual Field of FC1 is easily done and far more useful than the non-relational data you keep wanting to create.  If a data relationship exists between two tables it is far better to create a value in the tables that explicitly establishes a primary key and foreign key link than to hide it through an implicit relationship that is only visible through python code. ... View more

You missed the point of my code.  You have a dictionary with tuples, but only one tuple for only one record.  You need a list of tuples, with the list containing a separate tuple for each record in table two.  List comprehension only works for getting one of the records in a one to many key to record relationship, since each pass of a new record just replaces the last record. I know I am correct as I have written many scripts that do just this to handle one to many or many to many relationships.  I have no real understanding of how your code creates the InsertRow, but I know that the list comprehension dictionary is the reason that only record 5 from FC2 is written to your insert record. If 5 records in FC2 have the same key field values, the dictionary for FC2 needs to store this:  { (ID1, ID2) : [(FIELD0, FIELD1, FIELD2 FIELD3 ....) ,(FIELD0, FIELD1, FIELD2 FIELD3 ....) ,(FIELD0, FIELD1, FIELD2 FIELD3 ....) ,(FIELD0, FIELD1, FIELD2 FIELD3 ....) ,(FIELD0, FIELD1, FIELD2 FIELD3 ....) ]} The code I have suggested will store the 5 records of data from FC2 that all have the same key field values this way. ... View more

You cannot use list comprehension to populate your second dictionary when you need to store a list of records and not just a single record.  Something like the following may work, but then you need to walk through the list of tuples returned by the dictionary in your other code to make comparisons and to avoid duplication. sourceFC = "MyFC" sourceFieldsList = ["FIELD0","FIELD1","FIELD2","FIELD3"] # Build a summary dictionary from a da SearchCursor with unique key values storing a list of records.  valueDict = {}  with arcpy.da.SearchCursor(sourceFC, sourceFieldsList) as searchRows:      for searchRow in searchRows:          keyValue = searchRow[0]          if not keyValue in valueDict:              # assign a new keyValue entry to the dictionary storing a record in a list             valueDict[keyValue] = [searchRow[1:]]          else:             # append a record to the list if the key already exists in the dictionary             valueDict[keyValue].append(searchRow[1:])‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍ ... View more

The code behavior should have changed with that code adjustment.  Is it working now?  If not, what is it now doing that you don't want or not doing that you want? ... View more

I do not follow all of what the code is doing, but I cannot find a declaration of the rowStatus variable prior to line 40.  It seems that you always assign this variable on the first pass, but I would assign the rowStatus variable outside of your loop prior to line 32 to make sure that an unassigned variable error condition never occurs. The rowStatus variable does not seem to get reset in the loop, so once it is set to "Insert" it stays that way.  That should mean that every time lines 49 and 50 occur the condition is true, so every row should be treated as an Insert row and the variable is effectively doing the same thing every time.  You probably need to add a new line of code after lines 49 and 50 to reinitialize rowStatus, i.e.,             if rowStatus == "Insert"                 InsertRows_list.append(row)                 rowStatus = ""‍‍‍ ... View more

This Python expression won't work in the field calculator.  You need to declare it indented under a def method you have named and you need to have passed all of the field values in through that method.  Like VB Script, you can only set a field value by doing a calculation on a field and returning a value from your Pre-Logic Script code, not by using an expression internal to the calculation like: !Condition! = "1" So in Python it would be: Parser: Python Show Codeblock checked PreLogic Script Code: def setCondition(assetCondition):     if assetCondition == "1 - NEW":         return "1"     elif assetCondition ==  "2 - GOOD":         return "2"     elif assetCondition ==  "3 - POOR":         return "3"     elif assetCondition ==  "4 - UNSERVICEABLE":         return "4"     else:         return None‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍ Condition = setCondition(!AssetCondition!) ... View more

One thing I noticed in your latest illustration is that the first example point in FC2 has an incorrect value that violates the rules you have outlined.  It shows "x,y,z,x" in the picture and table, but was matched to FC1 as though it was actually "x,y,z,k".  These kinds of errors in the value you wrote vs the value you should have written could be detected and reported, but only if the first value always violates your rules. Second, for item 6 I want to be clear that I was not asking if the total number of characters could be greater than 2.  I was asking if the number of numeric digits at the end could be more than 0-9, For example, is EJ10 a possible value in field 2 of FC1, or would they always fall between EJ0-EJ9.  Are you saying that the numeric digits at the end can exceed 0-9? ... View more

You explanation of what is going on is helpful, but the details you are now giving raises some other questions. 1.  Are the real value in the Feature Name field going to be multi-letter actual community names?  For example would the actual values expressed as a,b for example in reality be real community names like Tulkarm,Bethlehem?  If so, will the community names sometimes consist of multiple words, like Los Angeles?  Knowing the true nature of the real data the program will be working with is crucial to writing the most efficient and error free code for parsing and concatenating your real data values.  There can be a vast difference in the coding approach I might use when the data only consists in single character values verses when the data consists in multi-character values. 2  Is there a significant potential for a lot of misspellings in the values in the Feature Name field in FC1 or in FC2 or both?  If the answer to question 1 is that the values are multi-character values that have specific spellings, there is a greater likelihood that a programmatic solution may not work for a large portion of your data.  Minor spelling variations or inconsistent use of abbreviations between the two FCs can render programs like this useless, just like they can render a standard Join or Relate useless.  Also, does the program have to account for possible variations in how community names are capitalized? 3.  How many data points exist in FC1 and how many points exist in FC2 in your real data?  Automated solutions may not be worth the effort if the data set is relatively small. 4.  Will this be a one time thing or will this process need to be regularly repeated because of changes in the number of points or value changes in the Ref field?  Automated solutions may not be worth it you only need to do this one time or extremely rarely. 5.  Are the abbreviations of the governorates in the REF field ever more than 2 characters?  What is the largest number of characters used as a governorate abbreviation?  Are the numbers at the end of every Ref value always just a single digit or can the numeric part of the REF ever be made up of 2 or more digits?  Knowing these things affects the parsing and concatenation processes I would use. 6.  Are the New REF values always limited to a single governorate and always limited to a single abbreviation prefix in the list?  If this is the rule, is there any chance that an invalid value exists that violates that rule by mistake? Bottom line is that understanding the true nature of the data and its quality, consistency and level of validation prior to running any code can be critical to the structure and design of a program and the level of error handling that has to be incorporated when you write a program that is going to have to do this much parsing and concatenating of data to match the data up correctly.  The violation of even one rule that is assumed can end up requiring the program to undergo substantial restructuring and can force the development of a variety of new subroutines, which can add a lot of debug time. ... View more