Latest Contributions by RichardFairhurst

What license level do you have?  An Advanced license gives you the most flexibility.  However, it is virtually certain that a custom script will be involved that employs cursors and dictionaries to do the analysis to find the closest vertex to the intersection and write a buffer distance field to the intersection point. Do the lines that make up the line network connecting to the intersection points always have an end that is identical to the intersection point, or do the lines go through the intersection points? ... View more

I completely agree with Joshua that only real Null values should be found by Python, since only real Null values will be found by SQL if I write an expression that uses IS NULL for any of the databases I use.  Whitespace and control characters are not Null and will not be selected using IS NULL with standard implementations of SQL.  Python follows the SQL standard.  Access queries might return results that are more similar to the VB implementation, but Access is in no way a reliable reference for established database "standards" recognized by professionals. ... View more

I use Attribute primarily Assistant for editing road center lines.  Methods I use include COPY_LINKED_RECORD to transfer related street name components from the Master Street Name table, EXPRESSION to update a variety of fields in response to user edits, INTERSECTING_FEATURE for Zip Code, City, and map grid info, LAST_VALUE, VALIDATE_ATTRIBUTE_LOOKUP for the Master Street Name Table, and X_COORDINATE and Y_COORDINATE for the line end coordinate data. For the Master Street Name table I use CASCADE_ATTRIBUTE, COPY_LINKED_RECORD, EXPRESSION, GENERATE_ID, and LAST_VALUE to concatenate name components, provide upper and lower case forms of each name, and fully spell out street types for recorded names. I also have used the GENERATE_ID method for several other feature classes. ... View more

Rebecca: It appears you have created code that uses embedded cursors.  The code must be rewritten to eliminate that configuration if you want to make it process efficiently, since embedded cursors are the absolute worst way to approach any problem.  Instead one of the cursors should load the data to a dictionary for any look up of data and only one update cursor should be processed at a time.  The code revision can result in a 500 fold improvement in speed if you are processing 10,000 records against 10,000 records, since my approach would only have to process 20,000 record reads while an embedded cursor would process up to 10,000,000 record reads.. ... View more

I do not base many processes on the order of ObjectIDs, but everything I see says that Order By on any other column is slower, which probably has more to do with the fact the ObjectID is always the first column in a geodatabase than any internal storage of the records relative to each other.  It is guaranteed that a sort on the 40th column in a database will be much slower when read from disk using Order By than an Order By on the ObjectID in the first column.  Most sorted processing I do is through in memory processing of lists or dictionaries, or through Summary Statistics outputs that position the sorted fields at the beginning of the schema.  Order By also should never be done on a column that is not indexed. In any event, order can play an important part of any process (drawing order, analysis, etc), and if I reorganize a database using the Sort tool or create a Summary Statistics table to make the ObjectID column reflect that order it absolutely does benefit performance of tasks that use that order.  Again this may be more because the ObjectID is always the first column in the schema, it is always indexed for quick access, and it has a known field type all of which optimize it for the quickest access and processing.  Constantly processing an SQL Order By on a random field column in the database that the user may have failed to index that has to repeatedly determine its field position and value type each time you want to run a sort algorithm is bound to be slower.  Column position in a database is not random and the first column is the fastest column to access. The documentation makes this a black box, so all we can argue is whether or not empirically it is useless to get an ordered ObjectID or not.  I say it is useful where quick access to ordered records is essential based on everything I have experienced, whether or not I can explain why that is occurring to your satisfaction.  I care far more about giving advice on the practical effect of doing or not something than I do about the actual internal mechanism used to achieve that effect in a Black Box that I cannot access.  So if my guesses on the Black Box are wrong, but the practical outcome is real nonetheless, my advice on the forum will always be of more value for the second than the first, and I am satisfied with that.  I also do not advise anyone to constantly recreate their feature class every time they want to insert a single record into the middle of the ObjectID order, since that is very inefficient, but periodically optimizing for order can be useful if you don't rely on the ObjectID representing the same record each time you access the database.  I actually don't sort most of my data, because in most cases the value of having an ObjectID in my master database consistently return the same record every time I access it is more valuable to me than having it represent a constantly changing order. Also the design of the ObjectID is primarily intended to support consistent access to the same record in a feature class, although that is only reliable if a feature class that has never been moved or recreated. ... View more

I only edit File geodatabase with cursors for performance reasons and download SDE data to that format for all geoprocessing.  The predictable order that the file geodatabase format provides only leads me to say that I have one more reason to continue doing what I do.  Under any circumstances, ObjectIDs do represent the order that records are created in a geodatabase and cannot be reused once issued or deleted without risking database corruption, and no other order is represented or maintained within the data automatically in ArcGIS without a custom written program.  When working with personal geodatbases I have seen the effect when a user corrupted the order and uniqueness of the ObjectID field through an Access update query run outside of ArcGIS, but nothing works correctly in ArcGIS if that happens. ... View more

The only documentation about the FID/ObjectID I have seen is here.  It is not a technical discussion of the implementation of the ObjectID, nor is it an exhaustive discussion of the implications of the ObjectID for a programmer, and it in no way contradicts the behavior I am reporting that I have repeatedly observed for a Search Cursor.  Few things are fully documented for programmers about the internal mechanisms Esri has implemented, and most everything that goes beyond the exposed interfaces that may have critical implications for the programmer has to be deduced through experimentation and testing and only gets documented through the forum. I do not have a document reference for this behavior.  However, having used Search Cursors for years on a daily basis it has been empirically established to a degree that I am certain that the FID/ObjectID is the established default order of the Search Cursor.  Clues that Esri databases with FID/ObjectIDs actively maintain some kind of internal order involving the FID/ObjectID for performance include the fact that FID/ObjectID fields are always automatically indexed by the system.  Search Cursor record counting is accurate and predictable and always ordered by the FID/ObjectID when used without any Order By parameter by the user.  FID/ObjectID lists returned by methods that access record Selection Sets always list the FID/ObjectID values in order. I will acknowledge that Update Cursors have sometimes shown erratic behavior as far as reading records, especially if combined with a parallel Insert Cursor operation.  Update Cursors have been known to read the same record two or more times during a single pass of the cursor, usually reading the same record again immediately after an updateRow operation during consecutive for loop passes, but this is a bug as far as I am concerned and is a behavior that has never been documented by Esri or explained.  That Update Cursor behavior is a source of complaint on the forum since it screws up all record counting procedures with an Update Cursor and no one has been able to make sense of it. ... View more

In ArcGIS databases designed by Esri, a concept of order and gaps is in a database has been created by the automatic creation and maintenance of the FID/ObjectID, which uniquely identifies each record at the time it is created.  New numbers are always appended to the end of the number order and any deleted record creates a gap in the FID/ObjectID numbers that cannot be filled, without transferring the data into a new table/FC.  Search cursors do in fact read the records in the FID/ObjectID order by default, which is the fastest order based on how the records are indexed and stored.  Cursors can be sorted, but all other record orders can only be read at the cost of a substantial performance hit when read directly from disk.  Most record sorting of tables initially is read in ObjectID order and is only resorted in memory through things like a tableview before being presented to the user to overcome this performance hit.  No other automatically maintained record numbering is associated with the Esri databases. ... View more

You should also change any search or update cursor to the da version, since they are 10 times faster than the old style Python cursors. Unfortunately the answer is no, you can't insert new records between existing records within an existing feature class.  ObjectIDs are issued consecutively and cannot be shifted and if deleted they cannot be reused.  The only safe way to get the records organized based on attributes is by using the Sort tool, which will create a new feature class.  This tool resets all ObjectIDs in the new feature class to fit the new order and removes any ObjectID gaps that were created by deleted records.  You then would replace the original FC with the new sorted FC. If you really insist on not creating a new feature class and are willing to put all of your data at risk, then after inserting every record, you could load everything to a sorted dictionary or list and then use an update cursor to overwrite every record, setting all editable fields with the data of the appropriate record that fits the order you want.  ObjectID gaps from deleted records would remain gaps after using this approach, but the records would be reordered without creating a new FC.  Aside from the risk to your data, if you have GlobalIDs or other non-editable fields that don't reset based on the changes you make then you probably don't want to do this, since there is no way their field values could be kept with the original record they were associated with. ... View more

1. Coordinates never get stored in English units, only in meters, so conversions happen. 2. The Z resolution you have chosen allows this level of movement to be considered the same as an exact value. 3. Z Resolution establishes a Z grid and the Z extent sets an upper and lower bound of possible coordinates, and sometimes this makes it impossible to make whole number coordinates.  Check and report these settings. 4.  The projection chosen for Z may apply some adjustments when combined with the chosen XY projection (although that is a black box to me so I am not sure how they may interact). 5. Base 10 to base 2 conversions. 6. That distance is less than 1/10th the width of an average human hair and is such a small amount of measure variance from your reported benchmarks that it clearly falls within the tolerances of standard Survey procedures. ... View more

I would use Linear Referencing to solve this problem.  After making each road into a route with measures, I would use the Locate Features Along Route tool for both the traffic volume and the animal sightings to create two separate point event tables.  Make sure the animal and volumes each have a unique ID field other than the ObjectID.  Also make sure the two tables use different field names for the measure parameter of the tool so that when they combine both with be preserved.  This will establish both sets of data relative to an identical road based measuring system and a common route reference that will be respected by the Linear Referencing tools (so an event does not jump from one road to another).  Since you have to use a radius to tie the offset points to the routes, use the match all routes option and not the closest route option.  Just throw out any routes that do not match the route associated with each sighting and volume (hopefully you can use a road name field associated with each data source to find the correct road to tie each event to). Next I would get each sighting tied to all of the volumes on the road.  This can be done by calculating the beginning and ending measure on each route (Python field calculation using either !Shape.FirstPoint.M! or !Shape.LastPoint.M!), joining that to the animal sightings and copying the table to a new table.  Now that each animal is associated to the measures of the full limit of its route and will ignore all other routes, you can use the Overlay Route Events tool to combine this animal table with the volume table matching the full route measures to the volume measure and outputting a point event while preserving all fields.  Because the AnimalSighting measure will be combined with the Volume measure now, a Separation field that holds the distance as measured along the road between the animal sighting and each volume point can be calculated using the VB Script formula Abs(Animal_Measure - Volume_Measure). Then run the Sort tool on the combined event table using the RouteID, AnimalSightingID and the Separation fields as the sort criteria.  Run the Summary Statistics tool in the Analysis toolbox, Statistics toolset to get the Min of the Object ID associate with each Route and AnimalSightingID value as the 2 case fields.  Run my tool that creates a single key from a multi field key.  Join on the new single field key and select SortedTable.ObjectID = SummaryTable.Min_ObjectID if you want the closest volume to the animal sighting or use SortedTable.ObjectID > SummaryTable.Min_ObjectID + 5 and SortedTable.AnimalSightingID = SummaryTable.AnimalSightingID if you want the top 5 closest volumes on that route.  Export the selection and you have an event table of all animal sightings combined with the closest volumes you chose, plus you will know the distance separating them along the route. You will be able to create several unique event layers by using different measures gathered at each step.  For example you can create line segments between the animal sighting and each volume point or a point for either the animal or the volume all using this one table.  If you preserved a side offset distance from the Locate Features Along Route tool used for the animal at the beginning you can offset the points or the line to the side of the line the animal was seen on using that field in the Make Route Event layer tool. All of the tools above work under a Basic license. ... View more

I believe the Editor has to be passed in along with the database and passed back from each function.  The operation fails because you are re-initializing the arcpy.da.Editor instance rather than passing it back and forth from method to method so you are only managing one instance.  By re-initializing, the previous operation that was started in one method becomes invisible to the next and forgotten.  A cursor behaves much the same.  You could not start a cursor in one method and then call the cursor's updateRow operation in another method if you were not passing the original cursor between them.  If you start a new cursor in the second method it knows nothing about the cursor started in the original method and the two cursors do not cooperate as one cursor, they conflict as two cursors on the same workspace.  So instead try: def openEdit(database, edit):      edit = arcpy.da.Editor(database)      edit.startEditing()      edit.startOperation()     return edit   def closeEdit(database, edit):      edit.stopOperation()      edit.stopEditing(save_changes=True):     return edit   # Then try running it like this:  db = database connection file edit = None edit = openEdit(db, edit)  with arcpy.da.UpdateCursor(fc, fields) as cursor:      for row in cursor:            for code in facilCodes:                if code == row[0]:                row[1] = "New Mexico Central"                cursor.updateRow(row)                count += 1                print("This information")  edit = closeEdit(db, edit) ... View more

It is good practice to only attempt one pass with an update cursor to completion, then after that use a new cursor to do any inserts.  You do not have to create a new field to work with the time stamp as an integer, and could instead do this within a python dictionary or list in memory, so that you would use a search cursor to load it all into memory, analyze it there, then use an insert cursor to create the new points.  That way no conflict between cursors would occur.  But without more info I cannot propose any code.  Post the code you are attempting so we have some idea of what you are actually trying. ... View more

The term "event loading" is not really one that I have heard used, but I am familiar with x/y events, geocoding, and linear referenced events.  I have seen demos where Excel tables are added to ArcGIS and it may have detected that it contained x/y fields or address fields and automated the creation of an XY Event Layer or a Geocoded event layer with little or no user input.  I don't really recall how that happened, since I don't use spreadsheets with ArcGIS very much. Every time I have added a file geodatabase table to a map I have had to either right click it and choose the Display XY Data option in the menu or run it through the Make XY Event Layer geoprocessing tool (in the Data Management Tools toolbox, Layers and Table Views toolset).  The same is true for events based on Linear Referenced Routes and measures, which have a Display Route Events option and a Make Route Event Layer geoprocessing tool (in the Linear Referencing Tools toolbox).  I have also always had to use the tools in the Geocoding Tools toolbox to set up and create geocoded address points.  So if you are having problems with the behavior I mentioned in the first paragraph, I can't help, but if you are having problems with using the tools I have mentioned in this paragraph, I can help. ... View more