|
POST
|
Although you may have a reason for doing this with sql geometries as opposed to a geoprocessing approach, with a processing time of over 17 hours and only the need for daily updates you should at least consider a geoprocessing approach as an alternative. You can use the Feature to Point tool with the inside option to extract the centroids of the parcels and then the Spatial Join tool with the One to One option with the Municipality polygons as the target and the centroids as the join type. Since you didn't mention how many points and polygons you were working with, I did a test on my own data as a reference. To extract over 800K parcel centroids in my County using the Feature to Point tool took 6 minutes and 13 seconds and to use the Spatial Join tool to join those points to 28 cities and the unincorporated area of my County to get the count of parcels and sum of acreages (an attribute retained by the Feature to Point tool) of the parcels in each took 23 minutes and 11 seconds. So the total processing time of both steps was 29 minutes and 24 seconds. Anyway, hopefully that should give you an idea of how much time a geoprocessing approach should take with your data and whether or not it is a viable alternative. I would be interested in knowing how much data you were processing so I could do the same comparison.
... View more
11-16-2014
09:10 AM
|
1
|
1
|
6743
|
|
POST
|
I use the Feature Vertices to Points tool on all lines with the Ends option and then Use Locate Feature Along Routes with those points. Since the points may intersect more than one Route, I use an attribute from the Pavement Management data to select the point event RouteIDs that made sense for each line segment. Then I use the Summary Statistics tool to use the original line FID associated with the points as the Unique Case field and get the Route and Min and Max measure, which reproduce the lines. This solves issues created by true curves in Geodatabase Routes that I have. So even with line segments, I never use the lines themselves to apply the Locate Features Along Routes, and I like the results I get from end points far better, since I do consider line on line tools wonky for the reasons you mentioned.
... View more
11-14-2014
03:39 PM
|
1
|
2
|
3467
|
|
POST
|
These two add-ins are the set of search tools I created for Riverside County. If you download the County's data and reconnect the data sources these tools can search for various items. The map contains the minimum search tool set up requirements. Any other layers can be added to the map without affecting the tools as long as the search layer names are not changed. I also have included a layer file to add to your existing maps as an alternative to using the Base map; however, in addition to the layer the Base map includes layout objects that work in conjunction with the tools to display a Title and scale bar that best match the tool search results. Finally, there is an instruction guide for using the Search for Road Segment tool Word 2007 document attached. The Add-Ins work for ArcGIS 10.6 or above. If you want a version of the search tools for ArcGIS Pro versions 2.8.0 to 2.9.x you can download it from my Riverside County ArcGIS Pro Search Tool Add-In Blog. How to Install the Search Tools For users of my search tools who currently see my tools in ArcMap: If you are currently using my search tools in ArcMap and you want to update them to the latest version, you just need to refresh the Add-Ins by doing the following: Close ArcMap if it is open. Save the attached SearchFormsNew1.zip file to your desktop and double click it. Unzip the files and double click the SearchFormsNew1.esriAddIn file. Click the Install Add-in button at the bottom of the Esri ArcGIS Add-In Installation Utility dialog. Click the OK button on the Esri ArcGIS Add-In Installation Utility dialog that says the Installation succeeded. Save the attached SearchRoadSegment.esriAddIn.zip file to your desktop and double click it. Unzip the files and double click the SearchRoadSegment.esriAddIn file. Click the Install Add-in button at the bottom of the Esri ArcGIS Add-In Installation Utility dialog. Click the OK button on the Esri ArcGIS Add-In Installation Utility dialog that says the Installation succeeded. You should now be able to open ArcMap with a map containing the appropriate Search layers and use the tools. For new users of my search tools or those with a new installation of ArcMap - To install the search tools for a new or fresh full installation do the following: Close ArcMap if it is open. Save the attached SearchFormsNew1.esriAddIn.zip file to your desktop and double click it. Unzip the files and double click the esriAddIns. Click the Install Add-in button at the bottom of the Esri ArcGIS Add-In Installation Utility dialog. Click the OK button on the Esri ArcGIS Add-In Installation Utility dialog that says the Installation succeeded. Save the attached SearchRoadSegment.esriAddIn.zip file to your desktop and double click it. Unzip the files and double click the esriAddIns. Click the Install Add-in button at the bottom of the Esri ArcGIS Add-In Installation Utility dialog. Click the OK button on the Esri ArcGIS Add-In Installation Utility dialog that says the Installation succeeded. Open ArcMap without loading a document. Expand the Customize menu on the main menu bar. Click Customize Mode in the drop down menu. Click the Commands tab in the Customize dialog window. Towards the top of the Categories list click the Add-In Controls Category. Click and Drag the Search Road Segments icon onto your main toolbar and drop it onto the toolbar. Scroll to the bottom of the Categories list and click the Transportation Category. Click and Drag the Launch Forms icon onto your main toolbar (probably next to the Identify tool) and drop it onto the toolbar. Close the Customize Dialog. Load a map that has the layers that have names beginning with the word “Search” in them. The attached Search Layers.lyr file and BAse Search Tool Map.mxd area available for those who do not have access to the internal Riverside County Transportation and Land Management standard maps. Click the Launch Forms button or the Search Road Segments button on your toolbar and perform a search. Message was edited by: Richard Fairhurst on November 22, 2014, December 3, 2014, and December 8, 2014 Updated the SearchFormsNew.esriAddIn to include a Point Address search tool and modified the Base Search Tool Map.mxd to include a Search Address Points layer to support the tool. Added a separate layer file as an alternative way to add the search capabilities to existing maps. Indicated the version of ArcMap supported by the Add-Ins. 12/3 - Corrected dialog behavior for APN Address Search tool. 12/4 - Created a Search for Parcel Owner tool. 12/8 - Added Installation Instructions and usage instructions for the Search for Road Segment tool. Message was edited by: Richard Fairhurst Fixed a minor measure unit conversion bug on the interface, Message was edited by: Richard Fairhurst - January 28, 2016 5:33 P.M. PST - Updated the SearchRoadSegment.esriAddin Tool to work with ArcMap 10.3 Message was edited by: Richard Fairhurst - August 23, 2016 9:54 A.M. PST - Updated the SearchFormsNew.esriAddin Tool to handle Address searches when the Owner1 fields are Null and altered the CaseReporter button to make the local path based on the user name. The SearchFormsNew.esriAddIn may require ArcMap 10.3 now. Message was edited by: Richard Fairhurst - August 25, 2016 7:15 A.M. PST - Updated the SearchRoadSegment.esriAddin Tool to fix bug in the Intersection Search tool that caused the Primary name to change when user selected and started typing over the Secondary Name, so that now the Secondary Name is edited as expected. Message was edited by: Richard Fairhurst - January 10, 2018 11:40 AM PST - I attached a 10.5.1 version of the main search tools named SearchFormsNew1.zip. If you have ArcMap 10.5.1 or higher installed, use the tools in the SearchFormsNew1.zip version instead of the tool in SearchFormsNew.esriAdding.zip version. The latest tool fixes bugs created as a result of RCIT changing the Parcel_Assessors feature class so that it stores Null values in text fields instead of "" values. Message was edited by: Richard Fairhurst - January 11, 2018 08:55 AM PST - I replaced the SearchFormsNew1.zip file with a revised version that replaced the LMS Cases search with the PLUS Cases search. The PLUS Cases search needs a layer called Search PLUS Cases that has data coming from the PLUS_ACTIVITIES feature class or a derivative subset feature class extracted from PLUS_ACTIVITIES with a field called CASE_ID. Message was edited by: Richard Fairhurst - March 14, 2018 10:26 AM PST - I added the SearchRoadSegment1.zip file with a revised version that updates the Search Road Segment tool for ArcMap 10.5. Message was edited by: Richard Fairhurst - July 2, 2018 11:28 AM PST - I updated the SearchFormsNew1.esriAddin.zip file with a revised version of the Search Plus Cases tool that allows case numbers to have up to 25 characters.. Message was edited by: Richard Fairhurst - January 7, 2019 04:47 AM PST - I updated the SearchFormsNew1.esriAddin.zip file with a revised to work when searching for intersections with apostrophes in one or both of the street names. Message was edited by: Richard Fairhurst - March 15, 2022 12:05 PM PST - I replaced the SearchRoadSegment1 zip file with the SearchRoadSegment1 (2).zip file to use current DFS path name for layer files. Message was edited by: Richard Fairhurst - June 28, 2022 03:57 PM PST - I updated the SearchFormsNew1.esriAddin.zip file with a revised to work when searching for PARCELS_CREST as the Search_Parcels layer and the CREST_ASSESSEE table for property owners in the Search by Address and Search by Owner tools. Message was edited by: Richard Fairhurst - September 20, 2022 04:45 PM PST - I updated Included a link to my blog for an ArcGIS Pro version of the Search Tools. Message was edited by: Richard Fairhurst - May 16, 2023 11:15 AM PST - I corrected an error that prevented the search forms from appearing if the CREST_ASSESSEE table did not exist in the map, changed the GDB_PUB Server where the old server was referenced and compiled for ArcMap 10.8.
... View more
11-13-2014
12:59 PM
|
1
|
0
|
7255
|
|
POST
|
Did you use the ERROR_FIELD option for the add error field parameter? The values of that field will tell you what is wrong. For example, if you have incorrectly spelled your Route ID values in the event table an error saying that the Route cannot be found will appear. If the measures that do not exist on your routes, a different error will appear. Not one number or letter (even an extra space at the end of a string) can be different in a Route line's Route ID value and the Route ID value in the event table. You have to examine the route ID and measures of your route lines and the event attributes to be sure they all line up.
... View more
11-12-2014
10:20 PM
|
0
|
0
|
953
|
|
POST
|
I don't have a clear idea of what you are now doing. Do the quantities somehow relate to the polygons you originally spatially joined? I would imagine that Summary Statistics, Dissolve, Spatial Join, Intersect or some combination can do most anything you need to do, but I don't have a clear picture of what shapes and data you are working with at this stage.
... View more
11-12-2014
10:08 PM
|
0
|
0
|
2010
|
|
POST
|
I would use Linear Referencing to do this. Each line you have created will need to be a Route (give each a unique Route Name) starting at the random point with a measure of 0 miles and then ending with a measure of 100 miles (be sure the M resolution and M tolerance are set to the level that gives you the precision you want). Use the Locate Features along a Route tool with your shoreline polygon as the features to be located. This will create one or more line segment events on each route. Run the Summary Statistics tool to find each Route Name as the unique case field and the minimum From measure of any line events created. The output will be the closest point on the line to the random origin point. Convert the table to a Linear Referencing Point Event Layer and you can confirm that all the points fall at the intersection of the line and the closest shoreline.
... View more
11-10-2014
04:53 PM
|
0
|
0
|
4654
|
|
POST
|
Esri won't take note of what you have written if you don't even bother to look at the idea I posted on their Ideas site and vote for it. Vote.
... View more
11-10-2014
01:24 AM
|
0
|
0
|
2464
|
|
POST
|
I am finding that cursors and dictionaries are now reducing my time doing field calculations by factors of nearly 16 times the speed. As a result, I now have determined that my time will be better spent abandoning all Field and Geometry Calculator operations to write my own set of tools to replace them with an interface that configures cursors and dictionaries. My latest script rewrite replaced 11 field calculations and one summary statistics operation with cursors and dictionary operations on 800K records and reduced the script time from 63 minutes to 4 minutes. As a result, I simply can no longer justify the amount of my time the Field Calculator and Geometry Calculator will waste if I continue using them on any large record sets, no matter how easy they are to set up. Now that I can design high performing tools of my own I soon will no longer ever have to fall asleep again at my computer while I tie up my ArcGIS Desktop waiting on calculations to complete. I can't wait.
... View more
11-10-2014
01:13 AM
|
4
|
0
|
956
|
|
POST
|
You can vote for my idea on the Ideas site here that shows how I think Esri could fix the Zoom To button behavior of the Go To XY tool by giving the user the option to set the default scale behavior of that button.
... View more
11-09-2014
11:41 AM
|
0
|
0
|
2464
|
|
POST
|
The only options I have found are to first set the Scale tool on the Tools toolbar to the desired scale you want before using the Go to XY tool and then only use the Pan To tool rather than the Zoom To tool, or else use the Zoom To tool and let it zoom to the extent it wants and then enter the scale you want into the Scale tool on the Tools toolbar. Both of these options keep the coordinate you entered in the center of the screen and result in a scale that you can control to the desired level. The Pan To option is the best since you only have to set the scale once if keep using the Pan To tool and you don't have to constantly reenter the scale after panning to each coordinate, and the Pan To behavior is the default when you press the enter button after changing the coordinates of the tool. With the Zoom To tool you always have to override the scale after finding each coordinate. The help for the Go to XY tool has nothing that indicates it is possible to change the Zoom To button behavior of the Go to XY tool. I tried changing the Data Frame Extent setting to use the Set Scale option, but then the Zoom To tool is disabled and you cannot change the scale of the map at all. Changing the Reference Scale of the Data Frame had no effect on the Go to XY tool either. Changing a layer's "Don't show layer when zoomed: Out Beyond (minimum scale)" setting had no effect on the Go to XY tool behavior. No solution to control the scale used by the Zoom To tool came up on any of the Tips and Tricks of the Go to XY tool I found online, suggesting no one has a real solution that avoids a two step process. I programmed my own coordinate locating tool using VB.Net, but I only bothered to design it to use the normal spatial reference of my data, and it has no other options other than to let the user control the scale the tool will use. Replicating a tool identical to the Go To XY tool is something I could do if I wanted to, but I have instead designed other tools that find locations based on other datasets that all my users rely on to get around. If I recreated the tool the only change I would make to its design is to add a number box on the tool that would let you enter whatever desired scale value you wanted the tool to use by default to control the Zoom To tool behavior, so that you could set the scale once and forget it and easily see what the current scale setting was if you did not like the scale the tool was using. I will add that suggestion to the Ideas site. With a number box on the tool controlling the scale they can initiate the tool to show the standard scale they currently are using and store the last user preference entered into the number box in the Registry so that the user only had to specify their preference once and forget it. Alternatively, they could configure it to set a default option added to the General Tab of the Data Frame to let you customize it for each map, but then you would need a help document to tell you where to go any time you wanted to make an adjustment to that setting. Anyway, user control of this behavior is a no brainer, since only the user should decide what scales work best for their workflows, not a lazy Esri programmer that did not even bother to consider that possibility.
... View more
11-09-2014
10:12 AM
|
0
|
2
|
2464
|
|
BLOG
|
I found one mistake in my code examples. Where ever I tried to convert a string to Title Case the syntax should not have been: updateRow[5] = Title(updateRow[5]) It should have been: updateRow[5] = updateRow[5].title() Note: Original Blog post has been corrected.
... View more
11-08-2014
07:52 PM
|
2
|
0
|
34563
|
|
POST
|
The only way I have found to avoid using the Project tool to do a conversion from a spatial projection that uses linear units (like State Plane coordinates) to angular units (like Decimal Degrees) is using the Geoprocessing Environment settings to set up any transformation option and the spatial reference parameter of a Python UpdateCursor. If you have ArcGIS Desktop 10.1 or later you should only use a Data Access UpdateCursor (da UpdateCursor), not the original Python UpdateCursor syntax. A note of warning, if you use this approach with an XY Event layer that has its original coordinates in linear units, like State Plane coordinates, you need to first duplicate those coordinates into a pair of new fields that you want to hold your Latitude (Y coordinate) and Longitude (X coordinate) and use those new fields with the XY Event Layer tool, because if you include the SHAPE@X or SHAPE@Y feilds in the UpdateCursor field list and use the updateRow method, the fields used to set the coordinates of the XY Event Layer will convert and be overwritten with new coordinate values that match the new spatial reference and transformation. The script below can create a new Latitude and Longitude field in an XY Event Layer table that has only State Plane coordinates, make a copy of the original state plane coordinates into the new Latitude and Longitude fields, and use an XY Event layer and a da UpdateCursor opened with a different spatial reference to convert the values in the Latitude and Longitude fields into the coordinates matching the new spatial reference. It can run the entire script to process over 131K records in an XY Event Layer in approximately 36 seconds, which is about 5 to 10 times faster than methods that used to use the Field Calculator.
# Import arcpy module
from time import strftime
print "Start script: " + strftime("%Y-%m-%d %H:%M:%S")
import arcpy
# Set or comment out Geographic Transformation
arcpy.env.geographicTransformations = "NAD_1983_To_WGS_1984_5"
# Use a table with State Plane NAD 83 coordinates
fc = r"C:\Users\RFAIRHUR\Documents\ArcGIS\DEFAULT.gdb\CL_INTERSECTIONS_PAIRS"
print "Finished Setup: " + strftime("%Y-%m-%d %H:%M:%S")
# Process: Add Latitude Field
arcpy.AddField_management(fc, "LATITUDE", "DOUBLE", "", "", "", "", "NULLABLE", "NON_REQUIRED", "")
# Process: Add Longitude Field
arcpy.AddField_management(fc, "LONGITUDE", "DOUBLE", "", "", "", "", "NULLABLE", "NON_REQUIRED", "")
print "Finish Adding Fields: " + strftime("%Y-%m-%d %H:%M:%S")
# Specify original X and Y fields and corresponding Longitude and Latitude output fields.
fields = ( 'LONGITUDE', 'X_COORD', 'LATITUDE', 'Y_COORD')
#Open a da UpdateCursor and duplicate the X and Y coordinates in the Longitude and Latitude fields.
with arcpy.da.UpdateCursor(fc, fields) as updateRows:
for updateRow in updateRows:
updateRow[0] = updateRow[1]
updateRow[2] = updateRow[3]
updateRows.updateRow(updateRow)
print "Finished UpdateCursor Longitude and Latitude edits " + strftime("%Y-%m-%d %H:%M:%S")
# Set a name for your XY Event Layer
CL_INTERSECTIONS_PAIRS_Events = "CL_INTERSECTIONS_PAIRS Events"
# Make an X/Y Event Layer with NAD 83 projection.
arcpy.MakeXYEventLayer_management(fc, "LONGITUDE", "LATITUDE", CL_INTERSECTIONS_PAIRS_Events, "PROJCS['NAD_1983_StatePlane_California_VI_FIPS_0406_Feet',GEOGCS['GCS_North_American_1983',DATUM['D_North_American_1983',SPHEROID['GRS_1980',6378137.0,298.257222101]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]],PROJECTION['Lambert_Conformal_Conic'],PARAMETER['False_Easting',6561666.666666666],PARAMETER['False_Northing',1640416.666666667],PARAMETER['Central_Meridian',-116.25],PARAMETER['Standard_Parallel_1',32.78333333333333],PARAMETER['Standard_Parallel_2',33.88333333333333],PARAMETER['Latitude_Of_Origin',32.16666666666666],UNIT['Foot_US',0.3048006096012192]];-118608900 -91259500 3048.00609601219;-100000 10000;-100000 10000;3.28083333333333E-03;0.001;0.001;IsHighPrecision", "")
print "Finished Making XY Event Layer: " + strftime("%Y-%m-%d %H:%M:%S")
# Use the Event layer as the cursor input
fcl = CL_INTERSECTIONS_PAIRS_Events
# Specify shape coordinate fields.
fields = ("SHAPE@X", "SHAPE@Y")
# Specify the Spatial Reference of the Cursor
latLonRef = "Coordinate Systems\Geographic Coordinate Systems\World\WGS 1984.prj"
# Create update cursor for feature class with a new Spatial Reference.
with arcpy.da.UpdateCursor(fcl, fields, ' ', latLonRef) as updateRows:
for updateRow in updateRows:
# Lat and Long convert just by using the updateRow method on the Shape fields
updateRows.updateRow(updateRow)
print "Finished Converting Longitude and Latitude fields: " + strftime("%Y-%m-%d %H:%M:%S")
# The coordinates in the Longitude and Latitude fields are now converted to Decimal Degrees.
print "Finish script: " + strftime("%Y-%m-%d %H:%M:%S")
Below is a script that adds a Latitude and Longitude field to a State Plane NAD 83 Point Feature Class and using an UpdateCursor with a different spatial reference is able to fill those fields in with the Decimal Degree coordinates of a WGS 1984 spatial reference. The underly geometry of the points is unaffected by the script. The entire script takes approximately 25 seconds to run on over 131K points.
from time import strftime
print "Start script: " + strftime("%Y-%m-%d %H:%M:%S")
import arcpy
# Set or comment out Geographic Transformation
arcpy.env.geographicTransformations = "NAD_1983_To_WGS_1984_5"
# Use a table with State Plane NAD 83 coordinates
fc = r"C:\Users\RFAIRHUR\Documents\ArcGIS\DEFAULT.gdb\CL_INT_PAIRS_POINTS"
print "Finished Setup: " + strftime("%Y-%m-%d %H:%M:%S")
# Process: Add Latitude Field
arcpy.AddField_management(fc, "LATITUDE", "DOUBLE", "", "", "", "", "NULLABLE", "NON_REQUIRED", "")
# Process: Add Longitude Field
arcpy.AddField_management(fc, "LONGITUDE", "DOUBLE", "", "", "", "", "NULLABLE", "NON_REQUIRED", "")
print "Finish Adding Fields: " + strftime("%Y-%m-%d %H:%M:%S")
# Specify shape coordinate fields.
fields = ("LONGITUDE", "SHAPE@X", "LATITUDE", "SHAPE@Y")
# Specify the Spatial Reference of the Cursor
latLonRef = "Coordinate Systems\Geographic Coordinate Systems\World\WGS 1984.prj"
# Create update cursor for feature class with a new Spatial Reference.
with arcpy.da.UpdateCursor(fc, fields, ' ', latLonRef) as updateRows:
for updateRow in updateRows:
updateRow[0] = updateRow[1]
updateRow[2] = updateRow[3]
updateRows.updateRow(updateRow)
print "Finished Converting Longitude and Latitude fields: " + strftime("%Y-%m-%d %H:%M:%S")
# The coordinates in the Longitude and Latitude fields are now converted to Decimal Degrees.
print "Finish script: " + strftime("%Y-%m-%d %H:%M:%S")
... View more
11-08-2014
01:49 PM
|
2
|
2
|
5072
|
|
BLOG
|
The Problems with Using the Field Calculator, Join and Summary Statistics tools The field calculator is a great tool and easy to set up, but it has several limitations that can seriously slow down scripted routines when you need to do a lot of data manipulation with large data sets. The most obvious limitation is that only one field can be calculated at a time. This often means having to process every record in the feature class multiple times in order to update several fields. If you need to do data transfers between feature classes using a join it is critical to index the join field in both feature classes to avoid calculations that take forever to process, and even then the speed is significantly slower when a join is involved. Also, if you actually need to use multiple fields to create a join that correctly matches records between feature classes your only option is to create a new field in both feature classes that concatenates those field values to make a join that works. Also, often before these joins can be created a summary table has to be created, which takes more time. Fortunately if you have ArcGIS Desktop 10.1 or later there is an alternative that can speed scripts like this up dramatically and make data transfers between feature classes both flexible and incredibly fast. The solution is to replace the field calculator with one or more data access cursors (da cursors) and joins and creating summary tables with one or more python dictionaries built using a da SearchCursor. Using a da UpdateCursor to Replace a Field Calculation within a Single Feature Class: Below is a simple script that will use a da UpdateCursor on a feature class to fill in a URL field with a hyperlink that is built using a value taken from another field. The strftime method is used to print out the time when the script starts and the time when it finishes. To adapt this script to your own data you only need to change the Feature Class path and name in line 7, the field names in the field list in line 9 and the URL string in line 14. This should perform about 7 times faster than a similar field calculation. from time import strftime
print( "Start script: " + strftime("%Y-%m-%d %H:%M:%S"))
import arcpy
updateFC = r"C:\Path\FeatureClass"
updateFieldsList = ["URLField", "ValueField"] # Example fields that will build a URL based on another field
with arcpy.da.UpdateCursor(updateFC, updateFieldsList) as updateRows:
for updateRow in updateRows:
# updateRow[0] is the URL field and updateRow[1] is the value field
updateRow[0] = r"
http://pic.asrclkrec.com/view_ParcelMaps.aspx?ParcelNumber=
" + str(updateRow[1])
updateRows.updateRow(updateRow)
print( "Finished APN_MAP field update: " + strftime("%Y-%m-%d %H:%M:%S"))
Using a da UpdateCursor to Replace Several Field Calculations within a Single Feature Class: Now lets say that we actually need to process 2 more field calculations to complete the update of this feature class. One calculation needs to add two other fields together, and the other needs to convert the values in a field from capitalized values to title case values. So if there are 100K records, then all of those records have to be calculated 3 times. All of this can be handled in a single pass of one da UpdateCursor through the 100K records. Because the records only have to changed once and the addition of more fields for the UpdateCursor to read or calculate takes very little additional time to process, the speed gain over 3 field calculations is much faster that the previous script. As a result, the more field calculations you can replace with a single pass of a da UpdateCursor, the more dramatically the speed of the script will improve. Below is the script that can do all three field updates (assume that there are no Null values in the fields being added together or changing case). To use this script with your own data you still only need to change lines 7, 9, and 14 (optionally change the calculations in lines 16 and 18). from time import strftime
print( "Start script: " + strftime("%Y-%m-%d %H:%M:%S"))
import arcpy
updateFC = r"C:\Path\FeatureClass"
updateFieldsList = ["URLField", "ValueField", "SumField", "NumberField1", "NumberField2", "TitleCaseField"] # Example fields that will build a URL based on another field
with arcpy.da.UpdateCursor(updateFC, updateFieldsList) as updateRows:
for updateRow in updateRows:
# updateRow[0] is the URL field and updateRow[1] is the value field
updateRow[0] = r"
http://pic.asrclkrec.com/view_ParcelMaps.aspx?ParcelNumber=
" + str(updateRow[1])
# updateRow[2] is the Sumfield and updateRow[3] and updateRow[4] are the NumberField1 and NumberField2 fields.
updateRow[2] = updateRow[3] + updateRow[4]
# updateRow[5] is the TitleCaseField that is having its Capitalized values replaced with Title Case values.
updateRow[
5
] = updateRow[
5
].title()
updateRows.updateRow(updateRow)
print( "Finished APN_MAP field update: " + strftime("%Y-%m-%d %H:%M:%S"))
Using a Python Dictionary Built using a da SearchCursor to Replace a Join Connecting Two Feature Classes: Another common data manipulation involves joining one feature class to another in order to transfer values from the Joined feature class to the Target feature class using the Field Calculator. However, features loaded into a Python Dictionary using a Search Cursor are much faster at matching join field values in one feature class to another than a Join. Combined with using a da UpdateCursor to replace the Field Calculator, the speed of these kinds of data manipulations can be even more dramatic than data manipulations on a single feature class. Example 1 - Transfer of a Single Field Value between Feature Classes Below is the code required to transfer the value of one feature class to another based on a common join value in another field. To adapt this script to your own data change lines 7 and 14 to the path and name of the source (Join) and update (target) Feature Class to match the feature classes you would normally join, change lines 9 and 16 to replace the field list with the name of the Join field and the name of the field being transferred. from time import strftime
print( "Start script: " + strftime("%Y-%m-%d %H:%M:%S"))
import arcpy
sourceFC = r"C:\Path\SourceFeatureClass"
sourceFieldsList = ["JoinField", "ValueField"]
# Use list comprehension to build a dictionary from a da SearchCursor
valueDict = {r[0]:(r[1:]) for r in arcpy.da.SearchCursor(sourceFC, sourceFieldsList)}
updateFC = r"C:\Path\UpdateFeatureClass"
updateFieldsList = ["JoinField", "ValueField"]
with arcpy.da.UpdateCursor(updateFC, updateFieldsList) as updateRows:
for updateRow in updateRows:
# store the Join value of the row being updated in a keyValue variable
keyValue = updateRow[0]
# verify that the keyValue is in the Dictionary
if keyValue in valueDict:
# transfer the value stored under the keyValue from the dictionary to the updated field.
updateRow[1] = valueDict[keyValue][0]
updateRows.updateRow(updateRow)
del valueDict
print( "Finished script: " + strftime("%Y-%m-%d %H:%M:%S"))
Example 2 - Transfer of Multiple Field Values between Feature Classes where there is a 1:1 Match between Field Sets Just like the Field Calculator the more fields being transferred from one feature class to the other, the more your script will speed up. If the transfer is a simple field for field transfer then the transfer fields can be handled in a for loop. Below is an example of a transfer of 5 matching fields between two feature classes. To adapt this code to your data you should again change the feature classes in lines 7 and 14 and the field lists in lines 9 and 16. from time import strftime
print( "Start script: " + strftime("%Y-%m-%d %H:%M:%S"))
import arcpy
sourceFC = r"C:\Path\SourceFeatureClass"
sourceFieldsList = ["JoinField", "ValueField1", "ValueField2", "ValueField3", "ValueField4", "ValueField5" ]
# Use list comprehension to build a dictionary from a da SearchCursor
valueDict = {r[0]:(r[1:]) for r in arcpy.da.SearchCursor(sourceFC, sourceFieldsList)}
updateFC = r"C:\Path\UpdateFeatureClass"
updateFieldsList = ["JoinField", "ValueField1", "ValueField2", "ValueField3", "ValueField4", "ValueField5"]
with arcpy.da.UpdateCursor(updateFC, updateFieldsList) as updateRows:
for updateRow in updateRows:
# store the Join value of the row being updated in a keyValue variable
keyValue = updateRow[0]
# verify that the keyValue is in the Dictionary
if keyValue in valueDict:
# transfer the values stored under the keyValue from the dictionary to the updated fields.
for n in range (1,len(sourceFieldsList)):
updateRow[n] = valueDict[keyValue][n-1]
updateRows.updateRow(updateRow)
del valueDict
print( "Finished script: " + strftime("%Y-%m-%d %H:%M:%S")) Example 3 - Transfer of Multiple Field Values between Feature Classes involving Complex Field Manipulations If you need to do more complex field transfers that do more than a simple transfer of data from one table to another you can do this using code similar to the example field calculations that were done on a single table. Here are the 3 field updates of my second field calculation example done between two separate tables with common join values instead of all within a single table. from time import strftime
print( "Start script: " + strftime("%Y-%m-%d %H:%M:%S"))
import arcpy
sourceFC = r"C:\Path\SourceFeatureClass"
sourceFieldsList = ["JoinField", "ValueField", "NumberField1", "NumberField2", "CapitalizedField" ]
# Use list comprehension to build a dictionary from a da SearchCursor
valueDict = {r[0]:(r[1:]) for r in arcpy.da.SearchCursor(sourceFC, sourceFieldsList)}
updateFC = r"C:\Path\UpdateFeatureClass"
updateFieldsList = ["JoinField", "UrlField1", "SumField", "TitleCaseField"]
with arcpy.da.UpdateCursor(updateFC, updateFieldsList) as updateRows:
for updateRow in updateRows:
keyValue = updateRow[0]
# verify that the keyValue is in the Dictionary
if keyValue in valueDict:
# updateRow[1] is the URL field and valueDict[keyValue][0] is the value field
updateRow[1] = r"
http://pic.asrclkrec.com/view_ParcelMaps.aspx?ParcelNumber=
" + str(valueDict[keyValue][0])
# updateRow[2] is the Sumfield and valueDict[keyValue][1] and valueDict[keyValue][2] are the NumberField1 and NumberField2 fields.
updateRow[2] = valueDict[keyValue][1] + valueDict[keyValue][2]
# updateRow[3] is the TitleCaseField and valueDict[keyValue][3] is the CapitalizedField values.
updateRow[3
] = valueDict[keyValue][3
].title()
updateRows.updateRow(updateRow)
del valueDict
print( "Finished script: " + strftime("%Y-%m-%d %H:%M:%S"))
Creating a Multi-Field Python Dictionary Key to Replace a Concatenated Join Field: Handling Multi-field join values that match records between two feature classes is also possible using dictionaries and cursors without having to do concatenation of separate fields into a new field. The concatenation is still necessary, but it can be handled on the fly in memory, which is much faster than calculating a field. Below shows an example of the single field transfer, but this time matching records using 3 separate fields to define the unique values between the feature classes used to match the records between the feature classes. from time import strftime
print( "Start script: " + strftime("%Y-%m-%d %H:%M:%S"))
import arcpy
sourceFC = r"C:\Path\SourceFeatureClass"
sourceFieldsList = ["JoinField1", "JoinField2", "JoinField3", "ValueField"]
# Use list comprehension to build a dictionary from a da SearchCursor where the key values are based on 3 separate feilds
valueDict = {str(r[0]) + "," + str(r[1]) + "," + str(r[2]):(r[3:]) for r in arcpy.da.SearchCursor(sourceFC, sourceFieldsList)}
updateFC = r"C:\Path\UpdateFeatureClass"
updateFieldsList = ["JoinField1", "JoinField2", "JoinField3", "ValueField"]
with arcpy.da.UpdateCursor(updateFC, updateFieldsList) as updateRows:
for updateRow in updateRows:
# store the Join value by combining 3 field values of the row being updated in a keyValue variable
keyValue = updateRow[0]+ "," + str(updateRow[1]) + "," + str(updateRow[2]
# verify that the keyValue is in the Dictionary
if keyValue in valueDict:
# transfer the value stored under the keyValue from the dictionary to the updated field.
updateRow[3] = valueDict[keyValue][0]
updateRows.updateRow(updateRow)
del valueDict
print( "Finished script: " + strftime("%Y-%m-%d %H:%M:%S"))
Replacing Successive Joins with Simultaneous Processing of Several Python Dictionaries Built using da SearchCursors When several joins have to be created and removed to do data manipulation between tables, the speed gains can be increased even more, since you can create several dictionaries and then process all of the join relationships simultaneously during a single pass of the update cursor on the updated feature class. All source tables should be read into dictionaries before processing the update cursor to achieve this benefit. The more joins that you can replace this way, the more your script speed will improve. Below is an example where two feature classes that would normally be joined to the updated feature class are replaced by dictionaries and the processed simultaneously by a single UpdateCursor operation. from time import strftime
print( "Start script: " + strftime("%Y-%m-%d %H:%M:%S"))
import arcpy
sourceFC = r"C:\Path\SourceFeatureClass1"
sourceFieldsList = ["JoinField1", "ValueField", "NumberField1", "NumberField2" ]
# Use list comprehension to build a dictionary from a da SearchCursor
valueDict1 = {r[0]:(r[1:]) for r in arcpy.da.SearchCursor(sourceFC, sourceFieldsList)}
sourceFC = r"C:\Path\SourceFeatureClass2"
sourceFieldsList = ["JoinField2", "CapitalizedField" ]
# Use list comprehension to build a dictionary from a da SearchCursor
valueDict2 = {r[0]:(r[1:]) for r in arcpy.da.SearchCursor(sourceFC, sourceFieldsList)}
updateFC = r"C:\Path\UpdateFeatureClass"
updateFieldsList = ["JoinField1", "UrlField1", "SumField", "JoinField2",
"TitleCaseField"]
with arcpy.da.UpdateCursor(updateFC, updateFieldsList) as updateRows:
for updateRow in updateRows:
keyValue1 = updateRow[0]
# verify that the keyValue1 is in the First Dictionary
if keyValue1 in valueDict1:
# updateRow[1] is the URL field and valueDict1[keyValue1][0] is the value field
updateRow[1] = r"
http://pic.asrclkrec.com/view_ParcelMaps.aspx?ParcelNumber=
" + str(valueDict1[keyValue1][0])
# updateRow[2] is the Sumfield and valueDict1[keyValue1][1] and valueDict1[keyValue1][2] are the NumberField1 and NumberField2 fields.
updateRow[2] = valueDict1[keyValue1][1] + valueDict1[keyValue1][2]
keyValue2 = updateRow[3]
# verify that the keyValue2 is in the Second Dictionary
if keyValue2 in valueDict2:
# updateRow[4] is the TitleCaseField and valueDict2[keyValue2][0] is the CapitalizedField values.
updateRow[4
] = valueDict2[keyValue2
][0].title()
updateRows.updateRow(updateRow)
del valueDict1
del valueDict2
print( "Finished script: " + strftime("%Y-%m-%d %H:%M:%S"))
Using a Python Dictionary Built using a da SearchCursor to Replace a Summary Statistics Output Table Another common need is to create a summary of a feature class for the unique values of a field or fields that cover multiple records. The Python Dictionary can be used instead of a Summary Table output to accomplish this. The benefit is that the output is stored in memory and is much faster to create than outputting an actual summary table. Below is an example building a summary dictionary and using it to update the source table with sum values matching that field and create a percentage field in the source table for each record that built the summary. from time import strftime
print( "Start script: " + strftime("%Y-%m-%d %H:%M:%S"))
import arcpy
sourceFC = r"C:\Path\UpdateFeatureClass"
sourceFieldsList = ["UniqueValuesField", "NumberField"]
# Build a summary dictionary from a da SearchCursor with unique key values of a field storing a list of the sum of that value and the record count.
valueDict = {}
with arcpy.da.SearchCursor(sourceFC, sourceFieldsList) as searchRows:
for searchRow in searchRows:
keyValue = searchRow[0]
if not keyValue in valueDict:
# assign a new keyValue entry to the dictionary storing a list of the first NumberField value and 1 for the first record counter value
valueDict[keyValue] = [searchRow[1], 1]
# Sum the last summary of NumberField value with the current record and increment the record count when keyvalue is already in the dictionary
else:
valueDict[keyValue][0] += searchRow[1]
valueDict[keyValue][1] += 1
updateFC = r"C:\Path\UpdateFeatureClass"
updateFieldsList = ["UniqueValuesField", "NumberField", "PercentField", "NumberSumField", "MeanField"]
with arcpy.da.UpdateCursor(updateFC, updateFieldsList) as updateRows:
for updateRow in updateRows:
# store the Join value of the row being updated in a keyValue variable
keyValue = updateRow[0]
# verify that the keyValue is in the Dictionary
if keyValue in valueDict:
# divide the current record's NumberField value by the sum of the NumberField to get a percentage value
updateRow[2] = updateRow[1] / valueDict[keyValue][0]
# transfer the sum of the NumberField stored under the keyValue from the dictionary.
updateRow[3] = valueDict[keyValue][0]
# divide the sum of the NumberField value by the record count of the NumberField to get a mean value
updateRow[4] = valueDict[keyValue][0] / valueDict[keyValue][1]
updateRows.updateRow(updateRow)
del valueDict
print( "Finished script: " + strftime("%Y-%m-%d %H:%M:%S"))
Conclusion Once the patterns are understood it is possible to do virtually everything that you can do with the Field Calculator, a Join or a Summary table using an update cursor or a python dictionary. Using these coding techniques I have been able to rewrite several scripts that did bulk Data Manipulations so that rather than taking more than an hour each to process, they now only take 3 to 10 minutes to process. The more of these operations that you can replace using code like this, the greater the savings in processing time you will achieve. The performance gains can be huge making it possible to keep data up to date without extensive waits or consuming large portions of your after hours batch processing time. It also solves the problems that arise when you cannot alter the source data schema to add indexes or field concatenations and making copies of the data is not practical. if you find any errors in the sample code, please let me know and I will post an update to this Blog to point out the correction. Also if you need more comments added to the code to help you understand what is going on, let me know and I will try to comment the code more fully. These code examples are designed to work on shapefiles, file geodatabase feature classes, and unversioned SDE feature classes. Additional code modifications are required to make them work with versioned SDE feature classes. I want to express my thanks to Chris Synder who first demonstrated the usefulness of these techniques to me through the Forum and helped me understand how to apply them.
... View more
11-08-2014
02:16 AM
|
48
|
91
|
70773
|
|
POST
|
For lines you can get Lat and Long values for the start, end or centroid points on the line. The calculations are: !Shape.FirstPoint.X! and !Shape.FirstPoint.Y! !Shape.LastPoint.X! and !Shape.LastPoint.Y! !Shape.Centroid.X! and !Shape.Centroid.Y! The Attribute Assistant can extract each of these coordinates if you change the Value Info field to S for the start of the line, E for the end of the line, or Null for the centroid. I use the S and E options for all of my road centerlines and then concatenate them with the length to form an attribute join field that represents the unique geometry of each line. I can detect changes this way. Attribute Assistant can also fill in one or more date fields automatically every time the geometry is created or altered or the attributes are altered. So you absolutely should invest the time in using it for all of your geometry or attribute alteration monitoring routines.
... View more
11-06-2014
05:31 PM
|
1
|
0
|
3492
|
|
POST
|
Latitude is the Y Coordinate and Longitude is the X Coordinate. So the coordinates for NAD 83 do go to a spot in Nevada about 3000 feet south of R R Valley Rd to the Southeast of Highway 6 if you enter them as: 38.591616 -115.531531 I presume that is the correct spot. Coordinates will only be returned by Attribute Assistant in the native coordinate system of the Feature Class and if it is not in WGS 1984 the coordinates returned will not work correctly with Google Earth. Attribute Assistant has no options to do on the fly projections from one spatial reference to another. This could mean that you have to maintain the point in two different feature classes if the coordinate system of all your base data is in Web Mercator (Auxiliary Sphere) and you also want WGS 1984 coordinates or you would need to do a separate geoprocessing step to do a temporary projection and coordinate calculation into a WGS 1984 projection, and either way then join and calculate the WGS 1984 coordinates into the Web Mercator feature class. Geoprocessing would normally not be live and copy pasting from one projection to another in an edit session is not recommended. I do the geoprocessing approach and use a python cursor and dictionary routine to do the transfer rather than a join and calculate for much better speed. Edit: Corrected myself about the projection Google uses, which is WGS 1984, not NAD 83.
... View more
11-05-2014
06:28 PM
|
1
|
2
|
3492
|
| Title | Kudos | Posted |
|---|---|---|
| 1 | 03-24-2026 11:37 PM | |
| 1 | 03-24-2026 08:01 PM | |
| 7 | 02-23-2026 08:34 AM | |
| 1 | 03-31-2025 03:25 PM | |
| 1 | 03-28-2025 06:54 PM |
| Online Status |
Offline
|
| Date Last Visited |
07-09-2026
12:59 AM
|