Latest Contributions by ChrisSnyder

‎09-11-2012

One clarification: The cursor method is ONLY faster when using the 10.1 data access cursors... I found using the "old" cursor model is SLOWER than the Merge or Append tools. For my particular case, I found in order of speedieness: 1. arcpy (non-data access) cursor method (slowest) 2. Merge tool 3. Append tool (ever so slightly faster than Merge tool) 4. arcpy.da cursor method (fastest BY FAR!)

‎09-11-2012

Note that virus scan software leaves the in_memory workspace alone (since it's not on disk)...

‎09-11-2012

Merge just runs Append in a loop... Both tools are slow since they seem to rebuild the spatial index file with each loop of Appending. Perfiormace degradation is noticable if you are merging >2 FCs and/or if they are large FCs. Using cursors is by far the best way: http://forums.arcgis.com/threads/66434-A-better-way-to-run-large-Append-Merge-jobs

‎09-11-2012

in_memory workspace -- the Add Fields happen very fast with no file-locking issues ..DeleteField runs like a champ in_memory too.

‎09-11-2012

I also had occasional issues with repetitive calls to AddField dropping the error: ERROR 000464: Cannot get exclusive schema lock. Either being edited or in use by another application. What I was able to ultimately deduce is that it was my virus scan software, which I assume, was identifying repetitive calls to AddField as "suspicious". As I recall, all of a sudden right after a McAfee upgrade, all my scripts started sporadically failing... All of them where the AddField was running in a loop. Upon disabling McAfee, the problem was solved. Possible solution: Try disabling/fiddling your virus scan software, or move to another virus scan software.

‎09-09-2012

See this Help topic: http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//000v00000051000000 Try using the featurelayer's SQL 1st... You may "have to" use the SelectLayerByAttribute tool though... I am not totally familiar with the arcpy.mapping module.

‎09-07-2012

It depends on what you want to do with the selected set, but typically I use the MakeFeatureLayer tool to apply rapid selections (runs faster than the SelectLayerByAttribute tool). The Select tool actually creates a new featureclass on disk, so has the additional overhead of writting to disk... The MakeFeatureLayer tool creates a "feature layer" that is an in memory reference to the selcted features on disk (but does not actualy store the geometry/attributes in RAM). Then you can use the feature layer for input into other tools just as you would a feature class. For example: arcpy.MakeFeatureLayer_management(infeatures, "my_feature_layer", "Job_Num = " + str(userinput)) arcpy.Clip_analysis("my_feature_layer", ....)

‎09-07-2012

Being frustrated at the unnecessarily long processing times of the Merge and/or Append tools for large datasets, I wrote some simple Python code (using the data access module in v10.1) to emulate the same functionality. Took my 2 hour Merge processing time to 11 minutes 🙂 I hope someone else can benefit from this code. Right now it is designed to just run on polygon FCs with the same schema, although it could be enhanced to do field mapping as well (not on my time though). It relies on a simple search and insert cursor using the arcpy.da module. BTW, ESRI surely gets some huge props from me for the new cursor model... import arcpy fcList = [a list of FCs that you want to merge together] outputFC = r"C:\temp\test.gdb\merge" for fc in fcList: if fcList.index(fc) == 0: arcpy.CopyFeatures_management(fc, outputFC) insertRows = arcpy.da.InsertCursor(outputFC, ["SHAPE@","*"]) else: searchRows = arcpy.da.SearchCursor(fc, ["SHAPE@","*"]) for searchRow in searchRows: insertRows.insertRow(searchRow) del searchRow, searchRows print "Appended " + str(fc) + "..." del insertRows

‎08-28-2012

How about something like this: ***untested code*** import arcpy myFC = r"C:\temp\test.shp" yearField = "YEAR" countyField = "COUNTY_CD" sequenceField = "SEQUENCE" #Integer concatField = "CONCAT" #text of sufficient length trackingDict = {} updateRows = arcpy.UpdateCursor(myFC) for updateRow in updateRows: yearValue = updateRow.getValue(yearField) countyValue = updateRow.getValue(countyField) if (yearValue, countyValue) in trackingDict: trackingDict[(yearValue, countyValue)] = trackingDict[(yearValue, countyValue)] + 1 else: trackingDict[(yearValue, countyValue)] = 1 sequenceId = trackingDict[(yearValue, countyValue)] updateRow.setValue(sequenceField, sequenceId) updateRow.setValue(concatField, str(yearValue) + "-" + str(countyValue) + "-" + str(sequenceId)) updateRows.updateRow(updateRow) del updateRow, updateRows

‎08-24-2012

In your database, what is the field value when there is "no unit field value"? Is it Null, '', ' ', or ?

‎08-24-2012

Don't include the SQL expression in your cursor statement... If your cursor is only returning records where "TDP_Status = 'GENERATED'", how would your else statement ever work... Do you see the issue?

‎08-24-2012

Since you are only quering records where 'TDP_Status" = 'GENERATED'' in your cursor, the else statment (the Buffer) will never fire off.

‎08-24-2012

Also, look into the Dice tool (new in v10.0): http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//001700000037000000. It will split a polygon into smaller polygons based on vertex counts... but you will not have any direct control on their size. As a post process, you could append polygons that were too small with their larger neighbors. The logic of that may be more work than you are willing to put into it though...

‎08-24-2012

Just to debate Curtis' statement about cursors vs. CalculateField... In my experience (which may be outdated now), when using the "PYTHON*" option in the CalculateField tool, the performace of the CalculateField tool and the gp/arcpy.UpdateCursor are exactly the same. A post on this topic from four years ago: http://forums.esri.com/Thread.asp?c=93&f=1729&t=262236#806786. In my mind, complex conditional expressions are much easier to deal with in an update cursor, and I generally reserve the CalculateField tool for only very simple expressions. I have found new arcpy.da cursors are significantly faster (for a number of reasons) and are much more akin to the performace and functionality of the ArcObjects cursors and more similar in performance to the "VB" option in the CalculateField tool.

‎08-24-2012

There would be a difference if the "unit" field was Null or simply a blank string of ''. Your code should work as is when unit is a Null, however, a blank string of '' is considered a valid value. Something like this should take care of both cases: def fulladdress(num, prefix, street, sttype, suffix, unit): if unit not in [None, '']: fulladd = num +' '+ prefix+' '+street+' '+sttype+' '+suffix+' #'+unit else: fulladd = num +' '+ prefix+' '+street+' '+sttype+' '+suffix fulladd = fulladd.strip() fulladd = fulladd.replace(' ',' ') return fulladd A tip for you: It is much easier to deal with and format complex conditional expresions in an UpdateCursor than in the "code block" of the CalculateField tool.

Online Status	Offline
Date Last Visited	‎08-30-2024 12:25 AM

My Ideas

Latest Contributions by ChrisSnyder

Re: ? append vs merge: which is faster?

Re: repeated AddField operations fails in 10.1, works in 10.0

Re: ? append vs merge: which is faster?

Re: repeated AddField operations fails in 10.1, works in 10.0

Re: repeated AddField operations fails in 10.1, works in 10.0

Re: Select Records

Re: Select Records

A better way to run large Append/Merge jobs

Re: Sequential ID by Two Fields

Re: Field calculation, concatenation

Re: I need help on the Update Cursor function

Re: I need help on the Update Cursor function

Re: Splitting an irregular polygon into smaller polygons of a size

Re: I need help on the Update Cursor function

Re: Field calculation, concatenation

Re: How to determine number of records using arcpy...

Re: how to install pywin32 for ArcGIS Pro 3.x

Re: Trouble installing Pythonwin in Pro 3.3.1

Re: Adding html link to AddMesage

Re: Python Copy_management throws exception "ERROR...