Latest Contributions by KimOllivier

I realise that you just wanted to do a temporary join in a script, but then what are you doing with the join? It is very slow. Be wary of the JoinField tool, it does not scale for large datasets. It can take several hours on a large table. :mad: So what else can a poor Dilbert do? Dictionary/UpdateCursor My first attempt was to read the data with a searchCursor into a dictionary, then write out using an updateCursor. Better, but a bit complicated and still very slow at 10.0, improved at 10.1. FieldMap with a copy I have recently discovered that you can do a temporary join and then use TableToTable or FeatureclassToFeatureclass with a Fieldmap. This is really fast in comparison, but it is even harder than 'cursor and dictionary' to set up a fieldmap in a script.:( My workaround is to run the command interactively, turn off unwanted fields, and then grab the Python snippet from the results window and paste it into a script. It is then a 'macro' that you can substitute the paths and names with variables. Use triple quotes to imbed quotes more easily. The disadvantage with the fieldmap method is that you get yet another copy, but that is normal for geoprocessing tools. you just have to delete and rename to cleanup. ... View more

It looks like Python wins over ArcObjects for development speed. 🙂 Which is exactly why I use it. I always mean to get back into C++ some day, but the huge 8.02 Object Model (still up on the wall) puts me off. Since the cursor speed has been improved at 10.1 a script might be fast enough to be practical. The limitations I mentioned are only because of the shortcut to get the lastPoint for the line direction. You can eliminate these cases (if present) by exploding to a temporary layer. If the next vertex (instead of the last) was retrieved then it would handle multi-parts and another tweak is possible for the unusual case of imbedded parametric curves. Or just explode them too. The answer could be put back on the original featureclass. I will have to finish the snippet above to generalise it. ... View more

Intellisense is available for python modules in Pythonwin by default, but not ArcGIS tools. Are you sure that you can get intellisense for arcpy.* functions? ... View more

I just want to compliment Richards answer. When a python script is run from within ArcGIS as a scripting tool it's possible to call arcobjects within the calculate field tool. I've used this approach sometimes when I wanted to access the fine grade geometry tools that ArcObjects has. The QueryPointAndDistance method on IPolyline is certainly a very useful bit of wizardry I use it regularly! I did manage to get an example ArcObjects call running for 9.4 presented by Mark Cederholm, but everything has changed at 10. Could you post a snippet to give us a hint? It would be a lot easier to use an existing function. What a pity that the Near tool (and many others) is only a subset of ArcObjects functionality. Update Mark has updated his demo for version 10 at http://www.pierssen.com/arcgis10/python.htm ... View more

To get intellisense to work in Pythonwin, you have to check out a licence first. Check out a licence by going to the interactive window in Pythonwin (not the Python window in ArcMap) and type in import arcpy Then wait a few moments until you have a licence. At 10 if you only want to check out a lower licence in a mixed enterprise environment then type in import arcview import arcpy I do not know of a way of automating this each time you open Pythonwin, but maybe there is a way. Someone? ... View more

Get the site to zip up the folder first. Then download the zip. Or use the new package format (*.pkg) when exporting from ArcMap which is the same thing. ... View more

You can set a Spatial reference on the cursor options. Then when you get the x,y coordinates they will already be reprojected ready to add back into the fields. You don't have to reproject the featureclass first. For example I needed the extent of each polygon in a different projection:    #     # WGS_1984 works for coverages defined with an equivalent custom Transverse projection definition     gp.GeographicTransformations = 'NZGD_1949_To_WGS_1984_3_NTv2'    srOut = gp.CreateObject("SpatialReference")    srOut.CreateFromFile("c:/arcgis/nzmg.prj")    # this is a good on-the-fly switch for the searchcursor     # srOut must be a com object, not a file ref or a factory code     rows = gp.SearchCursor(ds+"/polygon",'"PAR_ID" > 0',srOut)     n = 0     rows.Reset()     row = rows.Next()     while row  :         print >> f1,"%d,%s" % (row.par_id,row.shape.Extent.replace(" ",","))         row = rows.Next()         n +=1     del rows ... View more

Six hours! :rolleyes:I hope you found the bottleneck. That would break my personal "Cup of Coffee Rule": "If any single process takes longer than a cup of coffee, interrupt it and find a better way". Since you are also running out of memory, finding what is causing that will probably speed things up enormously as well. I like TruncateTable_management. I note that there is no help in the Beta for this new utility Esri, ... You don't say how many records you have, but I would expect a couple of million records to take less than half an hour. Suggestions to find the problem, don't give up until you can have that cup of coffee while it is still hot: I immediately see a red flag where you open SDE which will inevitably be across a sloooow network, or at least it will be using sloow handshaking. Could you try loading into a filegeodatabase on a local drive and then copy the file geodatabase in a single step? Maybe FME (or Data Interoperability Extension) might be faster? Maybe load the shapefiles separately directly into a filegeodatabase and then use some SQL queries to do the selection and editing, instead of doing it all with the cursor. Very large records with many fields will be slow to load into a database. You could use MakeQueryTable to create a subset and then write out the view to SDE. Does the database have any indexes? You should drop the indexes when you truncate otherwise every insert will trigger a re-index. Does it get slower with more data. Try loading the first 10% to see if that is faster in proportion. Even though you are careful to trigger a garbage collection with del statements, it clearly isn't working. Have a look at your memory usage using Task Manager. If it keeps going up, then that might be the solution. If you can restructure the script to use a function, sometimes that garbage collects better. You might be more successful if you could batch the transactions. The cursors are a bit simple here. But FME can do this better. I find using Python and tools with more than a million records hits some sort of limit, even with my 8 CPU workstation. So I partition the work to use less than 1M records and it completes in a few minutes instead of never. Even for aspatial SQL queries. ... View more

I miss this function too, especially when attempting to port  AML code to  Python tools. Here is a python script  that  I called FrequencyWithCase to add back the functionality. It was not too hard, I can add the case ID as the records are read in a single pass updateCursor. # FrequencyWithCase.py # do it properly ESRI # 19 May 2010 # add a case item to the source table # like Workstation import arcgisscripting,sys,os try :     table = sys.argv[1]     strFields = sys.argv[2] except :     table = "e:/data/nzpost/paf2010/geopaf.gdb/pafpost"     strFields = "postcode;postcode_1" gp = arcgisscripting.create(9.3) casefld = "case" ws = os.path.dirname(table) tabname = os.path.basename(table) gp.Workspace = ws if len(gp.Listfields(table,casefld)) == 0 :     result = gp.AddField(table,casefld,"LONG")     print result.GetOutput(0) else :     # name = gp.AddField(table,"case1","LONG")     print casefld,"Exists" if gp.ListFields(table,casefld) == None :     raise Exception,"no case field" seqfld = 'sequence' if len(gp.Listfields(table,seqfld)) == 0 :     result = gp.AddField(table,seqfld,"LONG")     print result.GetOutput(0) else :     # name = gp.AddField(table,"case1","LONG")     print seqfld,"Exists" if gp.ListFields(table,seqfld) == None :     raise Exception,"no sequence field" # make a dictionary with a combo key # populate the source table as we go # export the dictionary at the end # make a tuple of all fields # check if its in dictionary # if it is increment counter # otherwise add new entry # retrieve number # put number into index or back on table # dFreq = {} lstFields = strFields.split(";") print lstFields cur = gp.UpdateCursor(table) row = cur.next() n = 0 ccase = 0 while row:     combo = []     for f in lstFields :         combo.append( row.GetValue(f) )     combo = tuple(combo)     if dFreq.has_key(combo) :         thiscase,freq = dFreq[combo]         dFreq[combo] = (thiscase,freq+1)         row.case = thiscase         row.sequence = freq+1     else :         ccase+=1         dFreq[combo] = (ccase,1)         row.case = ccase         row.sequence = 1     cur.UpdateRow(row) ##    if n > 1000 : ##        break     n+=1     row = cur.next() del cur,row print print "case freq" keytab = dFreq.values() keytab.sort() ##for x in keytab: ##    print "%4d %4d" % x print len(dFreq),"cases" # write out frequency table with case item fTab = tabname+"Freq" if gp.Exists(fTab) :     gp.Delete(fTab) gp.CreateTable(ws,fTab) gp.AddField(fTab,casefld,"LONG") gp.AddField(fTab,"Frequency","LONG") dscTable = gp.Describe(table) dType = {"String":"TEXT","Integer":"LONG"} for f in lstFields:     fType = dType[gp.ListFields(table,f)[0].type]     gp.AddField(fTab,f,fType) # may not be a "long" in general case     print f,"added to",fTab    cur = gp.InsertCursor(fTab) for k in dFreq.keys():     ## print dFreq [0],dFreq [1],dFreq     row = cur.NewRow()     row.case = dFreq [0]     row.Frequency = dFreq [1]     n = 0     for i in k:         ##print i,n,lstFields         fld = lstFields         row.SetValue(fld, i)         n+=1     cur.InsertRow(row) del cur,row print "Done" ... View more

You need to use Linear Referencing. Use an attribute that groups the polylines into the same general track and build a route system. Braided routes are generally difficult to analyse subsequently for paths because they are ambiguous, but it might suit what you are  doing. I have numbered the segments with an ID and also added a ROUTEID (10, 20) to use in CreateRoutes. ... View more

Maybe try IDW Inverse Distance Weighting in Geostatistical Analysis, 3D Analyst or Spatial Analyst You can set the max and min neighbours to get a circle of influence which could be turned into an index. ... View more

Have a much closer look at the help for SpatialJoin. You will see on the RHS that the input data types must be "Feature Layer" so you have to use MakeFeatureLayer_management() first to make layers from the featureclasses. PS Avoid using CalculateField in a script, use a cursor. The CalculateField tool just wraps a cursor around the expression, but you lose all control of the calculation. If you use a cursor you can test for unexpected data, such as Null values or zero and take action, print error messages and handle different cases. The expressions are also much easier to write. Leave CalculateField for Modelbuilder. ... View more

This fault happens after installing a service pack for ArcGIS. There is no need to set the PYTHONPATH variable 1. Save the numpy folder in c:\python26\Desktop10.0\Lib\site-packages, or reinstall it again later.     save the file from c:\python26\Desktop10.0\Lib\site-packages\Desktop10.pth 2. Remove Python and re-install python 2.6.6 in the standard location c:\python26 3. Reinstall Pythonwin with pywin32-216.win32-py2.6.exe 4. Copy the path file remaining that points to the arcpy module: from c:\python26\Desktop10.0\Lib\site-packages\Desktop10.pth to c:\python26\Lib\site-packages\Desktop10.pth 5. Reinstall numpy or move the numpy folder to c:\python26\Desktop10.0\Lib\site-packages 6. You can then delete the directory c:\python26\Desktop10.0 completely so that you only have one installation. Test to see if Pythonwin opens with the correct version. Try opening the Python Help. Test a script tool from ArcGIS using the os module. Maybe this will be fixed at 10.1? ... View more

Quite right to worry about the performance. You certainly don't want 10 million featureclasses output, that will kill any file system. The cup of coffee rule states: "If any single process takes longer than a cup of coffee then interrupt it and find a better way." Just don't run a geoprocessing tool inside a cursor. Why do you have to call it for each feature? Maybe just run the Polygon_Volume_3D tool once? ... View more