Latest Contributions by KimOllivier

‎02-23-2011

I agree 5 hours is far too long. I have a rule of thumb that states "If a single process takes longer than a cup of coffee, then interrupt it and find a better way". I will not be able to produce benchmarks with times longer that a cup of coffee, life is too short. I now remember my recent 'better way' was to use a Workstation intersect which only took a few minutes, I have to admit that an ArcGIS intersect failed first on the same data and since I have a better way I was quick to take that rather than research the issue. I was already using a file geodatabase that is indexed and cleaned. It was already on a local high speed disk drive. It failed quickly. The idea of looping through the features using a script seems likely to take a very long time because of the overhead of starting an intersect process for each likely polygon/polyline combination. You would have to search to find which combinations to compare and this needs a spatial index to be efficient but even then it would be slow. Python is supposed to be a 'glue' of fast optimised tools, not the tool itself. So, alternative tools seem the best course. I have tried another intersect in ArcGIS 10.0 SP1 on an Acer laptop with Windows 7, i5 Intel 4 core processor, 4GB of memory Polygons 42,496, polylines 221,896, tolerance 1 metre, time 5min 43sec. Nothing wrong with that. So to your data not performing. Clean the layer, maybe generalize a little for the purpose of the overlay. Are they both in a local projection? In a local file geodatabase? Get out of SDE. On a local disk? Output to a local file geodatabase. Strip off unnecessary fields. Is your scratch workspace a file geodatabase (not just a folder)? Do you have any unusually large features? Split them up. Multi-parts are not good for geoprocessing, split into singleparts.

‎02-22-2011

It looks like a TIN would do the job. This creates lines from polygon centroids to adjacent points. You can easily calculate the bearing of each line using a Python function math.atan2(). You may not have the 3D module, so an alternative may be ET Geowizards that can create a TIN.

‎02-22-2011

This is not the way to make a subset of features, although with some editing and additions it would be possible. You cannot get a geometry object from a cursor without extracting each vertex to make a new object, you do not need to get area and perimeter because they are geometry properties. MakeFeatureLayer_management will create a layer using your definition similar to the cursor, but it will be an in memory bitmap that you do not need to access. Copy_management will take the featurelayer and create a new featureclass of the subset. import arcgisscripting, sys gp = arcgisscripting.create(9.3) getzip = sys.argv[1] gp.Workspace = "C:\\Student\\PYTH\\Database\\MySD.mdb" gp.MakeFeatureLayer_management ("Zipcodes", "Ziplayer","[ZIP] = %d" % int(getzip)) gp.Copy_management("Ziplayer","Polyblank")

‎02-22-2011

50K polygons? Surely not too large, I regularly run intersects with much larger numbers, 100 times larger. If it is too large then partition the dataset into smaller sets with a selection or definition first and combine the results. At 10.0 I understood that the partitioning is automatic.

‎02-13-2011

You would think that you would extract a geometry object using a cursor, but it doesn't work for some obscure reason. You have to extract out all the vertices, put them in an array and rebuild a geometry object array. Much too hard. If you copy the entire featureclass into a geometry object and loop through that array of objects then it will work as expected. They are examples in the 10.x help but they skip any advice on the appropriateness of the function for geoprocessing. You really don't want to "step through a featureclass applying a tool". The tools are designed just like SQL operators to work on a whole featureclass with an implied loop. Besides, a geometry object is stripped of its attributes, it's not a full feature. Just consider how we might have managed before 10.0 without geometry operators. Think of the tool processing the whole featureclass and then analyse the results with another tool. ModelBuilder is supposed to lead you in this direction. Although ModelBuilder can emit a script, it does not make very readable scripts, but at least it avoids cursors for routine geoprocessing tasks.

‎02-13-2011

The strange location of Python doubly installed by ArcGIS must have something to do with it. I don't like having a second copy of Python that Esri installs, sure to end in tears! I delete the copy and leave or put Python in the conventional location. (I keep the Desktop10.pth file and the numpy folder from site-packages that Esri adds, uninstall Python and reinstall manually in the conventional location. Then I copy back the two saved settings.) C:/Python25 or C:/Python26 is the Proper Place for Python IMHO. It is then very predictable for other installed packages and editors. Everything then works properly, especially Python Help files. There is no DOT in the path so that dirname and basename are easier to parse. Fixing that would be my wish for a service pack, but I doubt if it will happen because this was raised during the Beta testing. I didn't see any valid reason given to do it that way and have had nothing but trouble since with any installation configured like that if you are going to extend Python or do development.

‎02-13-2011

I have a singlepart featureclass of polygons (2.5M) with an indexed key that I want to convert to multipart polygons. There are less than 0.5% (12,480) that are actually multipart, the rest are single parts. They are all reasonable multiparts, nothing silly that spans the whole dataset, no universal polygon or unreasonably large polygon such as a huge road casing. Since there are only a few to process I thought: "Aha, select the polygons with multiple keys, process these and merge them back with the remaining singles". That dissolve of the subset does complete in human time. But I can't complete the process because merging single and multipart features is not permitted. Who knew that? I thought multiparts were to be avoided for analysis, now I know. Is there a way of merging multiparts without rewriting the entire tool in Python? I didn't think that multipart was a different feature type from single part. I had thought that a single part feature was just a multipart with one part. Is that not so? The simple way (Dissolve_management) on the whole featureclass takes hours for sample sets (and fails to complete when it runs out of memory on the full set), giving me time to write this... ah here it is: 'unable to read memory, crashed'. I should expect this by now, nearly any process on a large dataset seems to run out of memory far too often. Just a simple relate with a selection (keyfile select) takes hours just to select and fails to write a subset, compared to Workstation which takes seconds. I have tried using a Python set and creating an SQL query as a layer but this is also very slow when it comes to writing out the records. Maybe I should just write out the records when running a cursor to test the key against the set and write them out manually? But that is rewriting the tool which I hoped to avoid. I am using local file geodatabases and 9.3, Duo Quad (8 processors) and 4 GB of memory, TB disk. Any other ideas?

‎02-12-2011

How many different layouts do you need for one report? I can think of two. Portrait and landscape. I know everyone expects to have multiple layouts like ArcView 3, but there are equivalents. You can export pages into one print document at the end. Most of the variations can be handled with dynamic text element in an index map without programming and you can export all the pages into a single PDF document. If you want more variations than the basic dynamic pages you can program the layout. Within a layout you can adjust all the major elements, change text and move spare elements stored off the page on to the visible printable area with a bit of python. You can have multiple data frames in one layout. I can't see any issues with your workflow, but it is a bit sketchy. I suggest that you may need to just rebuild it and automate some parts a bit better and you surely would be able to avoid 35 MXDs. I have never had more that two. Maybe post a thumbnail of an example map/diagram?

‎02-12-2011

Don't need next() ? Yes you do, in anything but a SearchCursor because you need access to a row object. How would you do cur.insert(row) and cur.update(row) ? If your 9.3 script is working, why bother rewriting it? The arcgisscripting module is still there and it works just fine. Maybe write new scripts using arcpy, and leave the rest alone.

‎02-08-2011

First you must get the correct version of GDAL for Python 2.5 Then the install notes tell you to add the directory to the search path PATH in Windows. You cannot add path names using import. Here is the download page and instructions http://pypi.python.org/pypi/GDAL/

‎02-02-2011

It seems that geometry objects are not being properly returned from the cursor. If you create a geometry object and populate a geometry list then the comparison can work. In my test of 4 lines, 2 and 3 cross and 1 and 4 have a shared end. Only crosses() produced a True. A geometry list is not ideal because there are no attributes, but it should be a clue for your problem. # different method import arcpy g = arcpy.Geometry() g1 = arcpy.Geometry() g2 = arcpy.Geometry() gList = arcpy.CopyFeatures_management("c:/data/sample.gdb/exlin", g) g1 = gList g2 = gList n=0 for f1 in g1: n+=1 m=0 print for f2 in g2: m+=1 print n,m,f1.overlaps(f2),f2.crosses(f1) 1 1 False False 1 2 False False 1 3 False False 1 4 False False 2 1 False False 2 2 False False 2 3 False True 2 4 False False 3 1 False False 3 2 False True 3 3 False False 3 4 False False 4 1 False False 4 2 False False 4 3 False False 4 4 False False I don't particularly like this idea, it seems to be reinventing GIS in Python scripts. Would it not be better to use topology if you have a large dataset?

‎01-19-2011

@csny490 I believe that the benevolent dictator for life is recommending that you not use the string module. You can now use the string methods to accomplish most of what I believe you are trying to do, for instance I could not find any reference or discussion in the PEP's suggesting the string module will be deprecated. I did see an uninformed speculation in the year 2000 at version 1.5 thinking that it may happen after string functions were built-in. It is still there in 3.0, stonger than ever with the new string Template functions. http://docs.python.org/py3k/library/stdtypes.html#string-methods Since looking up the help I did find a better string function than string.capwords(<name>) called <name>.title() which does the same thing as capwords, but defines words as contiguous letters, so apostrophies and dashes are also counted as word boundaries (unlike capwords). This avoids using regular expressions. The help does give an elegant example to handle special case exceptions using a generalised regular expression. Therefore my one-liner can be even simpler and now handles name punctuation as is required for some strings such as roadnames. No need to import the string module even. gp.CalculateField_management(fc,"name","!name!.title()","PYTHON") Note that I do not have to handle variable numbers of words in the string, and there is no need to define a function in a block. At Version 10 you can use Python expressions in the interactive field calculator to replace VB expressions so this would be very easy. You will have to expand your comments regarding returning entire strings, I don't want the strings returned and they are not, they are written directly into the featureclass if you use CalculateField. "Letting the geoprocessor interpret code"? I have no idea what you are getting at.

‎01-16-2011

Seeing a geoprocessing tool inside a cursor always gives me the shivers. This looks like reinventing GIS in a Python script with an unnecessary loop. Maybe like emptying the ocean with a thimble. It surely will be very very slow calling union for a thousand separate features, and I have trouble understanding the need to do this. ArcGIS is a geo-relational system, at least it was before ArcObjects. The mindset is to find a tool that processes the whole dataset with an implied loop over the whole featureclass. Try to think of a problem in the same way as an SQL query. You don't usually open a cursor there. If the sources are suitably tagged, then a union between the two featureclasses will do all the unions in one step without a cursor.

‎01-16-2011

I can see that you are not working through the quote characters in a logical way. You cannot hide the double quote string in the middle of a double quoted string with single quotes. You have to use the wonderful escape character (\). So your calculate expression must be like this: gp.CalculateField_management(fc + "\\Parcels.shp", "!SIT_FULL_S!.replace('\"','')", "PYTHON", "") Just visually "unpack" the string to see what is happening. Even better make a separate expression string and print it out to see if you get what is expected. expression = "!SIT_FULL_S!.replace('\"','')" print expression gp.CalculateField_management(fc + "\\Parcels.shp", expression, "PYTHON", "") the expression without the backslash would be: "!SIT_FULL_S!.replace('" ...followed by a syntax error: ',")" - an unclosed single quote which you don't find because CalculateField does not run TabNanny before it attempts to run it. So it does not compile and "doesn't work". Actually this is picked up by TabNanny. I recommend that you use it in Pythonwin to trap this sort of syntax error. It is the little tick icon. This is why it is not a good idea to be sucked into using CalculateField instead of a cursor. Use a cursor in scripts. Leave CalculateField for Modelbuilder. With a cursor you can pre-check the syntax with TabNanny (I hope you always do that!), you can expand out the steps to make it clearer, you can test for unexpected data and correct it, and you can add a try/except block to trap more unexpected errors without stopping the whole process. you can print out current variables when it crashes to see what the faulty data was.

‎01-16-2011

Someone else just asked me the same thing. How about this one-liner? gp.CalculateField_management(fc,"name","string.capwords(!name!)","PYTHON","import string") (You have to import the string module because it's not a built-in string function.) There may be other exceptions such as dashes and apostropies that need Uppercasing such as O'Brien or other Double-Banger names. In this case use a cursor and regular expression import arcgisscripting import string import re gp = arcgisscripting.create(9.3) # settings fc = "d:/work/test.gdb/trail" # simple way but inadequate gp.CalculateField_management(fc,"name","string.capwords(!name!)","PYTHON","import string") print gp.GetMessages() # Handle imbedded quotes like O'Brien quoteabbrev = re.compile("'[a-z]") # just as fast as a calculate and we can trap exceptions, errors cur = gp.UpdateCursor(fc) row = cur.next() while row: x = string.capwords(row.name) if quoteabbrev.search(x): print x,name name = quoteabbrev.sub(quoteabbrev.search(x).group(0).upper(),x) cur.updateRow(row) row = cur.next() del row,cur print gp.GetMessages() I never got the VB equivalent ProperCase to work in a script.

Online Status	Offline
Date Last Visited	Friday

My Ideas

Latest Contributions by KimOllivier

Re: calculating polyline length in polygons

Re: determine quadrant (north, east, south, west) of adjacent polygons

Re: gp.InsertCursor method (ArcGIS 9.3)

Re: calculating polyline length in polygons

Re: geoprocessing tool output to geometry object in for loop

Re: "No module named os" when running Python Script in ArcGIS

How can I speed up a simple dissolve?

Re: Beta 10: Multiple Pages in a Layout

Re: Converting scripts to arcpy for use in ags 10

Re: How can I import gdal in python25?

Re: Can't get geometry.overlaps etc to work with Polylines

Re: Using Calculate Field in python?

Re: Selecting a Feature then Processing

Re: Deleting double quotation " in field, find/replace errors - script

Re: Using Calculate Field in python?

Re: Difference between feature iteration and Row i...

Re: Adding WFS to webmap using Python API

Re: ArcGIS 3.6.2 Pro Not Closing

Re: ArcGIS for Personal use new licensing from 202...

Re: How can I use OGC GeoPackages in ArcGIS Pro 2....