Latest Contributions by KimOllivier

I have always found that there are very small practical limits of tiles when using Mosaic. I have much more success by merging strips (or a box) of tiles, then merge the strips, then merge the final set. It all sounds a bit messy, so that is why I script the process, and it helps if the tile names have a pattern. Max count < 10 at a time. In the end I have a grid of 300 tiles that performs surprisingly well. For more a more general approach to the "13 hours" I use these strategies: Apply the "Cup of Coffee Rule". If a single process has not completed in the time to have a cup of coffee, then interrupt the process and find a better way. That might be a different tool, or different software even... I find that nothing of interest results leaving processes running too long, they are corrupt or incomplete because the system has run out of memory, disk space or otherwise and is useless. My suggestions are: Make sure your PC is tuned: plenty of local disk space, defragged, and a temp workspace defined as a file geodatabase (the default in ArcGIS 10). This ensures you have enough space for temporary files that are really large in raster processing. Do some calculations on the 'processing effort' if you can. Esri staff suggest you try a sample first, but that does not really tell you if the process will scale for larger sets. So maybe just tile the data and do it in pieces if a small part works in "Coffee Time". Have a close look at your data. Unexpected data is often the problem, conflicting projections, null values, missmatch of cell sizes, default tolerances, and many more. Check vectors are clean, and that you have all indexes set, use file geodatabases, not shapefiles. Make sure you do not have single large polygons covering the area that you are using for processing. Dice them up, there is a new tool for that. For mosaic, just calculate how large the target tile will be with 133 tiles and multiply up to see how much memory that will take uncompressed. Probably more than Amazon's servers. Basically ArcGIS will not warn you are attempting something impossible, so you have to be wary. ... View more

Web Mercator? Isn't this just a sop for a simple sphere that G**gle insists that must be used to map the world? Its not a proper projection because it has no datum. I think that is why there is no projection using a cursor option. Try pretending it is WGS84 and suck hard on the registration errors, which is evident on every road network overlay of the images in GE or GM. I use the projection option on cursors and it works for me. I can even get the transform between two datums to work as well if I match the projection names exactly to suit the transform. I could not find an easy way to project a shape object. I had to write out a temporary featureclass to the scratch workspace, not in_memory. (I got the shape object from an interactive pick using a tool, and then I needed the pick coordinate in lat/long to query a server.) ... View more

The best way to load CSV files is to define the schema in schema.ini in the same folder. This is by the Microsoft Text File driver that many Windows applications use including ArcGIS. It has no limits on file sizes and many other benefits. You can define the field separator, rename the field names, set fields types and text widths. If the first row has field names you can use them, skip them or rename them. It is very flexible and fast. I have only recently discovered how useful the schema.ini definition can be. It saves having to write a Python script to parse and load the data into a table. You can just add the CSV file and export it to a geodatabase so you can edit and use it. http://webhelp.esri.com/ARCGISDESKTOP/9.3/index.cfm?TopicName=Accessing_delimited_text_file_data If you are going to create an XY event table, be aware that this is in memory and may be very slow converting it to a featureclass, or it may even fail. For very large files it may still be better to use an insertCursor to write a point featureclass directly in Python. ... View more

You have to run it as a tool and add an output parameter. In the tool the parameter is set to DERIVED and in the script you set the parameter to the layer. ... View more

There is a developer sample written in VB6 that will export topological errors to a separate featureclass. This would help you find and edit the errors more easily. It is a DLL written for 9.2 but still works in 9.3. ... View more

Multipoint featureclasses are a separate class from point featureclasses. You can't 'leave the rest alone' in the same featureclass. It doesn't really matter, single points groups will be just the same as a normal point, after a dissolve the featureclass will be multipoint. The problem then is that the featureclass is incompatible with 'singlepoint' so you cannot merge or cut-and-paste, just as you cannot merge lines into polygons [unless you use Workstation]. A further issue you might be trying to solve is to aggregate the multipoints into a single point. This would require a different tool, which finds the centroid of the cluster to place a single point. ... View more

Yes, I see it there, it is what I use in 9.3. Since 9.3 locators are supported at 10, I assumed that this was there to support previous locators. We have a custom standardiser for 9.3 and its not clear how to create a custom standardiser at 10. I will have to investigate further. (I also see the help for it that you need an ArcInfo licence to use the standardiser, but the licence matrix contradicts that and has a tic for all licence levels.[ licence : noun, license:verb in English]) ... View more

ET Geowizards has a tool for this. http://www.ian-ko.com/ It costs a little but works really well for me and saves me a lot of coding. ... View more

I see you are using a cursor and looping through each feature to reconstruct a geometry object. At 10.0 you can read a whole featureclass into an array of geometry objects in one step without a cursor. You can also write out an array to a featureclass as well. That avoids the cursor both ways, but the disadvantage is that the attributes are separated. You don't say what you are doing with the featureclass in the DLL, I am curious what you have to do that is not already available in the geoprocessing tools. ... View more

There is a projection parameter option on the cursor objects that will reproject to new coordinates as you read the records. Getting a transform to work as well is a bit tricky, but basically set that as well as a string. # extent_xy.py # add extents of parcels in NZMG to a NZTM dataset for backward compatibility # update to use on the fly projection directly from NZTM layers # see Werner Flacke's book and script example # updated for 9.2 # Kim Ollivier 13 April 2007 # input NZTM coverage library, parcel and plabel coverages # output csv files to load and join back to coverages using AML # issues solved: #               add spatial reference to SearchCursor #               define a Geographic Transformation that works with a coverage #               because coverages cannot define NZGD2000 use WGS84 definition #               or the transform is ignored import os,sys,glob,arcgisscripting gp = arcgisscripting.create() def parcel(tile) :     print "Parcel",tile     ws = "e:/lib/nztm/tile/t"+str(tile)+"/data"     outfolder = "e:/lib/nztm/tile/t"+str(tile)+"/data"     ds = ws + "/parcel"     f1 = open(outfolder+"/e_parcel.txt","w")     f1.write("par_id,eminx_nzmg,eminy_nzmg,emaxx_nzmg,emaxy_nzmg\n")     gp.Workspace = ds     print ds     # this is a good on-the-fly switch for the searchcursor     # srOut must be a com object, not a file ref or a factory code     rows = gp.SearchCursor(ds+"/polygon",'"PAR_ID" > 0',srOut)     n = 0     rows.Reset()     row = rows.Next()     while row  :         print >> f1,"%d,%s" % (row.par_id,row.shape.Extent.replace(" ",","))         row = rows.Next()         n +=1     del rows     print n,"Polygons"     f1.close()     return def plabel(tile) :     print "Plabel",tile     ws = "e:/lib/nztm/tile/t"+str(tile)+"/data"     ds = ws + "/plabel"     outfolder = "e:/lib/nztm/tile/t"+str(tile)+"/data"     f2 = open(outfolder+"/e_plabel.txt","w")     f2.write("par_id,eminx_nzmg,eminy_nzmg\n")     gp.Workspace = ds     rows = gp.SearchCursor(ds+"/point",'"PAR_ID" > 0',srOut)     n = 0     rows.Reset()     row = rows.Next()     while row  :         n += 1         print >>f2,"%d,%s,%s" % (row.par_id,row.shape.Extent.split()[0],row.shape.Extent.split()[1])         row = rows.Next()     del rows     print n,"plabels"     f2.close()     return # ------ main ---- # srOut = gp.CreateObject("SpatialReference") srOut.CreateFromFile("c:/arcgis/nzmg.prj") # WGS_1984 works for coverages defined with an equivalent custom Transverse projection definition gp.GeographicTransformations = 'NZGD_1949_To_WGS_1984_3_NTv2' # ;New_Zealand_1949_To_NZGD_2000_3_NTv2' # but is this being used? inFC = "e:/lib/nztm/tile/t1009/data/plabel/point" # not at 9.3?? outFC = "in_memory/dummy1.shp" outFC = "c:/tmp/dummy1.shp" outPrj = "c:/arcgis/nzmg.prj" # cannot be a COM object or factory id 43040 geoTrans = "NZGD_1949_To_WGS_1984_3_NTv2" # ;New_Zealand_1949_To_NZGD_2000_3_NTv2" if gp.Exists("c:/tmp/dummy1.shp") :     gp.Delete("c:/tmp/dummy1.shp") # gp.OverwriteOutput = 1 gp.MakeFeatureLayer_management(inFC,"tlayer","PLABEL# < 3") gp.Project_management("tlayer",outFC,outPrj,geoTrans) try :     if sys.argv[1].upper() == 'ALL' :         lstTile = range(1001,1013)     else :         lstTile = [ tile for t in sys.argv[1].split(",")] except :     lstTile = range(1001,1013) print "begin extent_xy"    for t in lstTile :     print t     parcel(t)     plabel(t) ... View more

I find you have to export the unmatched addresses as a table, not a featureclass which has null shapes for the unmatched addresses. Open the table of the featureclass, make a selection STATUS = 'U' , hide the generated fields and export. (9.3, haven't tried 10.0) I wish we could just switch locators and rematch the filtered set, with a tag for which geocoder that matched the address instead of just an "A", it would be much tidier. You would think that you could change the "active" locator in the geocoding toolbar and that would then be used. I know that composite locators are designed to do this, but I need to pause and consider the errors, maybe do a global fix before running another locator. ... View more

This is very disappointing. I always standardize before geocoding to enable me to scan for global errors to be fixed before geocoding to improve the match rate, and even more important, to reduce false positives. I can populate missing fields from lookup tables, sort and scan for spelling mistakes, and diagnose reversed elements such as flat and street number. So now I will have to reinvent a standardiser using regular expressions in Python? It might therefore be a while before I switch to 10 locators. Maybe someone clever could generate a regular expression parser from the XML lot definitions? What do other geocoders do to pre-process the source tables? ... View more

The training course notes suggest that all the Reset methods are to be implemented "in the future". Meanwhile you have to close the cursor and reopen it to get the equivalent. ... View more

Kim, I don't seem to have any issues creating a FC with mixed states of single and multipart (made a singlepart dissolve, and then a multipart dissolve, and merge together - no error). What error do you get? I think maybe I'm not grasping your issue? Polygons are inherently multipart so the dissolved and undissolved subsets merged successfully. It applies to points. There is a Point featuretype and Multipoint featuretype which cannot be merged. I now have the difficult decision to have all parcel labels as multiparts or create two layers which then makes searching a single table, easy joins, counting all more difficult. I could abandon points and store the attributes on the polygons directly, but that has other inconveniences such as polygon label placement and text rotation is not possible. ... View more

Use a search cursor (read only) to get out the objectid and your counters into a dictionary. Then you can manipulate the list using direct tools on libraries. Finally open an update cursor and put back the changes. It is very hard to process using a cursor otherwise because you can only step through in one direction. ... View more