Latest Contributions by ChrisSnyder

This issue can happen if you create a feature layer (MakeFeatureLayer tool) from a a featureclass, add and populate a field to the featureclass, and then **for example** attempt to use the featurelayer as input to the Dissolve tool specifying the newly added field as the Dissolve field. The basic issue is that the feature layer is sort of like an in-memory snapshot reference to the featureclass. If the featureclass grows a soul patch and dyes its hair green, the feature layer (being a snapshot of sorts) is not aware of the change. To refresh the feature layer, just create it again (overwrite the old one) after you have added the field... or better yet, don't create it until your table schema is in it's final state. BTW: The issue can happen with a table/table view just the same. ... View more

I'll second that request. If you are willing to share, I'd also like to see that code too. ... View more

'Python Dictionary' is my answer to almost everything these days - It's an extremely powerful data structure for doing iterative-type analyses... as long as all your data fits. ... View more

If anyone is interested - here's an update of my os.spwanv psudo-parallel process method but now ***new and improved***using the subprocess module and better/fancier coding methods. It doesn't include any robust method (other than simple strings and return codes) of comunicating between parent and child processes but it could be altered to do that using the provided subprocess.Popen object (I guess via the .stdin method?). I'd be super happy to have someone smart show me how to elaborate on my simple example to include fancy stuff like handing numpy arrays to the child process or sending dictionary objects back to the parent script - something fancy like that! I am not there yet, but will probably have the need to do this eventually. For now it's a first stab and accomplishes what I need it to do... Here's the link: http://forums.arcgis.com/threads/33602-Arcpy-Multiprocessor-issues?p=177418&viewfull=1#post177418 ... View more

I finally got around to rewriting a psudo-parallel processing framework that uses the subprocess module instead of the old fashioned os.spawnv module. I couldn't get the multiprocessing module to work well for me, so I wrote my own code to do the same sort of thing: Regulate parallel processes. Note that I am not claiming this code is superior to the Multiprocessing module in any way (quite the oposite). However, since I wrote it, I undertand it, and that is priceless for me! I hope that others might find it usefull. I tried to provide usefull comments in the code. This is an updated (and frankly MUCH better) version of the code I had posted here:http://forums.arcgis.com/threads/33602-Arcpy-Multiprocessor-issues?p=116611&viewfull=1#post116611 Note the example parent and child scripts below don't use any ESRI objects, but they certainly could. In the parent script, all the subprocesses are conveniently tracked in a dictionary consisting of: jobDict[jobId] = [applicationExePath, childScriptPath,[list of input varibles], status {"NOT_STARTED", "IN_PROGRESS", "SUCCEEDED", "FAILED"}, subprocess.Popen object] If you can make this code better in some way or have an idea to make it better, please post it! PARENT SCRIPT EXAMPLE: #Import some modules import os, random, subprocess, sys, time #Process: Define a function that will launch the processes and update the job dictionary accordingly def launchProcess(jobId):     global jobDict #make this a global so the function can read it     inputVar1 = jobDict[jobId][2][0] #Input variables are being read from the job dictionary     inputVar2 = jobDict[jobId][2][1]     jobDict[jobId][4] = subprocess.Popen([jobDict[jobId][0], jobDict[jobId][1], str(inputVar1), str(inputVar2)], shell=False, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)     jobDict[jobId][3] = "IN_PROGRESS" #Indicate the job is 'IN_PROGRESS' #Determine how many processes to run concurrently numberOfProcessorsToUse = 3 #The number of processors you (the user) wants to use if numberOfProcessorsToUse > int(os.environ.get("NUMBER_OF_PROCESSORS")):     numberOfProcessorsToUse = int(os.environ.get("NUMBER_OF_PROCESSORS")) #Populate a "job dictionary" to keep track of all the subprocess jobs and all their various inputs and outputs childScriptPath = r"C:\csny490\simple_subprocess_child.py" jobDict = {} for jobId in range (1,21): #this loop is just showing how you can populate the jobId's and their input variables - in this example theere will be 20 jobs     inputVar1 = random.randrange(1,6) #this variable tells each subrocess how many seconds it will "sleep" for - between 1 and 5 seconds     inputVar2 = random.randrange(2,9) #in the child proces script, if inputVar2 > 6 it will throw an exception, and the subprocess will be 'FAILED' - I simply did this to create the possibility of failed subprocesses     #Format of jobDict[jobId] = [applicationExePath, childScriptPath, [list of input varibles], status {"NOT_STARTED", "IN_PROGRESS", "SUCCEEDED", "FAILED"}, subprocess.Popen object]     jobDict[jobId] = [os.path.join(sys.prefix, "python.exe"), childScriptPath, [inputVar1, inputVar2], "NOT_STARTED", None] #Process: Kick off some processes, monitor the processes, and start new processes as others finish kickOffFlag = False #Indicate the process kick off has not yet occured while len([i for i in jobDict if jobDict[3] in ("SUCCEEDED","FAILED")]) < len(jobDict):     if kickOffFlag == False:         while len([i for i in jobDict if jobDict[3] != 'NOT_STARTED']) < numberOfProcessorsToUse:             launchProcess([i for i in jobDict if jobDict[3] == 'NOT_STARTED'][0]) #Feed the appropriate jobId to the launchProcess() function         kickOffFlag = True #Set the flag as True once we have done the initial kickoff     for jobId in [i for i in jobDict if jobDict[3] == 'IN_PROGRESS' and jobDict[4].poll() != None]: #if an subprocess is listed as 'IN_PROGRESS' and polls as 0 (i.e. "done" but success or failure unknown)         if jobDict[jobId][4].returncode == 0: #return code of 0 indicates success (no sys.exit(1) command encountered in the child process             jobDict[jobId][3] = "SUCCEEDED"         if jobDict[jobId][4].returncode > 0: #return code of 1 (or another integer value) indicates failure (a sys.exit(1) command was encountered in the child process)             jobDict[jobId][3] = "FAILED"         if len([i for i in jobDict if jobDict[3] == 'NOT_STARTED']) > 0: #if there are still jobs = 'NOT_STARTED', launch the next one in line             launchProcess([i for i in jobDict if jobDict[3] == 'NOT_STARTED'][0])         print "--------------------------------------"         for jobId in jobDict:             print "Job ID " + str(jobId) + " = " + str(jobDict[jobId][3])         CHILD SCRIPT EXAMPLE: try:     import sys, time     var1 = int(sys.argv[1])     var2 = int(sys.argv[2])     print "Nothing to do, so sleeping for " + str(var1) + " seconds..."     time.sleep(var1)     if var2 > 6:         test = "oh crap" + 1 #concatenating a string an int here is intended to throw an exception on purpose to demonstrate a failure     print "Epic Success!"     #Note: If script runs through without error, return code will be, by default, 0 - indicating success except:     print "Epic Fail!" #Note: You can gather/parse all these print messages in the parent scrip using jobDict[jobId][4].communicate()     sys.exit(1)  #Note: If script fails, return code is 1 - indicating failure - you can set the return code to be > 0 if you want... ... View more

What is your memory consumption upon the 899th loop? Perhaps you are running out of memory. ... View more

Anything is possible... You could conceivably accomplish this via Python scripting. Feature Shifting: I don't think there is an out of the box way to do this, although you could use a "false easting" and "false northing" in a custom projection file to do this. Also, I have seen scripts posted on the forum that shift features by a systematic offset (but can't seem to locate them right now). The basic idea is to read through all the feature geometry and then write out some new features where all the vertices have a shift in their x/y coordinates (say like 20 feet to the north, 37 feet to the east). Feature Rotating: Again, I don't think there is an out of the box tool to do this (other than the rotate tool available via an edit session). You could write some Python code to do this... although the math for doing that is above my immediate geometry skills for sure... There is a toolbox tool for rotating rasters: http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//00170000007s000000 and I think there is some ArcObjects-type methods for handling vector rotation (like the edit sesion tool). This might be kind of hard in Python - and maybe better for ArcObjects if indeed there is already a method to do feature rotation there. So I assume this is some sort of monte carlo optimization you are trying to devise? http://en.wikipedia.org/wiki/Monte_Carlo_method I built a version of a monte carlo-based "optimizer" for estimating the number of animal territories (northern spotted owls) that could be supported given different scenarios about forest conditions over time. Basically (like what you are trying to do), it just tries a bunch of random permutations and builds up distribution of the results. It doesn't really seek to optimize anything exactly, but rather it generates a distribution of "likely outcomes". Some outcomes of course ended up with more territories than others - I guess you could think of more territories as being an optimization, although that's not what the project was really about. In retrospect, it was a pain and took me a long time to program, but in the end it was pretty cool and I learned a lot from doing it. That said, I don't think you would be able to effectively do this kind of thing without knowing how to program. As far as I know, there are no "out of the box" tools for this sort of spatial optimization. Network Analyst is really a tabular optimization (although there is a spatial component sort of) and uses this sort of method I think: http://en.wikipedia.org/wiki/Linear_programming ... View more

As soon as I apply more than one core, the processing is not stable any more Are you sure you aren't running out of memory? I have notice that a lot of the geoprocessing tools in ArcGIS v10 are far more "memory hungry" than in v9.3 (SelectByLocation and SpatialJoin being some examples). Which is probably why these tools in v10 appear to run faster... When an arcpy seasoned python.exe process runs out of memory while running a geoprocessing task, it tends to thow a rather obscure and unexpected errors in tools that otherwise (when run by themselves) run just fine. I think many times these seemingly random failures are actually just a reflection of if/when the individual processes have gone over their memory limitations. I haven't tried it myself, but if you have gobs of memory available, you can configure ArcGIS v10.0 to use more memory than the conventional 2.1 GB (which is actually more like 1.4 GB with overhead) - up to ~4GB orf RAM if running a 64-bit OS. An excellent post on the topic: http://forumsstg.arcgis.com/threads/26863-Memory-amp-CPU-settings-for-ArcGIS-10?p=150304&viewfull=1#post150304, which I am probably going to get around to trying this week. ... View more

Unfortunatly, the Dissolve tool in ArcGIS always dissolves the geometry into Multipart features and then (if you want) breaks it into Singlepart features - which works properly "most of the time". To be sure, always run the MultipartToSinglePart tool on the Dissolve tool output. So more direct to your issue.... As you disscovered, the Dissolve statistics always describe the Multipart shape  - not (like you might think) the derived Singlepart shapes. There is a work around though. The idea being that if you can tag yourt input features with a sort of unique spatial ID and Dissolve by this derived ID value, then your statistics come out the way you envisioned: describing the single part features. See attachment for latest rendition of some Python code that I wrote that does just that. ... View more

Check out this post: http://forums.arcgis.com/threads/50658-Iterate-with-leading-zeros The field calculator expresion equivalent should be: "str(!ORIG_MONTH!).zfill(2)" ... View more

Maybe a newbie question, but i didn't find any solution yet - Is it possible to add field into fc (with fieldName as a variable) and the to use something like updateRow.fieldName? (if using posted script, it is necessary to have "pre-added" fields - it would be much easier to add them in the beginning of the script...) Thanks! If the field is a variable, you have to use the .getValue (for read access) or .setValue (for write access) methods. For example: field1 = "FIELD1" field2 = "FIELD2" updateRows = arcpy.UpdateCursor(fc) for updateRow in updateRows:    field1Value = updateRow.getValue(field1)    updateRow.setValue(field2, field1Value / 2)    updateRows.updateRow(updateRow) del updateRow, updateRows ... View more

Very nice - This is actually quite usefull to me - I have done some projects in the past doing network anaylysis on logging roads... And I was looking for a way to apply some factors that would describe how "difficult" the road is to manuver (road type as well as each segment's vertical/horizontal complexity). When I get back to my routing project (months away), I'll try to post some code that uses z values as well. Thanks for posting the code... ... View more

I think you can do this using string literals: http://docs.python.org/release/2.5.2/ref/strings.html Somthing like: file = r"C:\temp\test\my cow is really a cow" + '/' + "horse" >>> print file 'C:\\temp\\test\\my cow is really a cow/ horse' Windows won't let you name a file with a / or \, so I couldn't even test it... Just an observation, but it is plain idiocy to name a file with a forward or backslash. As a "real user" of their data, you might have some good sway calling them up and complaining about their stupid file name... ... View more

Still doesn't get rid of the need for the conditional statement though... That said, I don't think there is a way to "directly" get the path you are looking for. This might be handy: http://en.wikipedia.org/wiki/Environment_variable#Default_Values_on_Microsoft_Windows ... View more