Latest Contributions by ChrisSnyder

Not sure I totally understand, but just specify the fields you want to keep in the Dissolve field list. ... View more

Peter - The basic method is shown here: http://forums.arcgis.com/threads/9555-Modifying-Permanent-Sort-script-by-Chris-Snyder?p=30010&viewfull=1#post30010 ... View more

if so, it might be more efficient to write your own code I thought about that some, but... I think that's why my employer pays ESRI like 100's of 1000's of $$$ a year... So I don't have to (or at least shouldn't have to!)... Sigh... Anyway, I've finally got this thing down now so that what used to take 2.5 weeks in v9.3 (single process, 6 year old machine) now takes ~6 hours (with x15 concurrent SA subprocess running on a MUCH faster machine). So I'm content for the time being... ... View more

Doesn't use wildcards exactly, but how about something simple like: urows = arcpy.UpdateCursor(surveyFc) for row in urows:     if 'C,' in updateRow.CODE         row.setValue("HOST", "Confier")     elif 'Hw,' in updateRow.CODE         row.setValue("HOST", "Hardwood")     else:         row.setNull("HOST")     urows.updateRow(row) del row, urows If you want to use "wildcards" in a cursor, I think you would want to use a regular expression-based method: http://code.google.com/edu/languages/google-python-class/regular-expressions.html ... View more

Not sure how all that hokey looking "\" syntax keeps getting propogated (oh yeah, model builder and the Help pages), but it is generally completely unneccessary. The wildcards of *, %, or ? depend of course on the database (and GIS data format) you are using... This "should" work: #Select anything with a 'C,' string patern in it (Assuming FGDB/SDE format) and make the HOST field = 'Conifer' urows = arcpy.UpdateCursor(surveyFc, "CODE LIKE '%C,%') for row in urows:     row.setValue("HOST", "Confier") #Note row.HOST = "Confier" also works     urows.updateRow(row) del row, urows ... View more

Hi Sebastian, That is a good idea to use the buffer geometry object instead. In my case I have a need to use those pnt buffer extents in several later scripts so I figured it would be faster just to build them once and reference those. The block idea is a good one too. I hadn't thought of that.... Maybe in v2.0... A simple way might be to cut my study area into quadrants (4 rectangles) and simply ID the pnt buffers that exist "entirely within" a quadrant polygon. Then select them four at a time (random selection of quad pairs QUAD_IDs composed of  1,2,3,4), and process each quad pair at once. For my purpose I note that limiting the extent makes it run about 10 times as fast as the entire study area. So it might take some fancier algorithm design... I really like your GRASS work around... Do you have any example script that make use of that? I'd love to see them. I have noted in the past that the ESRI cost<whatever> algorithms are only 8 direction which leads to some not-so-natural-looking outputs: http://forums.arcgis.com/threads/44464-Problems-with-cost-path-analysis-not-shortest-distance?p=151787&viewfull=1#post151787. It sure would be nice if they could include a Knights move option in the Cost* functions Hint, hint to any ESRI people that might see this... In my own expereince I have noted some funky behavior of random crashes in SA tools... in the past I was able to "fix" the problem by using only raster inputs... For example, converting the input point feature in CostDistance or Watershed to a raster 1st, and then using the raster version of the point as input (yes even if the dang point was smack in the middle of the pixel). An interesting note: I have this new fancy computer that absolutely screems (Xeon 2687 with SSDs in RAID0). Am am really excited to have this machine (and its 8 ,but effectivly 16, cores) so that I can finally relly leverage some subprocess scripting and SA functions (previous to v10 you couldn't run concurrent SA tools). I noted that my subprocesses (15 concurrent ones!) occasionally "collided" and would randomly crash and fail... Not neccessarily CostDistance... but any SA tool. Big let down after I thouight I'd be able to do all this fancy concurrent SA processing finally. I guessed that all the seperate SA processes must be *** &^%$ing STILL*** trying to write some scratch temp file somewhere (occasoonally all at the same time). So after some real hair pulling, I figured out that creating some dummy TEMP directories BEFORE launching the subprocess script completely fixed the problem - *** boy was I happy to get that thing working!*** So my point: Could it be that you have concurrent SA processes that are all wrtting to the same TEMP folder and occassionally "colliding"? If it helps, here's the latest incarnation of my "master" script that calles the scripts that calls the CostDistance building script as a subprocess : # Description # ----------- # This script is the master script that builds all the cost distance grids # Written for Python version: 2.7.2 and above (PythonWin) # Written for ArcGIS version: 10.1 SP0 # Fuzzy logic score scale of 0-100, 50 being neutral # Author: Chris Snyder, WA Department of Natural Resources, chris.snyder(at)wadnr.gov # Date created: 4/1/2010 # Last updated: 9/27/2012, csny490 try:     #Import system modules     import sys, os, time, traceback, subprocess     #Defines some functions     def launchProcess(jobId):         global jobDict         tmpDirName = r"C:\Temp\gp_tmp_" + time.strftime('%Y%m%d%H%M%S')         os.mkdir(tmpDirName)         os.environ["TEMP"] = tmpDirName         os.environ["TMP"] = tmpDirName         os.environ['TEMPDIR'] = tmpDirName         inputVar1 = jobDict[jobId][2][0]         inputVar2 = jobDict[jobId][2][1]         inputVar3 = jobDict[jobId][2][2]         jobDict[jobId][4] = subprocess.Popen([jobDict[jobId][0], jobDict[jobId][1], str(inputVar1), str(inputVar2), str(inputVar3)], shell=False)         jobDict[jobId][3] = "IN_PROGRESS" #Indicate the job is 'IN_PROGRESS'         time.sleep(2) #Give the subprocesses (especially arcpy enabled subprocesses) some time to catch up     def showPyMessage():         try:             print >> open(logFile, 'a'), str(time.ctime()) + " - " + str(message)             print str(time.ctime()) + " - " + str(message)         except:             pass             #Specifies the root directory variable, defines the logFile variable, and does some minor error checking...     root = r"C:\csny490\nso_model"     if os.path.exists(root)== False:         print "ERROR: Specified root directory " + root + " does not exist... Exiting script!";time.sleep(3);sys.exit()     scriptName = sys.argv[0].split("\\")[-1].split(".")[0] #Gets the name of the script without the extension      dateTimeStamp = time.strftime('%Y%m%d%H%M%S') #in the format YYYYMMDDHHMMSS     logFile = root + "\\" + scriptName + "_" + dateTimeStamp + ".log" #Creates the logFile variable     if os.path.exists(logFile)== True:         os.remove(logFile)         message = "Deleting existing log file " + logFile + "... Recreating " + logFile; showPyMessage()     #Process: Make sure the "cost_grids" dir exists...     if os.path.exists(root + "\\cost_grids") == False:         os.mkdir(root + "\\cost_grids")     #Process: Defines some variables     scenarioDict = {"NA": 'No Action', "LP": 'Landscape'}     decadeList = [0,1,2,3,4,5,6,7,8,9]     pathToScenarioDecadeGrids = root + "\\decade_scenario_grids"        #Determine how many processes to run concurrently     numberOfProcessorsToUse = 15 #The number of processors you (the user) wants to use     if numberOfProcessorsToUse > int(os.environ.get("NUMBER_OF_PROCESSORS")):         numberOfProcessorsToUse = int(os.environ.get("NUMBER_OF_PROCESSORS"))     #Process: Define the python.exe path and slaveScript path     childProcExePath = os.path.join(sys.prefix, "python.exe")     childScriptPath = r"\\snarf\am\div_lm\ds\gis\projects\oesf_eis_draft_second\nso_models\scripts\nso_model_scripts_v20120912\STEP5B_path_distance_slave_v101_20120912.py"     #Process: Loop through the decades and scenarios     jobDict = {}     for decade in decadeList:         for scenario in scenarioDict:             jobDict[(scenario, decade)] = [childProcExePath, childScriptPath, [root, scenario, decade], "NOT_STARTED", None]           #Process: Get those jobs going!     while len([i for i in jobDict if jobDict[3] != 'NOT_STARTED']) < numberOfProcessorsToUse:         launchProcess([i for i in jobDict if jobDict[3] == 'NOT_STARTED'][0]) #Feed the appropriate jobId to the launchProcess() function     while len([i for i in jobDict if jobDict[3] in ("SUCCEEDED","FAILED")]) < len(jobDict):         time.sleep(10)         for jobId in [i for i in jobDict if jobDict[3] == 'IN_PROGRESS' and jobDict[4].poll() != None]: #if an subprocess is listed as 'IN_PROGRESS' and polls as 'None' (i.e. "done" but success or failure unknown)             if jobDict[jobId][4].returncode == 0: #return code of 0 indicates success (no sys.exit(1) command encountered in the child process                 jobDict[jobId][3] = "SUCCEEDED"                 message = "SUCCESS: " + str(jobId) + " completed successfully..."; showPyMessage()             if jobDict[jobId][4].returncode > 0: #return code of 1 (or another integer value) indicates failure (a sys.exit(1) command was encountered in the child process)                 jobDict[jobId][3] = "FAILED"                 message = "ERROR: " + str(jobId) + " failed..."; showPyMessage()             if len([i for i in jobDict if jobDict[3] == 'NOT_STARTED']) > 0: #if there are still jobs = 'NOT_STARTED', launch the next one in line                 launchProcess([i for i in jobDict if jobDict[3] == 'NOT_STARTED'][0])     #Process: Final check for failures - if any, exit(1)     if len([i for i in jobDict if jobDict[3] == 'FAILED']) > 0:         message = "ERROR: These jobs were an epic fail:"; showPyMessage()         for jobId in [i for i in jobDict if jobDict[3] == 'FAILED']:             message = str(jobId) + " failed..."; showPyMessage()         sys.exit(1)     message = "ALL DONE!"; showPyMessage()     except:     message = "\n*** PYTHON ERRORS *** "; showPyMessage()     message = "Python Traceback Info: " + traceback.format_tb(sys.exc_info()[2])[0]; showPyMessage()     message = "Python Error Info: " +  str(sys.exc_type)+ ": " + str(sys.exc_value) + "\n"; showPyMessage()     sys.exit(1) ... View more

"a very large amount of the geoprocessing tools found within the ArcToolbox doesn't effectively utilise the processor or memory... This is a common trend among all the Geoprocessing Tools found within ArcToolBox" There are some notable exceptions, but I have generally found the oposite to be true. Most gp processes occupy 100% of a single CPU, and more or less, efficiently manage memory. In your specific case, it's probably more likely that there is something awry in your work flow. Could it be that you are reading/writting this data over a slow network connection? ... View more

and I have tried fixing it by adding cursor.reset() arcpy.CompressFileGeodatabaseData_managementgc.collect() but no luck so far. Did you mean to use Compact instead? Compress makes the FGDB theoretically smaller (although not for Raster datasets), and read only. Run the Uncompress tool to make it writable again. Maybe not totally related, but I am rewritting some v9.3 code to v10.1 that runs CostDistance in a loop many thousands of times... probably simliar to what you are doing. I'd enjoy swapping some performace best practices if you are willing... I like how you get the point geometry from the cursor use it as direct input to the CostDistance tool. Hope you don't mind I use it... Some tricks I have found: 1. Buffer your input points layer, so that you can set the analysis extent to the point's buffer polygon to limit processing time/extent of eachcostdistance surface. For example: arcpy.env.extent = arcpy.da.SearchCursor(nestSitesBufferFC, "SHAPE@", "POINT_ID = " + str(pointId)).next()[0].extent 2. Use Grid format! CostDistance runds way faster when using Grid format inputs and outputs. 3. Limit the number of grids in a single folder (this might apply to FGDB rasters as well?). I wrote some code that makes a new folder every 100th loop, which is where I noticed performnace taking a real nose dive. I suspect it has to figure out a scratch name every time, and the more rasters you have in a workspace, the longer it takes to come up with a unique scratch name since it has to list the contents every time. See: http://forums.arcgis.com/threads/67762-Spatial-Analyst-scratch-raster-naming-system-causing-slowness-in-v10.0-.sa-syntax ... View more

Did you accidently run the Compress tool on that FGDB (thereby making it read only)? FYI: Many raster functions (inclusing CostDistance) still operate significantly faster in GRID format vs FGDB raster format. ... View more

How can i represend in arcmap the value max? Use the SummaryStatistics tool: http://resources.arcgis.com/en/help/main/10.1/index.html#//00080000001z000000 ... View more

"I am new to spatial analyst and I need to create maps using regression in GIS would you able to help me" That is a pretty tall order there... but here's something to get you started: http://resources.arcgis.com/en/help/main/10.1/index.html#//005p00000023000000 You might want to check out these forums instead: http://forums.arcgis.com/forums/110-Spatial-Statistics http://forums.arcgis.com/forums/107-Spatial-Analyst http://forums.arcgis.com/forums/100-Geostatistical-Analyst ... View more

I guess this works, but it's not as pretty and takes longer making all the lookup scratch files. gridList = arcpy.ListRasters("nst_*") i = 0 for grid in gridList:     i = i + 1     tmpGrd = arcpy.sa.Lookup(grid, "NST")     tmpGrd.save("tmp_" + str(i)) gridList = arcpy.ListRasters("tmp_*") nstMaxGrd = arcpy.sa.CellStatistics(gridList, "MAXIMUM") nstMaxGrd.save("max_nesting") for grid in gridList:     arcpy.Delete_management(grid) ... View more

Followers of this thread may want to know that Esri plans to add 64 bit background processing as an optional additional install for Desktop 10.1 SP 1, which will let you run models and scripts in 64 bit. This could be pretty useful, especially for big geospatial data processes that can crash 32-bit geoprocessing because of memory requirements. Cool!!!!!! ... View more