Latest Contributions by PhilMorefield

‎09-28-2012

http://dusk.geo.orst.edu/gis/geostat_analyst.pdf The formulas for the crossvalidation summary statistics can be found in Appendix A on page 273 (page 279 of the pdf, since the first six pages aren't numbered). Perfect. Thanks Eric!

‎09-27-2012

As for which is better, it's really a judgement call. Personally, I still like the model on the left because both the root-mean-square and average standard error are lower than the model on the right. A large difference between the RMS and the average standard error can indicate model problems, but a root-mean-square standardized of .85 indicates that the problem is not severe in this case. And the one point on the x-axis of the LPI model is also concerning. When we change the graphic, we'll find an example where a lower RMS clearly does not imply a better model. Eric- Is there a document that shows the various Prediction Error formulae (e.g., Mean Standardized Prediction Error, Root-Mean-Square Standardized, etc.), as implemented by ESRI? I ask because different sources uses lightly different names for these metrics, and I know there can be subtle nuances to how they are calculated.

‎09-26-2012

The data you are trying extract by using getValue() is an array-like object (i.e., it's two dimensional). As you can see here, the getValue() method only works on scalars (e.g., an integer, a float, etc.). Consider doing this instead: # retrieve first time step var[0] # retrieve second time step var[1] Is your goal to convert these data to rasters? If so, I would skip writing them to csv and use the NumpyArrayToRaster() function. If what you want is a just a list of the points and values written to a text file, you'll need to look at reshaping the array-like object your data are currently in.

‎09-25-2012

I just did the procedure Phil provided and it worked well for me. However, why on earth can you not iterate through a list of rasters and have each one add together? I spent a good 10 hours trying to figure out why this would not work. Seems to me a very stupid limitation since this works for multiple other tools where you can iterate through lists of files. -Andy You can, it just isn't as clean: import arcpy, os arcpy.CheckOutExtension("Spatial") arcpy.env.overwriteOutput = True arcpy.env.workspace = r"C:\Users\Documents\GIS\Site Selection Example\Output Datasets\Cons_Raster" rasterList = arcpy.ListRasters("*") print rasterList sumEmpty = 'Yes' for rasName in RasterList: rasObject = arcpy.Raster(rasName) if sumEmpty == 'Yes': sumRas = rasObject sumEmpty = 'No' else: sumRas = sumRas + rasObject # Save the output sumRas.save("C:/data/sumRas.tif")

‎09-20-2012

Right-click on the tool, and select "Edit". You should see the script open up. On line 10 you should see this: gp = arcgisscripting.create(93) Change it to this: gp = arcgisscripting.create(9.3) That got the tool working for me, using the example data. I'm on AGD 10.1, Windows XP.

‎09-20-2012

The key piece of information is this: <type 'exceptions.AttributeError'>: 'list' object has no attribute 'reset' Here, 'attributes' is a property of a describe object. Apparently the geoprocesser used to return in iterator, but now it returns a Python list. Python lists don't have a 'reset' method. Thus the error. You'll need to attach the data you are using in order for someone to troubleshoot the tool.

‎08-13-2012

Even though I voted for some python modules to be included, what I'd really like to see is that ESRI work with Enthought to make sure there is a version of the Enthought Python Distribution that is compatible with each version of ArcGIS. That way, I can just use the 100+ libraries already provided in the EPD together with ArcGIS. This is already true in the latest Enthought EPD 7.3 and ArcGIS 10.1, both of which use Python 2.7 and Numpy 1.6. So everything works very nicely together. But how long will this very nice situation last? Let's make it last longer than just these current releases! -Rich Signell I couldn't agree more. Great idea.

‎08-13-2012

Another commenter suggested ESRI work with Enthought, Inc. to coordinate on an ArcGIS-compatible version of the Enthought Python Distribution (EPD) at every release of ArcGIS. This is a great idea, and would render polls like this unnecessary! 😉 I think there are a lot of questions on the forums that could be preemptively answered with code analysis tools, like pylint. How about including a free, lightweight Python IDE that already has pylint integrated? I know the ESRI folks are fans of Pyscripter, which is great. Spyder is another possiblity. IEP is another solid IDE, though it lacks pylint integration at present. I don't see much value in only bundling packages like gdal and shapely. 99% of the functionality in those modules is already included in ArcGIS. PySAL has some nice tools but, again, is already included in the EPD. As an aside, I see netCDF4 listed above. I've found this to be incompatible with ArcGIS 9.x and 10.0 sp5 on Windows XP 32-bit. Great thread!

‎07-26-2012

The first piece of advice I can give you is to forget rpy or rpy2. They haven't been supported on Windows for quite some time. Unless you are using a Unix/Linux operating system, you will be heading down a long and disappointing road. Of course, as soon as I say this someone builds a Windows installer for rpy2... As for implementation I would write an R script with parameters that are passed from Python. ESRI has an example calling a clustering script: http://resources.arcgis.com/gallery/...2-643E624A8925 I also stumbled across a very intriguing method for interfacing R and ArcGIS through Python that I have not tried yet. I attached the PDF. The PDF you attached is interesting; I hadn't seen that before. Both methods seem a bit klugey, in my opinion. But they are well documented, so those are definitely options. I guess it all depends what the purpose of the project is. Calling an existing R function is very different from programming the algorithm yourself! Fortunately, a true integration of R into ArcGIS is "under consideration" by ESRI. Fingers crossed.

‎07-25-2012

I doubt you will find a answer to your problem here. This is not an algorithm implemented in ArcGIS and requires some fairly complex statistics. The first piece of advice I can give you is to forget rpy or rpy2. They haven't been supported on Windows for quite some time. Unless you are using a Unix/Linux operating system, you will be heading down a long and disappointing road. You can probably find the statistical functions and distributions you need in scipy. Or you could certainly do everything using only R. You don't even need ArcGIS for this, except to make pretty maps. Have you seen this yet?

‎07-18-2012

I don't think the image you tried to attach is showing up. There are few things I would check. First, make sure all of your rasters have the same extent and projection. If any part of any raster has no data, your output for that cell will have no data. You could also set the processing extent to the union on all inputs, just to be thorough. I'm assuming you checked the value for 'rasterList' and all of the file names or included and look okay? As a matter of best practice, you shouldn't have spaces or special characters in file names or paths; you currently have some spaces in your folder names.

‎07-18-2012

You don't need to iterate through each raster in your list. You just need to provide the tool with the list of rasters one time, and it will add them together. import arcpy, os from arcpy import env from arcpy.sa import * arcpy.CheckOutExtension("Spatial") arcpy.env.overwriteOutput = True env.workspace = r"C:\Users\Documents\GIS\Site Selection Example\Output Datasets\Cons_Raster" rasterList = arcpy.ListRasters("*") print rasterList # Execute CellStatistics outCellStatistics = CellStatistics(rasterList, "SUM", "NODATA") # Save the output outCellStatistics.save("C:/data/sumRast.tif")

‎07-16-2012

Chris- After looking over your script and the Daymet files, I'm going back on what I said. Going outside of ArcGIS really is the best solution (IMHO) for bulk processing netCDF data, but would require more background and a more complex script to handle all of the calendar/date information. To start with, keep an eye on the syntax for the arcpy.MakeNetCDFRasterLayer_md. I've highlighted a change in blue that makes the input match the documentation. But when you export to Python snippet, it shows the syntax as you already had it. I'm not sure which one is 'correct'. Here's what I think the problem is: you're creating a new Raster Layer ('prcp_Layer') at each iteration, but never deleting them. I think ArcMap and/or your RAM are getting clogged up with files in memory, which is all a Raster Layer is. So your original hunch may be correct! I've made some changes in red to your original script which tidies things up. I've also put everything in side of a try/except clause, so that we can get more information if you continue to have problems. #Convert NetCDF files to Imagine img files #Jason Geck jgeck@alaskapacific.edu #Make a separate raster *.img file for each dimension value (time) in a netcdf file # 2012.07.03 Modified by Sergio Bernardes/Chris Strother import arcpy, os, time, datetime, calendar, traceback, sys # Set local variables arcpy.env.overwriteOutput = True arcpy.env.scratchWorkspace = "C:\\Scratch\\" path = "C:\\PRCP\\" path_rasters = "C:\\PRCPRasters\\" extension = ".nc" dirList=os.listdir(path) #Loops thru all days of year (including leap days) #Year info Yr = range (1982,2008) #Month info allmnths = range(1,13) try: for fname in dirList: if fname.lower().endswith(extension): tile, Yr, datatype = fname.split("_") intYr = int(Yr) #print tile, yr, datatype for mnths in allmnths: Lastday = calendar.monthrange(intYr, mnths)[1] MRange = range(1,Lastday+1) for dyy in MRange: dyys =int(dyy) mnthsss = int(mnths) yrs = int(Yr) stringMonth = str(mnthsss) stringDays = str(dyys) a = stringMonth.zfill(2) + "/" + stringDays.zfill(3)+"/"+Yr b = stringMonth.zfill(2) + "_" + stringDays.zfill(3) +"_"+Yr print a,b inDate = "time " + a prcp_nc = path + fname prcp_Layer = "prcp_Layer" + b test_img = path_rasters + tile + "_" + b + ".img" # Process: Make NetCDF Raster Layer arcpy.MakeNetCDFRasterLayer_md(prcp_nc, "prcp", "x", "y", prcp_Layer, "", [["time", inDate]], "BY_VALUE") print "Created NetCDF Layer for " + prcp_Layer temp = arcpy.Raster(prcp_Layer) temp.save(test_img) arcpy.Delete_management(prcp_Layer) del temp # Process: Copy Raster # arcpy.CopyRaster_management(prcp_Layer, test_img, "", "", "", "NONE", "NONE", "") print "Created Raster for " + prcp_Layer print "------------- finished" except: tbinfo = traceback.format_tb(sys.exc_info()[2]) print "Traceback Info:\n" for item in tbinfo: print item + "\n" print "Error Info:\n{0}: {1}\n".format(sys.exc_type, sys.exc_value)

‎07-11-2012

Chris- First off, if netCDF data are something you'll be working with regularly, I would strongly urge you to spend some time learning how to use a netCDF module in Python. There are several, some of which work fine outside of ArcGIS but cannot be used in script tools and such. Crunching many/large netCDF files with arcpy is clunky, at best (sorry ESRI!). Let me know if you decide to go that route and I can save you a lot of time figuring out where to start. More than likely, your problem is occurring when you make your calls to arcpy. I would use print statements to check each and every one of the input parameters to those tools and make sure your variables are correct. How large are your netCDF files? Unless they are extremely resolved spatially, I doubt it's a memory problem. If you don't find anything there, pin down what line is throwing the error. Are any output files being produced and, if so, are they correct?

‎06-20-2012

Thank you! I meant to write earlier that this script idea worked like a charm. I used an inner loop with Select by Dimension to evaluate each month in each netcdf file. I noticed you had a different method elsewhere in this thread. I'll try that next time. Thanks again, Pam PS - I can't find the 'green checkmark' that is supposed to indicate this answered my question so I'll just say, "This answered my question!". Great! I'll also mention that, compared to third-party Python packages, processing netCDF data in ArcGIS is very, very slow (sorry ESRI). For big jobs, or for folks working with netCDF data regularly, I would recommend ScientificPython. That way you can work with each step in the 'time' dimension as a numpy array, do whatever averaging or calculation is needed, and just use ArcGIS at the end to produce your GIS-friendly output.

Online Status	Offline
Date Last Visited	‎11-11-2020 02:23 AM

My Ideas

Latest Contributions by PhilMorefield

Re: questions about cross-validation

Re: questions about cross-validation

Re: NetCDF to csv script

Re: Adding multiple rasters together using python

Re: Huff Model

Re: Huff Model

Re: Should Esri ship third-party modules (like scipy) with ArcGIS?

Re: Should Esri ship third-party modules (like scipy) with ArcGIS?

Re: Kulldorff's Spatial Scan Statistic

Re: Kulldorff's Spatial Scan Statistic

Re: Adding multiple rasters together using python

Re: Adding multiple rasters together using python

Re: Arcpy NetCDF cache problems

Re: Arcpy NetCDF cache problems

Re: Batch Processing for Make NetCDF Raster Layer

Re: Batch Processing for Make NetCDF Raster Layer

Re: ArcGIS Pro 2.1: Importing arcpy from "outside"...

Re: Python Script tool templates

Re: Python script running slower in PyScripter tha...

Re: ArcGIS Pro 2.1: Importing arcpy from "outside"...