Geometry and Raster Objects in Memory

[StephenBond]

From Esri's documentation and Zandbergen's Python Scripting for ArcGIS Pro book, it seems that  

geoms = arcpy.management.CopyFeatures(in_fc, arcpy.Geometry())

(where in_fc is some input feature class) should write the geometry objects in the input feature class to memory, not a new feature class. This appears to be the case when running the code from a Jupyter Notebook in the browser or a .py file, but it writes out a new feature class if run from an ArcGIS Notebook in ArcGIS Pro (v 3.0.3). In this case, instead of containing the geometries, the geoms variable will contain the path to the new feature class as a string.

Similarly, the same sources suggest something like 

hillshade = arcpy.sa.Hillshade(input_dem)

 should also just write output to memory until the save method is called. However, this code also creates a new raster when run from a Notebook in Pro.

There seems to be some subtly to this that I'm missing. Can anyone shed some light on this? Is this expected behavior? Is there some setting I need to change to avoid producing a bunch of intermediate output?

[ShaunWalbridge · Accepted solution]

The first case is a known bug (BUG-000143771) that we will address in the next release. For the raster case, I'm less clear that would work by default, could you share what gave you that impression? If the raster is large enough, it would exhaust memory, and I would expect that the raster to be written to the disk by default. I believe it creates a temporary raster, so if for example you were chaining together operations, you could run save and allow the intermediary rasters to be discarded. Or if you explicitly know the raster will fit in memory, you can write the output to it directly with e.g. memory\hillshade as the output.

[StephenBond]

Thanks for the reply! My understanding of what should happen with rasters mostly comes from the Zandbergen book (pg. 328): "The result from running the Slope tool is stored only in memory as a temporary raster object." I suppose it doesn't make a ton of sense to assume this will always be the case, but this statement seems pretty explicit, especially in combination with everything from the previous page or so. I am, however, seeing now that I misread the official documentation on this, as it does specify "the output is a temporary raster object on disk". So, what's the default location for the temporary rasters?

---

[ShaunWalbridge]

Probably for most cases where you want to work with more Python rasters, you'll want to start with arcpy.Raster, I think its documentation covers the general problem and some of your questions well, and while Zandbergen's book is solid, the documentation is generally authoritative. In the example you provided, the result will be an arcpy.Result object, but if you unwrap this with hs_raster = hillshade.getOutput(0), you'll get the resulting raster path. The location will be specified by the value of arcpy.env.scratchWorkspace. Using the raster class instead gives you a little more insight into the rasters and will let you work with them in a nicer way.

[StephenBond]

I must still be missing something here. The following prints class 'arcpy.Raster.Raster':

elev = arcpy.Raster("some_dem.tif")
hillshade = arcpy.sa.Hillshade(elev)
print(type(hillshade))

...which is a raster object, not a result object, and doesn't have a getOutput method. hillshade.isTemporary returns True, but the temporary raster output seems to persist in the scratch workspace after code completion/Pro restart/PC restart. Am I doing something incorrectly here?

[ShaunWalbridge]

Right, the function returns a Raster object because you passed in a raster in the previous step. In your original example of 

hillshade = arcpy.sa.Hillshade(input_dem)

the type of input_dem isn't explicit If you pass in a string or layer name, then hillshade will be a Result object, but if you are passing in a Raster object, you'll get a Raster result (this is how I'd recommend working with rasters from ArcPy). The scratch workspace can be different things, it isn't guaranteed to be a temporary location but is configuration dependent. You could set it to a temporary location yourself, you can also use the ARCTMPDIR environment variable to affect this behavior. If the location is e.g. a temporary folder inside of TEMP, it still won't be removed automatically, but would be cleaned up by some Windows processes or tools.

[BrianMcAuslan1]

Hi, thankfully found this thread unfortunately not before wasting a lot of time

I ran the hillshade  tool and then exported the code to a notebook which failed on the save method as shown

AttributeError                            Traceback (most recent call last)
In  [56]:
Line 25:    out_raster.save(r"C:\Users\hack\AppData\Local\Temp\ArcGISProTemp24040\Untitled\Default.gdb\HillSha_dem_98
AttributeError: 'Result' object has no attribute 'save

Looked at the ESRI sample code for the Hillshade tool and that also shows the .save method

Found an ESRI video which also encourages the use of the save method on raster output

https://mediaspace.esri.com/media/t/1_dekwby9u 

Anyway finally got this working

import arcpy
arcpy.ImportToolbox(r"C:\Program Files\ArcGIS\Pro\Resources\ArcToolBox\toolboxes\Spatial Analyst Tools.tbx")

arcpy.env.extent = "1262500 5002500 1263000 5003000"

out_raster = arcpy.sa.HillShade(
in_raster="dem_8m",
azimuth=315,
altitude=45,
model_shadows="NO_SHADOWS",
z_factor=1
)

#the type being returned by this function is a result object SO need to cast the raster to the raster type

Raster(out_raster).save(r"C:\Users\hack\AppData\Local\Temp\ArcGISProTemp24040\Untitled\Default.gdb\HillShade_dem")

arcpy.Delete_management(Raster(out_raster), '')

Hopefully helps someone