arcpy.sa.Sample() Spatial Analyst "Sample" method

[JamesCrandall]

Devloping an implementation that will build a GDB table from the raster cell values of a series of input raster datasets using the sa.Sample().  It works exactly as intended when I specify the output table to a File Geodatabase on disk but it fails if I try to write this table to the in_memory space.

Please highlight the obvious thing I am missing here.

This succeeds if FGDB is on disk:

ws_output = r'\\NetworkPath\GDB\Conc2.gdb'
elevrasname = "tabElev"
outelevtab = ws_output + "\\" + elevrasname
arcpy.sa.Sample(rasters, transect_in, outelevtab)

This fails:

outelevtab = "in_memory"
elevrasname = "tabElev"
outelevtab = ws_output + "\\" + elevrasname
arcpy.sa.Sample(rasters, transect_in, outelevtab)

Fails with:
"\\mypath\asc2raster.py", line 700, in extract_raster_values
    arcpy.sa.Sample(rasters, transect_in, r'in_memory\tabElev')
  File "C:\Program Files (x86)\ArcGIS\Desktop10.1\arcpy\arcpy\sa\Functions.py", line 1350, in Sample
    resampling_type)
  File "C:\Program Files (x86)\ArcGIS\Desktop10.1\arcpy\arcpy\sa\Utils.py", line 47, in swapper
    result = wrapper(*args, **kwargs)
  File "C:\Program Files (x86)\ArcGIS\Desktop10.1\arcpy\arcpy\sa\Functions.py", line 1344, in wrapper
    resampling_type)
  File "C:\Program Files (x86)\ArcGIS\Desktop10.1\arcpy\arcpy\geoprocessing\_base.py", line 498, in <lambda>
    return lambda *args: val(*gp_fixargs(args, True))
ExecuteError: ERROR 999999: Error executing function.

[NeilAyres]

This succeeds if FGDB is on disk:

ws_output = r'\\NetworkPath\GDB\Conc2.gdb'
elevrasname = "tabElev"
outelevtab = ws_output + "\\" + elevrasname
arcpy.sa.Sample(rasters, transect_in, outelevtab)

This fails:

outelevtab = "in_memory"
elevrasname = "tabElev"
outelevtab = ws_output + "\\" + elevrasname
arcpy.sa.Sample(rasters, transect_in, outelevtab)

James,
in your second bit of code, outelevtype is being assigned twice and there is no assignment for ws_output.
So perhaps that second bit should be...

ws_output = "in_memory"
elevrasname = "tabElev"
outelevtab = ws_output + "\\" + elevrasname
arcpy.sa.Sample(rasters, transect_in, outelevtab)

I also find it is better to use that os.path.join(var1, var2) construct to piece variables together to make paths.

Cheers,
Neil

[JamesCrandall]

Thanks for the reply, Neil.  That was an oversight on my part in my haste to post up the problem code.  It definitely appears that the output of sa.Sample cannot be set to the in_memory space --- can you or anyone confirm this?

All of these fail with the same ERROR 999999:

elevrasname = "tabElev"
ws_inmem = "in_memory"
outelevtab = ws_inmem + "\\" + elevrasname
arcpy.sa.Sample(rasters, transect_in, outelevtab)

outelevtab = "in_memory\\tabElev"
arcpy.sa.Sample(rasters, transect_in, outelevtab)

outelevtab =  r'in_memory\tabElev'
arcpy.sa.Sample(rasters, transect_in, outelevtab)

arcpy.sa.Sample(rasters, transect_in, "in_memory\\tabElev")

Thanks for the os.path.join idea.  I will get that into my py lexicon more!  Unfortunately it has the same error 999999:

elevrasname = "tabElev"
ws_inmem = "in_memory"
outelevtab = os.path.join(ws_inmem, elevrasname)
arcpy.sa.Sample(rasters, transect_in, outelevtab)

This succeeds:

elevrasname = "tabElev"
ws_output = r'\\NetworkPath\GDB\Conc.gdb'
outelevtab = ws_output + "\\" + elevrasname
arcpy.sa.Sample(rasters, transect_in, outelevtab)

[curtvprice]

Sample is a really old tool that dates back to ArcInfo Workstation GRID.

Have you tried Extract Values To Points or Extract Multi-Values To Points instead?

My guess is the newer tools may play nicer with the in_memory workspace.

I tend to reserve the in_memory workspace for things that I know will be very small datasets, to make sure I don't bollux up my RAM if I am not successful in cleaning up scratch data.

With the new scratchGDB environment (guaranteed writable scratch location) it's a lot easier to implement scratch files on disk, and file GDB is pretty fast.

[JamesCrandall]

Sample is a really old tool that dates back to ArcInfo Workstation GRID.

Have you tried Extract Values To Points or Extract Multi-Values To Points instead?

My guess is the newer tools may play nicer with the in_memory workspace.

I tend to reserve the in_memory workspace for things that I know will be very small datasets, to make sure I don't bollux up my RAM if I am not successful in cleaning up scratch data.

With the new scratchGDB environment (guaranteed writable scratch location) it's a lot easier to implement scratch files on disk, and file GDB is pretty fast.

Thanks for the input, Curtis.

Actually, I started this implementation using arcpy.gp.ExtractValuesToTable_ga() but that meant processing individual rasters at a time.  So, I looked at the arcpy.sa.Sample() method to allow me to pass in the entire set of rasters in the hopes of gaining some performance --- and I did!  It takes approx 1/2 the time to process which is huge gains.

So...  I hoped to use the in_memory space for even more performance.  Since the output of this is a table, I wasn't completely worried about filling up the RAM --- I make sure to remove intermediate results/outputs as quickly as possible.  That output converts quickly to a numpy array (arcpy.da.TableToNumPyArray) for additional processing.

My experience with in_memory has been good and I tend to look for opportunities to use it as much as I can BUT now you are making me question that if you worry about unsuccessful cleanup!

Anyway -- I will try the other methods you mention.

[curtvprice]

My experience with in_memory has been good and I tend to look for opportunities to use it as much as I can BUT now you are making me question that if you worry about unsuccessful cleanup!

I've had very good experiences with it too, but have seen complaints on the forums (like yours) when in_memory fails on particular tools. One of those trust-but-verify situations -- just warning you to keep your eyes open. 🙂

[NeilAyres]

James,
tested this code :

# test sa.sample
import sys, os, arcpy
from arcpy import env
from arcpy.sa import *

arcpy.CheckOutExtension("Spatial")

InputFGDB = "c:/Data/ESRI-SA/ArcForums/TestData.gdb"
env.workspace = InputFGDB
RasList = arcpy.ListRasters()
print RasList
inPnts = "SamplePnts"
outTab = "in_memory/sampTbl"

Sample(RasList, inPnts, outTab, "NEAREST")

env.workspace = "in_memory"
tbl = arcpy.ListTables()[0]
recs = arcpy.GetCount_management(tbl).getOutput(0)
print "Table {} records {}".format(tbl, recs)

listFields = arcpy.ListFields(tbl)
for f in listFields:
    print f.name

which gave me this....

>>> 
[u'IP_bi', u'RES_bi']
Table sampTbl records 200
OBJECTID
SamplePnts
X
Y
IP_bi
RES_bi
>>> 

So no errors on my side. No answer for you, sorry.
Cheers,
Neil

[JamesCrandall]

James,
tested this code :

# test sa.sample
import sys, os, arcpy
from arcpy import env
from arcpy.sa import *

arcpy.CheckOutExtension("Spatial")

InputFGDB = "c:/Data/ESRI-SA/ArcForums/TestData.gdb"
env.workspace = InputFGDB
RasList = arcpy.ListRasters()
print RasList
inPnts = "SamplePnts"
outTab = "in_memory/sampTbl"

Sample(RasList, inPnts, outTab, "NEAREST")

env.workspace = "in_memory"
tbl = arcpy.ListTables()[0]
recs = arcpy.GetCount_management(tbl).getOutput(0)
print "Table {} records {}".format(tbl, recs)

listFields = arcpy.ListFields(tbl)
for f in listFields:
    print f.name

which gave me this....

>>> 
[u'IP_bi', u'RES_bi']
Table sampTbl records 200
OBJECTID
SamplePnts
X
Y
IP_bi
RES_bi
>>> 

So no errors on my side. No answer for you, sorry.
Cheers,
Neil

Well now that is just entirely frustrating.  I copied your simple setup/code, replacing the sources and get the same error I was before.

ArcGIS 10.1 SP1 (Build 3143)
Citrix Deployment

Also, I ran this from PythonWin as well as a source script executed from a Toolbox.

Thanks again!