Strange behavior of arcpy.Contour_3d

MattWeber2 · ‎04-02-2020

Hello,

i have a script for ArcGIS Pro (2.4.3) which essentially does this:

1. arcpy.env.workspace = "in_memory"
2. arcpy.CopyFeatures_management("fc", "copy")
3. Stuff with copy
4. arcpy.Contour_3d("dem", "contour_lines", 10)
5. More Stuff with copy

While the stuff in before Contour_3d works fine, the code after throws an exception, that "copy" doesn't exist.

After some testing, it seems, that contour_3d makes arcpy somehow 'forget' where to look for fc. I used Describe to find out, what was going on, and couldn't find any of the fc that where used and created during run anymore, including the just created "contour_lines".

What I noticed was, that doing anything not fc-related with the workspace fixed the issue.

arcpy.AddMessage(arcpy.env.workspace)
arcpy.AddMessage(arcpy.ListFeatureClasses())
arcpy.env.workspace = "in_memory"

All of those made the rest of the script work like a charm.

Another interesting Issue I noticed while searching for the error, is the interaction between Contour_3d and the new memory-workspace.

The reason I used the old in_memory-workspace was, that I created the dem with arcpy.MakeImageServerLayer_management() (which doesn't work with the new memory-workspace).

So, thinking the problem might lie within in_memory, I replaced in_memory with memory and used some pre-created dem instead.

So, strangly enough, the first error I got was:

ERROR 000880: Output feature class: 's extension is empty for the output feature class.

And I had to fix ".shp" to the name of the fc ("contour_lines.shp"), which surprised me, since I can't usually create fc with Extension in the memory-workspace.

Just to be sure I tried some other functions, all of which threw the expected Error:

ERROR 000354: The name contains invalid characters

Next strange thing is, running the tool a second time got me this Error:

ERROR 000287: Fail to create output feature class.

Apparently Contour_3d can't overwrite it's output. (why does the memory result even persist between script runs?)

Oh. and the first Problem wasn't fixed by changing to the new version neither.

So what is going on with Contour_3d, it seems like some serious Bugs to me, or am I doing something completly wrong?

Cheers

DanPatterson_Retired · ‎04-02-2020

The scripting example shows the arcpy.env.workspace being set to a file gdb not to 'memory'. the warning about memory is worth noting

Use of memory-based workspaces in a Python is only valid for geoprocessing tools. Memory is not a general-purpose virtual directory where you can write files or other data.

Temporary or other outputs are set to memory (eg r"\memory\something")

Write geoprocessing output to memory—ArcGIS Pro | Documentation

Contour_3d supports a load of environment variables, I would examine those relating to current workspace and scratch workspace

Contour—Help | Documentation

Try the process again setting the env.workspace to point to your project gdb, and setting the results of contour to memory, then copy to the gdb or if a shapefile is needed, a folder.

The rest of what you noted may be real or a cascading result of what you did

MattWeber2 · ‎04-03-2020

Thanks,

I Always understood the Paragraph with you cited as refering to using memory workspaces for everyday work.

So if I have a script like:

1. arcpy.Clip_analysis("to_clip_1", "clipping", "tmp_1")
2. arcpy.lCip_analysis("to_clip_2", "clipping", "tmp_2")
3. arcpy.Dissolve_management("tmp_1", "tmp_2", "Output_to_folder")

where the tmp-results are supposed to be in the memory workspace, and the output in some folder or gdb, do I need to specify it like this?
   1. arcpy.env.workspace = path_for_output
   2. arcpy.Clip_analysis("to_clip_1", "clipping", r"memory\tmp_1")
   3. arcpy.Clip_analysis("to_clip_2", "clipping", r"memory\tmp_2")
   4. arcpy.Dissolve_management("tmp_1", "tmp_2", "Output_to_folder")

If arcpy.env.workspace = "memory" sets the Workspace to some file gdb, where is this file gdb located? When I describe a fc and print the catalogPath, the output is just "memory\fc".

I will try your suggestion for the specific script, once I am sure that I got it right and will report on the results.

Cheers

DanPatterson_Retired · ‎04-03-2020

The final result that you want to use/save is to your file gdb probably the project one, or create a locally save gdb solely for that process

MattWeber2 · ‎04-03-2020

Sorry, but somehow your anwser makes me wonder if I asked my question the right way.

1. My tool shows unexpected behavior when I use what I believed to be the memory-workspace

(arcpy.env.workspace = "in_memory"/"memory")

2. You say that I am not using the memory-workspace, but rather a file gdb and the issues might be related to either that or some environment variables.

3. I ask (intended to) ask about the proper way to save to memory-workspace

4. You explain how to save the final result.

So, apparently I made a mistake with one of the questions, sorry for that.

To rephrase the issue. I have a Toolbox, that does a lot of stuff like clipping, dissolving, union, create contours, calc geometries, etc. etc. The Toolbox is primarily used to enable non-GIS experts to calculate complex results and ensure comparibility of the results.

95% of the results created during runtime are just intermediat steps and not needed, meaning I'd like to have them in a temporary Location. Since they are created by standard ArcGIS GP-tools like Clip or Union, I thought the memory-workspace would be the ideal solution. (Quoting from the offical documentation paqe:

so it is an ideal location to write intermediate data created in a ModelBuilder model or Python script.

Now, it seems like I misunderstood something essential about how to use the memory-workspace (or I misunderstood your first anwser), so let me ask again.

Is it possible to set the env.workspace to a memory-workspace?

Do I need to set the location for each output that I want to be temporary manually to the memory-workspace? (i.e. r"memory\tmp_out")

Sorry for the long text, with the somehow unrelated seeming questions, but without a clear understanding about how to save temporary results, I don't think I can try your proposed solutions for the actual question.

Cheers

DanPatterson_Retired · ‎04-03-2020

set each one as r"\memory\whatever" ... at least that is what I have used without issue... maybe others have had success with

arcpy.env.workspace = .... but your case suggests that you shouldn't