topic Re: Can't set output as intermediate data in ModelBuilder Questions

Can't set output as intermediate data

RobBlash — Tue, 29 Sep 2015 19:23:51 GMT

I have a model where I can't set some of my outputs as intermediate data. I have some intermediate data in the model, that's great, but I have more data that needs to be checked as intermediate so it gets deleted once the model has run. One of those can be set to managed, but intermediate is grayed out. I just want it to be intermediate, not managed. There's no documentation (that I can find) that will explain what restricts an output from being called intermediate. Any insight would be greatly appreciated before I lose my mind.

Re: Can't set output as intermediate data

ChrisDonohue__GISP — Tue, 29 Sep 2015 19:41:07 GMT

Part of whether intermediate data is deleted or not is based on how the model is run.

Data marked as intermediate data in a model is automatically deleted when the model is executed using its dialog box or when the model is executed from the Python window. Intermediate data will not be automatically deleted when the model is executed from the ModelBuilder window in order to maintain a model's has-been-run process state.

Source: ArcGIS Desktop

So for starters, which way are you running your model?

Chris Donohue, GISP

Re: Can't set output as intermediate data

RobBlash — Tue, 29 Sep 2015 19:48:20 GMT

I'm running it from the toolbox where the model is saved. Intermediate data is being deleted, but I want to tag more data as intermediate so it is also deleted. For now I added a process to delete the data but IMO that seems cheap/unnecessary if the data could simply be tagged intermediate and automatically deleted.

Really what I want is a better understanding so I can make more informed decisions when building models in the future. But as I said, the help seems to brush this one off as not so important.

Re: Can't set output as intermediate data

WesMiller — Tue, 29 Sep 2015 20:35:13 GMT

Have you tried Using in-memory workspace—Help | ArcGIS for Desktop

Re: Can't set output as intermediate data

DanPatterson_Retired — Tue, 29 Sep 2015 20:37:36 GMT

so you are saying that some of the files that are being created (ie intermediate) during the model are not being deleted? any particular tool? are there scripts in the model? a visual might help if it isn't too onerous, with the places where intermediate data are not being deleted, noted.

Re: Can't set output as intermediate data

RobBlash — Tue, 29 Sep 2015 21:00:47 GMT

I apologize if I'm not being clear. I have a model with ~20 processing steps. Some of the outputs are checked as intermediate (see clip 1 example point distance with output marked as intermediate). These behave as expected, i.e. the intermediate output is deleted once the model has run.

Clip 1:

I have a second point distance that runs a few steps later with slightly different input. The option to check this data as intermediate is grayed out. I can't figure out why. I can set it as managed though?

Clip 2:

To top it all off, if I add a delete process to the model, that very same output is then checked as intermediate (model builder checks it, I didn't do it). If I remove the delete process then it reverts back to the second screen shot.

Clip 3:

Re: Can't set output as intermediate data

RobBlash — Tue, 29 Sep 2015 21:02:43 GMT

I'll look into this, thanks. Have you experienced any issues with this (i.e. memory leaks/corruptions etc. causing unexpected results)?

Re: Can't set output as intermediate data

DanPatterson_Retired — Tue, 29 Sep 2015 21:45:02 GMT

Ok there is this

http://desktop.arcgis.com/en/desktop/latest/analyze/modelbuilder/a-quick-tour-of-managing-environments.htm

then this....

http://desktop.arcgis.com/en/desktop/latest/analyze/modelbuilder/creating-tool-with-modelbuilder-tutorial.htm

And this

https://pro.arcgis.com/en/pro-app/help/analysis/geoprocessing/modelbuilder/modelbuilder-changes-in-arcgis-pro.htm

which states that intermediate and managed parameters are removed in ArcGIS Pro.

I did notice that you don't seem to have that box as a parameter...perhaps that is why it is a managed parameter. I can understand why you might want it to be as you have made it and I understand the need to delete it ... effectively, it is a managed parameter...one that is used but (perhaps) the user need not interface with.

I can understand the rationale both ways, and I think your approach is correct and valid.

In conclusion... as you inevitably move forward to ArcGIS Pro, this discussion will be moot ... the option will no longer be there and the new rules will govern. Perhaps just consider this a minor inconvenience that one has to endure

Re: Can't set output as intermediate data

RobBlash — Thu, 01 Oct 2015 11:30:03 GMT

I appreciate the response, but I do not see my organization moving to a "Pro-only" environment any time soon, so that point doesn't quite apply. Regardless, it seems like pro is going to decide what to delete for you, which is my all-time favorite thing that any piece of software does for me. Can't wait!

I'll run through that tutorial at lunch today and see if it sheds any light on my situation, but aside from that my original problem still stands (Why can't I check the intermediate property for that output?). The fact that pro will make this choice for me some day in the future makes the question even more important to me.

Re: Can't set output as intermediate data

curtvprice — Tue, 06 Oct 2015 01:17:13 GMT

To top it all off, if I add a delete process to the model, that very same output is then checked as intermediate (model builder checks it, I didn't do it). If I remove the delete process then it reverts back to the second screen shot.

Model data elements are only settable as intermediate if they are truly intermediate, i.e. the data element is the output of one tool and also the input to another tool in the chain. When you added the Delete tool to the chain, that made it intermediate (between the point distance and delete tool).

Make Feature Layer is an exception to this because it doesn't really 'consume' the dataset, just creates a layer pointing to it. (I often use Delete to clean up feature layers BTW.)

Fortunately, you seem to have a good handle on preconditions used with the delete tool so you can set it up to do what you want.

Re: Can't set output as intermediate data

AndresCastillo — Fri, 26 May 2023 14:10:50 GMT

Thank you @curtvprice ,

This makes a lot of sense.

I notice a scenario in which there are additional things that would affect the ability to set the element to intermediate data.

Even a data element is 'truly intermeditate' as you have described above, if that intermediate element is connected downstream to both a tool that 'consumes' the dataset, and a tool that 'creates a layer view that points to the dataset', the option to set intermediate disappears.

In my case, the output to the 'Copy Features' output was not able to be set as intermediate, even though it served as input to both the 'Table to Excel' (consumes the data) and 'Select Layer By Attribute' (makes a view of the data) GP tools.

See the attached pictures of how the 'set intermediate' option appears once the downstream links are removed to the 'Select Layer By Attribute' gp tools.
@RobBlash

According to your clip 2 above, that seems to be what is going on.

@DanPatterson_Retired

I am able to set variable as intermediate in ArcGIS Pro 2.8.8 still.