Do you use Background or Foreground processing, by default? Comment why below.
Foreground and background processing—Help | ArcGIS for Desktop
Foreground (i.e Background Processing unchecked), as I have been burned too many times by things going awry with Background Processing.
Chris Donohue, GISP
I voted for "Foreground" for a few reasons.
I just happened to find your needle in point 2 this morning, under "Using the in-memory workspace with background processing" in this link: Foreground and background processing—Help | ArcGIS for Desktop
In short, background processing spins off into its own process and doesn't share RAM with the current ArcMap/Catalog process. So, to get the in-memory feature class back into the current process, it writes to disk, then back to memory to shuttle in between.
If I'm debugging an an app, I want to see it run, realtime. Once it's stable, I'm likely to have it deployed on a Server.
I like these polls Darren! I like seeing what others have chosen and their reasoning.
I'm with Chris on the foreground processing.
I also like Joshua's reason of:
First, for small tasks or processing that I do within ArcMap, the time it takes to have the application initialize and execute background processing is longer than the task itself. In short, foreground processing is much quicker.
Thanks, Adrian. I find myself stuck in routines that I can't even remember why, so it's good to see what others are doing and why.
For the record, I use foreground processing to take advantage of the in-memory workspace.
I appear to be in the minority, but my default is to use background processing. I prefer to start the task then carry on working and pick up the results later.
That said, if there are any problems (which there often are), I'll rerun the tool in foreground as it is way easier to troubleshoot.
I almost always use foreground processing as I need to have all the geoprocessing history saved (helps a lot come documentation time). Turning on background processing somehow turns off this feature and the geoprocessing history never gets updated. I am still using 10.2.0 and I don't know if this bug has already been fixed in 10.3 or later (wastes a lot of time this upgrading thing) but until then I will stick with foreground processing.
In any case, I also like having that progress window open as a visual cue that a process is still on-going as my version of multiprocessing usually involves having two (or more) copies of Arcmap/ArcCatalog running on one (or more) virtual machine.
All your past GP history can be found at %APPDATA%\ESRI\Desktop10.3\ArcToolbox\History, stored as XML. I don't know if it's ever cleaned up or if you have to delete manually when it gets too full. I've certainly got results going back to last June.
Yes indeed, one can find all your activities in your Arcmap sessions in there. I am however wanting the GP history to be stored for each individual feature class so that tracing what was done previously will be easier particularly, where the source data was taken from as I sometimes have the source data (in various versions/stages) stored in different locations. Having it in the metadata is also good as it stays with the feature class.
Normaly I prefer Background processing, simply because of the fact that I can kill a process fast without killing my project!
Foreground - Most of the tools or processes I'm running do not take a long time (so I don't have to wait long) and I'm usually waiting for those results for the next step (so I don't have anything to do in the meantime). Plus I like to see the text scroll as it's processing.
I use background processing when I can, since it allows me to keep working while the tool runs. When authoring my own tools, I tend to use the results pane as a log file. But I use foreground processing too, since some things don't work in the background.
Digging up an old thread because I found it while searching for information about the costs/benefits of foreground vs. background GP. It seems like the answer to the oversimplified question of "which is better?" is both highly situational and also rife with personal preference. I got to thinking, and decided to share my thoughts with the next geo-nerd to stumble upon this thread and bonus points if it happens to be exactly three years from today.
I prefer background processing, but for a different combination of reasons than y'all have expressed.
I too find myself tasked with many small processes chained together with one tool's output directly feeding the next tool's input, and I also like being able to continue poking around (say with the map layout) while they run. But I also notice the lag when the 64bit background instance has to initialize.
And I often utilize in_memory because I often switch between running my tools as standalone python scripts and running them in Arc, and I don't feel like changing all that mess and maintaining two different scripts (switching the tool's parameters from their static python variables to Get/SetParameterAs is pretty simple if you set each parameter up with both options instead of just one, you can just comment out the one you don't need for the moment).
I definitely notice the fact that using the in_memory space with 64bit background GP takes its sweet time to first make a copy of the dataset to disk before then loading it into RAM. But the times when I most need in_memory are when I am handling massive datasets and the processing time is night and day compared to running it solely from disk. So it's totally worth it for smaller tasks to take slightly longer and then chew through the monsters like butter compared to the inverse, if we're being binary about things.
Also I've gotten spoiled by all my intermediate data vanishing afterwards. I can now actually use my default.gdb as a place to keep files that have no other home (like a "Miscellaneous Data" folder) since I have so much less junk cluttering it up with nonsensical and sometimes offensive filenames. And if I'm chugging through a script in the python shell, the in_memory data is still at my fingertips if I need it afterwards.
So for my "best of both my worlds": when I'm working in ArcMap, I leave background processing on. I set up a quick model in MB to string together as many tools as I can and save it in a toolbox (or I just use a script tool I've written). I then run this in the background, enabling me to continue doing other stuff, and don't really notice the slowdowns as much as I would if I were waiting on each one to complete to begin the next. Since my most common workflows don't actually require me to examine the intermediate data, I can usually just pass it straight on to the next tool. If I hit a point where I figure I'm gonna need to look at the output, I pinch the model off at that point and get it to running.
However, this is just like my opinion, man. I rarely open ArcMap itself unless I need to kick out a map or visually analyze a dataset. For most of my plain ol' geoprocessing, I just run it in IDLE/Shell because that's more efficient for me personally. So I like to have my hands free while running processes in ArcMap to be designing the map layout or tinkering with symbology or to start planning the framework for the group of tools I need to run next. I have no idea whether background or foreground would technically run faster for my use-case, or whether that difference is even perceptible in the real world, but my work seems more efficient doing it my way and that's good enough for me (and more importantly for my employer).
If you try to rebuild an address locator from SDE data with 64-bit background geoprocessing, it will corrupt the address locator every time.
Retrieving data ...