topic Re: Force batch to run processes sequentially in Geoprocessing Questions

Force batch to run processes sequentially

JonathanBailey — Mon, 25 Oct 2010 02:28:31 GMT

I have a model that I wish to run in a batch process. Within this model, I join an input table to another table, then calculate a value in the input table.

When running this model a single time, it works correctly. I can calculate the field value in the input table using the expression [%Join Table%].FieldName.

However, when I run this model in a batch process, it seems that each instance of the model is run in parallel, so that multiple joins are performed on the input table at the same time. As a result, I can't be sure of the names of the fields in the join table, and so the batch process fails.

Is there any way to force a batch to run each instance separately?

Thanks,

Jon.

Re: Force batch to run processes sequentially

LoganPugh — Thu, 09 Jun 2011 22:11:54 GMT

Bump.

This needs to be addressed I think.

Running Models in Batch mode results in parallel operation when often (almost always?) you want them to run in sequence.

The only workaround I know of is to write a Python script to simply call the model, but this seems extremely tacky.

Re: Force batch to run processes sequentially

LukeWebb — Mon, 13 Jun 2011 13:25:03 GMT

BUMP 🙂 subscribed etc - and a link to my same post in the ArcGIS desktop general forum in case you want to subscribe to it... fingers crossed theres a work around somewhere or bug fix on way (Give us the option, sequencial, or paralell)...

http://forums.arcgis.com/threads/31135-ModelBuilder-in-Batch

Thanks

Re: Force batch to run processes sequentially

HugoAhlenius — Tue, 14 Jun 2011 11:25:32 GMT

Sounds like something to enter into http://ideas.arcgis.com!

Re: Force batch to run processes sequentially

BruceHarold — Wed, 22 Jun 2011 15:58:28 GMT

Hi All

In this case the recommended approach is to use an iterator instead of a batch input. Iterators run the entire mode for each iteration vs. batch which runs the model once and runs each tool n times for each execution. In this case use the feature class iterator if all the feature classes are in one workspace or use iterate multi-value to select the set of feature classes.

http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/A_quick_tour_of_using_iterators_for_iteration_Looping/002w0000001w000000/

Re: Force batch to run processes sequentially

LoganPugh — Wed, 29 Jun 2011 17:07:13 GMT

The problem with multivalue iterators is, unless I am mistaken, they only let you modify one model parameter per model iteration. The batch dialog lets you specify a different value for every model parameter for each iteration. What I need is the ability to specify each model parameter differently per iteration like with the batch dialog, but force the processes within the model to run sequentially like with an iterator.

Is there a way to get the best of both worlds (other than the aforementioned -- write a Python script to call the model)?

Re: Force batch to run processes sequentially

LukeWebb — Tue, 05 Jul 2011 11:59:01 GMT

The problem with multivalue iterators is, unless I am mistaken, they only let you modify one model parameter per model iteration. The batch dialog lets you specify a different value for every model parameter for each iteration. What I need is the ability to specify each model parameter differently per iteration like with the batch dialog, but force the processes within the model to run sequentially like with an iterator.

Is there a way to get the best of both worlds (other than the aforementioned -- write a Python script to call the model)?

Yes, 2nd that! To share a tool to our less GIS savvy users here (I would rather send them a tool that is verified than a workflow) I have to send them a seperate script to call the tool if there going to do any batch, and then explain how to use it as I do not have time to code a nice UI like the "batch" window.

Thanks

Re: Force batch to run processes sequentially

LukeWebb — Tue, 05 Jul 2011 12:01:28 GMT

Yes, 2nd that! To share a tool to our less GIS savvy users here (I would rather send them a tool that is verified than a workflow) I have to send them a seperate script to call the tool if there going to do any batch, and then explain how to use it as I do not have time to code a nice UI like the "batch" window.

Thanks

Out of interest, why is it implemented in this fashion? For my work I would always prefer it to work sequencially allowing completed data to come through quicker, rather than wait for them all to start finishing at once? I am sure there is a good reason!?

Re: Force batch to run processes sequentially

LoganPugh — Wed, 03 Aug 2011 19:45:17 GMT

Out of interest, why is it implemented in this fashion? For my work I would always prefer it to work sequencially allowing completed data to come through quicker, rather than wait for them all to start finishing at once? I am sure there is a good reason!?

I can't think of any.

Re: Force batch to run processes sequentially

BBicking1 — Fri, 05 Aug 2011 18:27:07 GMT

Hey Jonathan,

How are you?
Please include your model here with some sample data and the ArcGIS version and OS you're working on.

Thanks,

Barbara Bicking

Re: Force batch to run processes sequentially

LukeWebb — Wed, 26 Oct 2011 13:46:27 GMT

I am still very interested in this topic! I would love to be able to choose every parameter in my model and run it in batch - sequentially!

The problem has been well described above, I dont have the model or time to hand but I describe a very simple model in this post with the same problem: http://forums.arcgis.com/threads/31135-ModelBuilder-in-Batch, it is easy to reproduce.

The problem with multivalue iterators is, unless I am mistaken, they only let you modify one model parameter per model iteration. The batch dialog lets you specify a different value for every model parameter for each iteration. What I need is the ability to specify each model parameter differently per iteration like with the batch dialog, but force the processes within the model to run sequentially like with an iterator.

Is there a way to get the best of both worlds (other than the aforementioned -- write a Python script to call the model)?

Re: Force batch to run processes sequentially

GraemeBrowning — Fri, 26 Oct 2012 06:32:26 GMT

I've just encountered what I believe to be the same issue over at Stack Exchange GIS so I can confirm that it is still happening at ArcGIS 10.1.

Has anybody seen a bug number assigned to the issue?

If anyone from the Analysis and Geoprocessing team needs a simple test case that demonstrates this just let me know and I can email it.

- Graeme

Re: Force batch to run processes sequentially

LukeWebb — Mon, 13 May 2013 14:04:22 GMT

Still not happy with this feature!!!! Why cant my users use the batch window for my models like they can for every other tool!!!!!

I would be satisfied if ESRI can either:

Explain simply to me why it works like this?? And when it could possibly be useful so that I can try and implement this feature?

OR

Aknowledge and Fix the bad design.

Please??

Re: Force batch to run processes sequentially

RustyRex — Mon, 29 Jul 2013 21:36:29 GMT

This is a major bummer. I have no idea why a model in batch would behave this way. It pretty much put the tool I was creating in a dead end. Does anyone know if a script tool in batch behaves the same way?

Re: Force batch to run processes sequentially

RHirner — Mon, 11 Nov 2013 12:28:03 GMT

Still present in ArcGIS 10.2 �?�

Re: Force batch to run processes sequentially

JohnDavies1 — Fri, 05 Sep 2014 18:57:49 GMT

Wow! I've been having this problem too! So aggravating! I'm running 10.2.2. I guess this is still an issue, not workarounds yet?

Re: Force batch to run processes sequentially

Osvaldo_JoséRibeiro_Pereira — Sat, 30 Apr 2016 14:49:31 GMT

It seams that this discussion keeps going on without any solution. I've been having the same problem wit ArcGIS 10.3 and it would be really hard to use interactors given the size of my model. Any solution?

Re: Force batch to run processes sequentially

curtvprice — Wed, 04 May 2016 03:32:20 GMT

This is a long time issue with Model Builder. I'm sure it has something to do with how Batch and Model Builder work or it would have been fixed.

I have found a workaround... create a Python script tool that calls the model, and batch that.

See: Re: Model Builder + Batch Creator do not work together for an example.

Re: Force batch to run processes sequentially

OsvaldoJosé_Pereira — Thu, 23 Jun 2016 12:40:36 GMT

I found a quite simple solution for this problem. You just have to follow three steps when building you rmodel and it will work perfectly. 1.) You have to create a Geodatabase (don't use the default geodatabase from ArcGIS). I've created one in the same folder named "Scratch"; 2.) You need to set all of your output data as non-intermediaries, cause they need to be saved in order to run the model; 3.) Finally, when defining the output of each operation in model builder you have to use the sentence %i%. This is useful to say the mode that it should generate individual intermediate data to each Input of my batch list. For example, if I have a map algebra operator in my model, I should set the name of the output data as Raster_Decli%i%. As a result I would have Raster_Decli0, Raster_Decli1, Raster_Decli2 (...).