idea Parallel processing of batched geoprocessing tasks in ArcGIS Pro Ideas https://community.esri.com/t5/arcgis-pro-ideas/parallel-processing-of-batched-geoprocessing-tasks/idi-p/972929 <P>It's well known that ArcGIS only takes advantage of multiple processing threads to allow a <STRONG>single</STRONG> geoprossesing task to occur in the background while still interacting with the GUI. Any further processing is placed in a queue. A signifficant improvement on this (other than making the whole program multi-thread capable) would be to make batch geoprocessing tasks utilise more than the single thread currently used. If I run a batch of 4 Clip processes, there's probably no reason that at least two can't be run on parallel threads on a modern workstation (even my 3 year old workstation has 4 cores).<BR /><BR />This could also provide some relief for the fact that ArcGIS for Desktop is 32-Bit and hence somewhat hamstrung by limited memory access - processing large datasets which run out of memory could be done by breaking them down into tiles and running the required geoprocess on the tiles in parallel.</P><!-- content transformation source ID: 087E00000004PCz --> Fri, 11 Aug 2023 21:58:19 GMT AdamForknall 2023-08-11T21:58:19Z Parallel processing of batched geoprocessing tasks https://community.esri.com/t5/arcgis-pro-ideas/parallel-processing-of-batched-geoprocessing-tasks/idi-p/972929 <P>It's well known that ArcGIS only takes advantage of multiple processing threads to allow a <STRONG>single</STRONG> geoprossesing task to occur in the background while still interacting with the GUI. Any further processing is placed in a queue. A signifficant improvement on this (other than making the whole program multi-thread capable) would be to make batch geoprocessing tasks utilise more than the single thread currently used. If I run a batch of 4 Clip processes, there's probably no reason that at least two can't be run on parallel threads on a modern workstation (even my 3 year old workstation has 4 cores).<BR /><BR />This could also provide some relief for the fact that ArcGIS for Desktop is 32-Bit and hence somewhat hamstrung by limited memory access - processing large datasets which run out of memory could be done by breaking them down into tiles and running the required geoprocess on the tiles in parallel.</P><!-- content transformation source ID: 087E00000004PCz --> Fri, 11 Aug 2023 21:58:19 GMT https://community.esri.com/t5/arcgis-pro-ideas/parallel-processing-of-batched-geoprocessing-tasks/idi-p/972929 AdamForknall 2023-08-11T21:58:19Z Re: Parallel processing of batched geoprocessing tasks https://community.esri.com/t5/arcgis-pro-ideas/parallel-processing-of-batched-geoprocessing-tasks/idc-p/972930#M9644 <HTML><HEAD></HEAD><BODY>It would be really nice if geoprocessing tasks were thread assignable. <!-- content transformation source ID: 00aE0000000WVK0 --> </BODY></HTML> Sat, 06 Oct 2012 02:30:58 GMT https://community.esri.com/t5/arcgis-pro-ideas/parallel-processing-of-batched-geoprocessing-tasks/idc-p/972930#M9644 AaronAddison 2012-10-06T02:30:58Z Re: Parallel processing of batched geoprocessing tasks https://community.esri.com/t5/arcgis-pro-ideas/parallel-processing-of-batched-geoprocessing-tasks/idc-p/972931#M9645 <HTML><HEAD></HEAD><BODY>SP1 for ArcGIS 10.1 Desktop will add paralelel processing. Stay tuned. <!-- content transformation source ID: 00aE0000000Wv9a --> </BODY></HTML> Tue, 30 Oct 2012 13:27:56 GMT https://community.esri.com/t5/arcgis-pro-ideas/parallel-processing-of-batched-geoprocessing-tasks/idc-p/972931#M9645 VytautasGipiskis 2012-10-30T13:27:56Z Re: Parallel processing of batched geoprocessing tasks https://community.esri.com/t5/arcgis-pro-ideas/parallel-processing-of-batched-geoprocessing-tasks/idc-p/972932#M9646 <HTML><HEAD></HEAD><BODY>don't hold your breath vgipiskis <!-- content transformation source ID: 00aE0000000WzQI --> </BODY></HTML> Fri, 02 Nov 2012 05:33:13 GMT https://community.esri.com/t5/arcgis-pro-ideas/parallel-processing-of-batched-geoprocessing-tasks/idc-p/972932#M9646 AdamForknall 2012-11-02T05:33:13Z Re: Parallel processing of batched geoprocessing tasks https://community.esri.com/t5/arcgis-pro-ideas/parallel-processing-of-batched-geoprocessing-tasks/idc-p/972933#M9647 <HTML><HEAD></HEAD><BODY>Background geoprocessing at 10.1 accomodates 64-bit architecture, and you're limited to (IIRC) 60% of your total system memory rather than 2 GB. It's still possible to run out of memory (as I just did a moment ago with a Model) but it's a marked improvement over foreground geoprocessing. <!-- content transformation source ID: 00aE00000030mTH --> </BODY></HTML> Wed, 31 Jul 2013 23:30:56 GMT https://community.esri.com/t5/arcgis-pro-ideas/parallel-processing-of-batched-geoprocessing-tasks/idc-p/972933#M9647 JoeChampagne 2013-07-31T23:30:56Z Re: Parallel processing of batched geoprocessing tasks https://community.esri.com/t5/arcgis-pro-ideas/parallel-processing-of-batched-geoprocessing-tasks/idc-p/972934#M9648 <HTML><HEAD></HEAD><BODY>indeed it does&nbsp;<A class="ibtUserLinkNormal ibtUserLink" href="https://c.na9.visual.force.com/apex/ideaProfileActivity?c=09a300000004xET&amp;u=005E0000001mYnyIAE" style="color: rgb(44, 113, 4); font-size: 11px; background-color: rgb(242, 246, 235); text-decoration: none; font-family: verdana;" target="_blank">jchampagne2&nbsp;</A>, however it does not accommodate parallel processing, which is the who point of this idea :-) <!-- content transformation source ID: 00aE00000031FA8 --> </BODY></HTML> Fri, 16 Aug 2013 19:34:06 GMT https://community.esri.com/t5/arcgis-pro-ideas/parallel-processing-of-batched-geoprocessing-tasks/idc-p/972934#M9648 AdamForknall 2013-08-16T19:34:06Z