Need some serious help with Tasks

584
4
12-27-2021 06:06 PM
profchamberlain
New Contributor

I've looked through a whole range of different resources to understand queued tasks and the online support through GitHub and various videos (e.g. https://www.youtube.com/watch?v=9tQKOMoLa2w) just don't get me to a correct output.

I've got an add-in that uses tasks to perform various raster-based analysis functions. Here is an overview of the program:

  1. Main task starts
  2. Program gets a number of key locations (as vectors)
  3. Analysis creates a new raster using a listener with a concurrentQueue.
  4. A separate process calculates that new raster cell by cell, until it has calculated a value for all cells in the raster.
  5. The system continues to iterate through each location and sums all values for each calculation for each location into a composite raster.

The problem is that now I am trying to save a different raster for each location, as well as the composite raster and saving is done out of order. I think I have correctly created a means to store the individual and composite value separately by saving out the individual location raster once it is complete, then resetting that array and starting over. Meanwhile the composite raster array just keeps accumulating values. Yet, the individual rasters are saved after the composite raster and clearing the individual location raster does not seem to be working.

I have a decent amount of experience coding, but queues and sync functionality are new to me. I can't help but think something is awry with how saving and clearing the raster arrays work with tasks and queues. Anyone available to help with this?

0 Kudos
4 Replies
CharlesMacleod
Esri Regular Contributor

Can u post the code please?

Sounds like you may be using System.Threading.Tasks.Task rather than a QueuedTask? Although QueuedTask has the same semantics as a "System" Task, it's behavior is quite different.

 

0 Kudos
profchamberlain
New Contributor

Hi @CharlesMacleod, I posted you via a private message with code and further details. My plan is to find the solution and then post the summary here with relevant details to help others out. I think the code is a little complex to be posted here for now.

0 Kudos
KirkKuykendall1
Occasional Contributor III

Have you considered treating each location as a separate dimension in a multidimensional dataset?

(BTW, graphics like this sure make it easier to understand the problem statement.)

KirkKuykendall1_0-1640727538270.png

 

0 Kudos
profchamberlain
New Contributor

To be a little more direct: each location is a viewpoint and the analysis is a kind of viewshed that I've built. Viewpoints are stored in an array and the queue iterates through each viewpoint and calculates the resulting viewshed (again not the standard ESRI viewshed).

If you are suggesting to put the result for each viewshed into an array, yes, I've thought about that. However, the reason I'm trying to write the result for each view, clear that result and start over is that each raster can be quite large (hundreds of MB) and often the analysis needs to be run on hundreds (and in once case soon a few thousand). That makes for a large memory requirement.

I've printed out some diagnostics and it looks like the original thread needs to complete before the subsequent threads that are created. Apparently, what I'd like to do is create some kind of nested queue. So the first queue is run, then when another is created, I'd like to finish #2, then create number 3, then finish #3, etc. When when all done, #1 gets finished. Is there a way to do this?

0 Kudos