Managing a deferred list of query tasks

[TracySchloss]

My goal is to select a legislative district by number, using it's geometry as input to look up different features in that district. There will be lots of different layers the user can turn on/off (schools, nursing homes etc) and the query will select only those features that are currently visible within their district and create a list.    The query tasks must therefore be generated on the fly.

I am using a posted Fiddle example for a deferred list for identifytasks, modifying it for query tasks instead.  (Thanks Brett for such a great example BTW) I have made good progress up to a point, I see that each of my queries are executing.  But I'm getting bogged down in how to manage the multiple sets of results that are getting returned as output from each of the tasks.

I have an empty array defined as qTaskList to hold all the query tasks needed for the current visibility.

qTaskList = [];

In previous functions I have defined a single Query, called multiQuery that will be used for all the queryTasks I execute.  It is defined as:

multiQuery = new esri.tasks.Query();

   multiQuery.outSpatialReference = map.spatialReference; 
   multiQuery.returnGeometry = false;
   multiQuery.spatialRelationship = esri.tasks.Query.SPATIAL_REL_CONTAINS; 
   multiQuery.outFields = ["*"];

These are supposed to generate a set of querytasks, which as put in a deferredlist and then executed. 

 
function runQueries(geometry) {
    multiQuery.geometry = geometry;
    //Create an array of all layers in the map
    var layers = dojo.map(map.layerIds, function(layerId) {
      return map.getLayer(layerId);
    }); 
    layers = dojo.filter(layers, function(layer) {
        return layer.getImageUrl && layer.visible;
    }); //Only dynamic layers have the getImageUrl function. Filter so you only query visible dynamic layers
     var qtasks = generateQTasks(layers);
    var defTasks = dojo.map(qtasks, function (task) {
        return new dojo.Deferred();
    }); //map each query task to a new dojo.Deferred
    var dlTasks = new dojo.DeferredList(defTasks); //And use all of these Deferreds in a DeferredList
      dlTasks.then(showQueryResults); //chain showQueryResults onto your DeferredList

          for (i=0;i<qtasks.length;i++) { //Use 'for' instead of 'for...in' so you can sync tasks with defTasks
        try {
            qtasks.execute(multiQuery, defTasks.callback, defTasks.errback); //Execute each task
        } catch (e) {
            console.log("Error caught");
            console.log(e);
            defTasks.errback(e); //If you get an error for any task, execute the errback
        }
    }
}
//function used to determine which layer visibility of each ArcGISDynamicMapServiceLayer
function generateQTasks(layers) {
   qTaskList.length = 0;
    dojo.map(layers, function (layer){

        if (layer.id !== "districtLayer") {//don't run querytasks against the layer used in the findtask, it's not needed

         var visLayers = getVisibleLayers(layer);
          dojo.forEach(visLayers, function (layerId){
            var qTask = new esri.tasks.QueryTask(layer.url+"/"+layerId);
            qTaskList.push(qTask);
        });
}
});
    return qTaskList;
}

function getVisibleLayers(myLayer) {    
var visibleLayers = [];
    var items = dojo.map(myLayer.layerInfos, function (info, index) {
        layerID = info.id;
        if (info.visible && !info.subLayerIds) {
            visibleLayers.push(layerID)
        }
    }); 
    return visibleLayers;
}

This is where I start to get lost.  I've added break points and it does look that I have an array, successResults, each of which is a featureSet (I think!).  But when I try to process these individually, it only seems to process the first featureSet, not all of them.  I know I also have issues in the next function, queryGeometryResultsHandler_toGrid, which is supposed to generate a titlePane in a div for each out the ouput.  Probably that will be a whole other thread!  For now all I'm trying to do is properly deal with my array of featureSets.

function showQueryResults (queryResults) {
        var successResults = [];
    queryResults = dojo.filter(queryResults, function (result) {
        return queryResults[0];
    }); //filter out any failed tasks
    for (i=0;i<queryResults.length;i++) {
        successResults = successResults.concat(queryResults[1]);//these are all the successful querytasks
    }
 numberQueryTasks = successResults.length;
//need to process each set of query results
    dojo.forEach(successResults, function(taskResults) {
 console.log("within dojo foreach of successResults");
  queryGeometryResultsHandler_toGrid(taskResults);
    });
 
}

[ReneRubalcava]

DeferredList results are little funky when you first start dealing with them. The result of a DeferredList is 2-dimensional arry (array of arrays).

A result will look like this.

[
  [ true, { name: "I am result 1." } ],
  [ true, { name: "I am result 2." } ],
  [ true, { name: "I am result 3." } ]
]

Each array inside the main array is a result of a deferred. If the first element in the array is true, it resolved sucessfully, if it is false, something went wrong and the second element would be the error message.

You can view the source code for DeferredList here.
https://github.com/dojo/dojo/blob/master/DeferredList.js#L32-L58
The highlighted snippet shows how the results of a DeferredList are built.

Knowing how the results are returned from a DeferredList makes parsing the data a little easier.

In your snippet, I think if you change this bit here, it should work.

queryResults = dojo.filter(queryResults, function (result) {
        // change this here, you were returning queryResults[0]
        // result[0] would be either true or false and this filters out all false results
        return result[0]; 
    });

Then the next bit of code you use to concatenate the results should work.

I have this snippet of similar code that I have used to clean up the results and it's always worked for me.

function cleanResponse(res) {
 var i = 0,
  len = res.length,
  results = [];
 for (; i < len; i++) {
  // checks to see if the resolved value is true
  if (res[0]) {
   // concatenate the array with the resolved result
   results = results.concat(res[1]);
  }
 }
 // creates an array of only your results or FeatureSets
 return results;
}

With that, DeferredList is now a deprecated module and it is recommended that you look at the dojo/promise/all module. The API is much cleaner to work with and doesn't involve worrying about 2-dimensional arrays.
http://dojotoolkit.org/reference-guide/1.9/dojo/promise/all.html

Hope that helps.

[TracySchloss]

I get what you're saying about changing what is returned from the dojo.filter, but it didn't help.  Is there a recommendation on which is the better way to run my queryGeometryResultsHandler_toGrid?   I tried dojo.map first:

function cleanQueryResults (queryResults) {
    var successResults = [];
    queryResults = dojo.filter(queryResults, function (result) {
        return results[0];
    }); //filter out any failed tasks
    for (i=0;i<queryResults.length;i++) {
        successResults = successResults.concat(queryResults[1]);//these are all the successful querytasks, an array of featuresets
    }
//then process each set of query results
dojo.map(successResults, function (results) {
 queryGeometryResultsHandler_toGrid(results);
 
});
}

Then I switched to dojo.forEach.  It seems like to like dojo.map is the better choice.  I can see that queryGeometryResultsHandler_toGrid executes one time, but it doesn't ever come back to this to process the other feature sets.  Should I be setting a variable for dojo.map and then returning something from that function?   That function is still a work in progress.  It is supposed to be creating individual titlePanes within an existing div.   I commented nearly everything I had out (it was copied from another project and needed tweaking anyway).  I expected this to execute for each one of my featureSets that are contained in successResults. 

//output query results to grid sample function
function queryGeometryResultsHandler_toGrid(results){
    //puts the results of the query into a grid
    var tp;
    var myDiv = dojo.byId("attributeDiv");
    myDiv.innerHTML = "";
    var resultsLength = results.features.length;
    var grid;
    var allItems = dojo.map(results.features, function(feature){     
        return feature.attributes;
    });    
     for (var i = 0; i < resultsLength; i++) {             
        var items = allItems.slice(i, (i+1));   
        var data = {
            items: items
        };       
        var currentStore = new dojo.data.ItemFileReadStore({
            data: data
        });       
        var currentLayout = [];
        var addField;

          tp = new dijit.TitlePane({ id: 'gridPane_'+paneId, title: paneId});        
            myDiv.appendChild(tp.domNode);
 
      /*  for (fieldName in items[0]) {
            if (fieldName == "FID" || fieldName == "OBJECTID" ) {
                addField = { field: fieldName, formatter: makeZoomButton };
               currentLayout.push(addField); 
            }
            if (fieldName == "ROWGUID" || fieldName == "Shape" || fieldName == "CITY_NAME" || fieldName == "FID") {
              //  addField = { field: fieldName, hidden: true };
              //  currentLayout.push(addField);
            }  else {
                addField = {
                field: fieldName,
                width: "120px"
               };
            currentLayout.push(addField);
           }
        }   */    

    // Create Grid 
   /*
            grid = new dojox.grid.DataGrid({
                id: paneId+"_grid",
                store: currentStore,
                structure: currentLayout
            }, document.createElement('div'));
            
             dojo.byId(paneId+"_gridPane").appendChild(grid.domNode);
            grid.startup();
   grid.canSort = function(){
            return false;
        };
        tp.set("content",grid);
  */
    }
}