Custom Arcade Data source returns fewer records than it gets

[JCGuarneri]

I'm having trouble with an Arcade data source that I want to use in a dashboard (Enterprise 11.1). The goal is to take some inspections in Cityworks and sort them into bins based on what year in a 5-year cycle they fall in, to help managers rebalance them.

The problem is, the expression returns fewer values than it pulls in initially. Even more perplexing, the exact number of records returned each time is different:

 

Here's the code I'm using (server names changed to protect the innocent). Is there something I'm missing in the code that could cause that?

var portalItem = 'RanDOmP0rta1iDNUm85r';
var thisPortal = Portal('https://myserver.mydomain.com/portal');
var inspections = FeatureSetByPortalItem(thisPortal, portalItem, 58,['InspectionId','PrjStartDate','ShopDescription','DistrictDescription']);

//create dict to store updated features
var cycleYears = {
  'fields': [{'name':'InspectionID', 'type': 'esriFieldTypeOID'},
      {'name':'ShopDescription', 'type': 'esriFieldTypeString'},
      {'name':'District', 'type': 'esriFieldTypeString'},
      {'name':'CycleYear','type': 'esriFieldTypeDouble'}],
  'geometryType': '',
  'features': []
}
var count1 = Count(inspections);
console(`Records in: ${count1}`);

//put the year in each record into a bin
//append to dict
for(var f in inspections) {
  var cycleYear = (Year(f.PrjStartDate) % 5) +1;
  var new_f = {'attributes': {'InspectionID':f.InspectionID,
        //'PrjStartDate':f.PrjStartDate,
        'ShopDescription':f.ShopDescription,
        //'Text1': f.Text1,
        'District': f.DistrictDescription,
        'CycleYear': cycleYear
  }}
  Push(cycleYears.features, new_f)
}

//convert to feature set and output as data source
var inspCounts = FeatureSet(Text(cycleYears));
var count2 = Count(inspCounts)
console(`Records out: ${count2}`);
return inspCounts;

[KenBuja · Accepted solution]

It's because you set the InspectionID field in the dictionary as an esriFieldTypeOID. In testing with my data, if I use an InspectionID field that has unique values, I get the correct number of records. If I use a field without unique records, it only returns the number of unique records. Changing the field type to esriFieldTypeInteger returns all the records, regardless of whether the value are unique or not.

var cycleYears = {
  'fields': [{'name':'InspectionID', 'type': 'esriFieldTypeInteger'},
      {'name':'ShopDescription', 'type': 'esriFieldTypeString'},
      {'name':'District', 'type': 'esriFieldTypeString'},
      {'name':'CycleYear','type': 'esriFieldTypeDouble'}],
  'geometryType': '',
  'features': []
}

[JCGuarneri]

Ah, that worked! Thank you. Theoretically, those are all unique. But I'm happy that works. Exposed another thing I need to sort out, but this get's me very close to the finish line.

[KenBuja · Accepted solution]

With some more testing, I found a way to get the result faster, using the GroupBy function. Give this a try

var portalItem = "RanDOmP0rta1iDNUm85r";
var thisPortal = Portal("https://myserver.mydomain.com/portal");
var inspections = FeatureSetByPortalItem(
  thisPortal,
  portalItem,
  58,
  ["InspectionId", "PrjStartDate", "ShopDescription", "DistrictDescription"]
);

GroupBy(
  inspections,
  "OBJECTID",
  [
    { name: "InspectionID", expression: "1", statistic: "MAX" },
    { name: "ShopDescription", expression: "1", statistic: "MAX" },
    { name: "District", expression: "DistrictDescription", statistic: "MAX" },
    {
      name: "CycleYear",
      expression: "MOD(EXTRACT(YEAR FROM PrjStartDate), 5) + 1",
      statistic: "MAX"
    }
  ]
);

[JCGuarneri]

You're not kidding about that being faster! I had to make a couple minor modifications, but it is returning the correct number of values for each cycle year. I can't seem to get it to populate values correctly for ShopDescription and District (returns NaN), but I was considering splitting out those categories at the Cityworks URL side, anyway.

Thank you for your help!