Efficiency of using Dashboard and Arcade to load a complicated feature layer

[PanjiBrotoisworo2]

Hi everyone,

This is just a general question on efficient data management. I have a single dynamic Survey123 form that acts as a all-in-one form. However, that leads to all the variables being crammed into a single feature layer item. But I utilize groups and repeaters so everything is not in a single layer as seen in the picture below.

My question is:

Despite everything being crammed into a single feature layer is it still efficient since everything is separated into separate layers within the feature layer? I only load a single layer such as "pier" when using ArcGIS Arcade. If this starts to scale with more data and more groups/repeaters will the dashboard start to slow down?

I use a data expression to create a layer using the "FeatureSetByPortalItem" function and I select individual layers to load. Overall, there may be hundreds of columns in the entire thing but each individual layer would only have 10 columns or so.

Thanks for any clarifications on this matter.

[AndreasHall]

There are not that many recommendations on this subject available. I have found a recommendation to limit the amount of layers per service to 20 here: Server configuration details—ArcGIS Hub | Documentation (along with some other general recommendations). I have experience of a feature service with 70+ layers that was split into several with maximum of 20 layers/service that greatly improved the loading time of a map consuming all layers. This was for layers hosted in ArcGIS Online. 

I do not know if there are specific implications for your Arcade use case.

[PanjiBrotoisworo2]

Thanks @AndreasHall, so it sounds like it may be best to split those layers into separate feature services if I want the dashboard to be responsive once the data scales up. I can probably look into webhooks with Survey123 in that case. Thanks for the clarification.

[jcarlson]

It really depends on the data expression. I have a service with 7 layers in it. In one dashboard, I have a data expression which only works with one or two of the layers, and loads about as fast as the rest of the dashboard. In another, I have an expression that has to work with nearly all the layers at once, and it can take several minutes to load.

Elsewhere, I also have a data expression working with a single layer, but it takes longer than the other two to run, simply because it has so many features to iterate over.

Can you share the expression?

[PanjiBrotoisworo2]

Here is the code in the code block below. There are some areas where its efficiency can be improved such as not using the `*` character for columns and probably just doing a join instead of looping through the featureset. But I just started learning Arcade yesterday so I hope to learn more about it soon.

Ideally I would just extract 4 columns from each layer in this code example instead of using `*`.

// Write an expression that returns a FeatureSet.
// Documentation: https://arcg.is/3c419TD
// Samples: https://arcg.is/38SEWWz

// Get featuresets
var fsSteel = FeatureSetByPortalItem(
    Portal('https://www.arcgis.com'),
    '2b673ac8555e417d817ef6a14947e05f',
    1,
    ['*'],
    false
);

var fsPier = FeatureSetByPortalItem(
    Portal('https://www.arcgis.com'),
    '2b673ac8555e417d817ef6a14947e05f',
    2,
    ['*'],
    false
);
var fsTrench = FeatureSetByPortalItem(
    Portal('https://www.arcgis.com'),
    '2b673ac8555e417d817ef6a14947e05f',
    3,
    ['*'],
    false
);

// Combine featuresets and ignore null data
var features = []
var feat;
for (var f in fsSteel) {
	if (!IsEmpty(f['steelClient'])) {
	    feat = {attributes: {
	        joinID: f["parentglobalid"],
	        client: f["steelClient"],
	        problemType: "steel",
	        hasProblem: f["steelHasProblem"]
	        }}
	   Push(features, feat)
	}
}
for (var f in fsPier) {
	if (!IsEmpty(f['pierClient'])) {
	    feat = {attributes: {
	        joinID: f["parentglobalid"],
	        client: f["pierClient"],
	        problemType: "pier",
	        hasProblem: f["pierHasProblem"]
	        }}
	   Push(features, feat)
	}
}
for (var f in fsTrench) {
	if (!IsEmpty(f['trenchClient'])) {
	    feat = {attributes: {
	        joinID: f["parentglobalid"],
	        client: f["trenchClient"],
	        problemType: "trench",
	        hasProblem: f["trenchHasProblem"]
	        }}
	   Push(features, feat)
	}
}


var combinedDict = {
    fields: [
        {name: 'joinID', type: 'esriFieldTypeString'},
        {name: 'client', type: 'esriFieldTypeString'},
        {name: 'problemType', type: 'esriFieldTypeString'},
		{name: 'hasProblem', type: 'esriFieldTypeString'},
    ],
    geometryType: '',
    features: features
}

return FeatureSet(Text(combinedDict))