nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

How to make data expression that finds intersecting features faster?

NygrenOhto

How can I make the following data expression faster? I have a feature layer with 5500 rows and another one with 3175 and it takes this script 5:30 minutes to run through. That's way too long.

I am trying to create an indicator that compares data from two different years in certain areas in a city by checking intersecting population data squares with regions of a city.

var portal = Portal("https://arcgisportal.xyz.fi/arcgis/");
var areas = FeatureSetByPortalItem(portal, 'xxxx719ebb25xxxx968694axxxxxxxx', 3, ['*'], true);
var sql = "area_name = 'xyz'";
var filtAreas = Filter(areas, sql);

var stat2021 = FeatureSetByPortalItem(portal, 'xxx8abda0xxxx483xxxxxxxx', 0, ['he_vakiy'], true);
var sql2 = "he_vakiy > 0"
var stat = Filter(stat2021, sql2)

//var stat = Intersection(stat, filtAreas)

var features = [];
var feat;

for (var poly in stat) {
//Console(poly)
var pts = Count(Intersects(poly, filtAreas));
Console(pts)
feat = {
'attributes': {
'he_vakiy': poly['he_vakiy']

}
};

// Push feature into array
If(pts > 0){
Push(features, feat);
}
};
// Create dict for output FeatureSet
var out_dict = {
'fields': [
{'name': 'he_vakiy', 'alias': 'population', 'type': 'esriFieldTypeInteger'}
],
'geometryType': '',
'features': features
};

// Convert dictionary to feature set.
return FeatureSet(Text(out_dict));

Find more posts tagged with

ArcGIS Enterprise

Accepted answers

All comments

NygrenOhto

I managed to make it A LOT faster by using Disjoint() instead of Intersect().

Sample:

var portal = Portal("https://arcgisportal.xyz.xyz/arcgis/");
var areas = FeatureSetByPortalItem(portal, 'xxxxxxebb254f1xxxx9xxxx1cdfxxx', 3, ['*'], true);
var sql = "area_name = 'xyz'";
var filtAreas = Filter(areas, sql);

var stat2021 = FeatureSetByPortalItem(portal, 'xxxxbda0144xxxx3e8476d6xxxx', 0, ['he_vakiy'], true);
var sql2 = "he_vakiy > 0"
var stat = Filter(stat2021, sql2)


var features = [];
var feat;

for (var poly in stat) {
    for (var filtArea in filtAreas)
    {
        //Console(poly)
        If(Disjoint(poly, filtArea)) {
            continue;
        }
        feat = { 
            'attributes': { 
                'he_vakiy': poly['he_vakiy']
            
            }
        };
        
        Push(features, feat);
    }
    
    
    
    // Push feature into array
    
    
};
// Create dict for output FeatureSet
var out_dict = { 
    'fields': [
        {'name': 'he_vakiy', 'alias': 'population', 'type': 'esriFieldTypeInteger'}
    ],
  'geometryType': '', 
  'features': features 
}; 

// Convert dictionary to feature set. 
return FeatureSet(Text(out_dict));

-Ohto

NygrenOhto

I tried this change as well, but it didn't make any significant changes to the speed. I didn't time it but based on how long it felt.

Thanks for the help!

-Ohto

NygrenOhto

There are 5500 rows that the script needs to check intersections for against one area polygon that is filtered. I guess that is a lot for data expressions, I was just hoping it would be fairly fast as checking for intersections in a pop up is much faster with the same data. I using the following in the pop ups:

var m1 = Intersects(FeatureSetById($map, /* statistics_2021 */ "xxxxxxx3542-layer-8"), Buffer($feature, -1, 'meters'));
var m2 = sum(m1, "he_vakiy")

I'll have to keep trying different approaches to see if it's possible to make it fairly fast. Thank you for all your help so far!

-Ohto

JohannesLindner

Oops, you're right. I think I misremembered my tests with Count(). Thanks for the correction!

jcarlson

Using Count with a featureset is not a for loop, it sends a single query with returnCountOnly set to true.

jcarlson

In that case, the expression's execution time has less to do with the amount of data, and more with the number of features being evaluated for. Depending on what you're doing, sometimes there's just no easy way to cut that down.

Try arbitrarily limiting the number of input feautures and timing it, or else, add some Console functions to write out how long each iteration is taking. That won't help the full expression, but could at least narrow down the issue to "you have a lot of features".

JohannesLindner

Count() uses a simple for loop internally, so depending on the featureset, it can be really slow. You don't actually need the count here, you just need to check if the featrueset is empty or not. Another way to do that is to call First() on the fs. If the fs is empty, First() will return null.

So, in your code above, delete lines 17 & 18, and replace line 27 with this:

if(First(Intersects(poly, filtAreas)) != null) {

Keep Josh's changes, too. They might not bring a huge speed difference (my change probably won't, either), but they are considered best practice.

NygrenOhto

Hi and thank you for answering.

The script still works but it's not any faster.

-Ohto Nygren

jcarlson

One way to speed things up is to trim how much data is coming and going from the feature services. Unless you're using every field in a service, don't use ["*"] in your function. Only request the fields you absolutely need for your output.

And I do mean output. You can still filter on the field area_name without bringing that field into your expression. The Filter function merely supplies a where parameter to the FeatureService query endpoint, so you can include any field in the SQL statement, so long as the server has access to a field by that name.

Secondly, even though you're doing a spatial intersection between your layers, you don't need to include the geometry of the layer being intersected. When I write the function

Intersects(some_feature, other_featureset)

what happens on the server side is a request to

portalurl.../other_featureset/FeatureServer/0/query?geometry=<geometry from some_feature>&geometryType=<some_feature's geometry type>&spatialRel=esriSpatialRelIntersects

To put it plainly: you only need the geometry of the single feature, not the other layer. As long as you supply the feature to populate the intersecting geometry, the server will handle the spatial intersection for you, since it has access to the shape column of the other layer.

All that said, try re-writing line 2 as follows.

var areas = FeatureSetByPortalItem(
    portal,
    'xxxx719ebb25xxxx968694axxxxxxxx',
    3,
    [],
    false
);

Does that still work? And if so, does it perform better?

Join the Conversation

Signed in members can post, follow updates, and more. New here? Register a free account.

Quick Links

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.