Sorry for the long post, but it's an interesting problem!
EDIT: My post kept getting flagged for "invalid HTML" no matter what I did, so I had to post a plain-text version. I'm trying to add the formatting back in.
Limit Your Server Calls
The issue is probably related to how your for loops are structured. If you open your browser's developer tools and watch what's going on when an expression like this actually evaluates, you'll see that functions like Within submit a call to the REST endpoint of the service.
So, picking this apart:
- For every point in your service
- Make a call to the fire layer
- Make a call to the weather layer
- If either of the above have one or more result:
- Loop over every feature in the fire layer
- Loop over every feature in the weather layer
For 150k points, and those other layers having 286 and 4063 features in them at the time of writing this, that means every hit on the initial point intersection kicks off 4,300 spatial overlays. If every one of your points intersected, we're talking hundreds of millions of operations. If I was your browser, I'd crash too!
Now, thankfully, those nested Within functions in the fire / weather loops are feature-to-feature, so there shouldn't be additional server calls made, otherwise we'd be talking hundreds of millions of server requests. But doing a spatial operation is still going to take some time, and any time your point has a hit in either of those forecast layers, it will result in a separate operation for every polygon in both services.
Here's a tip. When you make a spatial call against a FeatureSet, that happens on the server. Your intersecting point has to be included, but the other layer's geometry is actually not needed! So when you create the FeatureSets for your weather and fire polygons, go ahead and set those to false for returning the geometry. That will cut back considerably on the amount of data going back and forth.
Also, when we include a filter based on the Count of the features, it modifies the request to return the count only. And I'm using "intersects", not "within". For point/polygon overlays, they're the same thing, unless the point is exactly on the boundary of the polygon, which would be pretty rare for unrelated layers.
Reuse Your FeatureSets
In your expression, you use Within(pnt, fire_poly) in your condition on line 46. But then when the condition is met, your sort of go backwards and recreate the same thing. If you just assign Within(pnt, fire_poly) to a variable and return it, the result will be a FeatureSet of all fire polygons intersecting with your point. By reusing this FeatureSet in our nested loop, there will be no need to loop through all the fire polygons. Eliminating those nested "within" functions will prevent the hundreds of millions of spatial operations.
Try replacing lines 42-58 with this:
for (var pnt in pt_fs) {
var in_fire = [];
var in_event = [];
var xs_fire = Intersects(pnt, fire_poly)
var xs_weather = Intersects(pnt, weather_poly)
// populate fire array if any
if (Count(xs_fire) > 0) {
for (var fire in xs_fire) {
Push(in_fire, fire['IncidentName'])
}
}
// populate weather array if any
if (Count(xs_weater) > 0) {
for (var event in xs_weather) {
Push(in_event, `${event['Severity']} ${event['Event']}`)
}
}
RAM vs Network Speed: Memorize
For such a big expression, even cutting down on the amount of server back-and-forth won't matter much when you've got to wait for 300,000+ separate GET requests to finish.
I wrote about a custom function that could help you here. Given that your fire and weather layers are fairly small, we could push them entirely to your browser's memory, then run the intersections. In this way, we only make server calls on the initial load, and none at all in our "pnt in pt_fs" loop.
Stick this at the top of your function, and then implement it in your weather/fire FeatureSet functions. Oh and a little caveat, if we're moving this into RAM instead of using the original REST endpoint, we've got to flip the returnGeometry back to true.
function Memorize(fs) {
var temp_dict = {
fields: Schema(fs)['fields'],
geometryType: '',
features: []
}
for (var f in fs) {
var attrs = {}
for (var attr in f) {
attrs[attr] = Iif(TypeOf(f[attr]) == 'Date', Number(f[attr]), f[attr])
}
Push(
temp_dict['features'],
{attributes: attrs}
)
}
return FeatureSet(Text(temp_dict))
}
var portal = Portal("https://www.arcgis.com");
var fire_poly = Memorize(FeatureSetByPortalItem(
portal,
'd957997ccee7408287a963600a77f61f',
1,
['IncidentName'],
true
));
var weather_poly = Memorize(FeatureSetByPortalItem(
portal,
'a6134ae01aad44c499d12feec782b386',
6,
['Event', 'Severity'],
true
));
None of my test data I have handy currently intersects with any fire or weather areas, so I can't really say if any of this will solve the problem, but I can say that running a version of your original expression kept going for over 10 minutes, whereas with the changes I made, it was under a minute.
- Josh Carlson
Kendall County GIS
