How much does does using a relative date filter effect CDN, scalability and performance?

DrewDowling

We have a hosted feature service that is updated frequently through a ArcGIS Velocity feed. The hosted feature service is published as a public hosted view and consumed in a public instant web app.

The hosted view has editing disabled and CDN enabled.

Typically the hosted view will display between 10 and 100 polygons.

We wish to make the app as performant as possible as we are expecting large spikes in usage. We are trying to follow the Best practices to optimize scalability guidelines. Unfortunately, due to a limitation of ArcGIS Velocity analytics, we need to use a relative date filter in the hosted view definition. The filter we are using is update_date within the last 5 minutes. This is necessary to filter out features that are no longer in the Velocity feed. (We can't use Gap Analysis as it doesn't work on polygon features).

The best practices guide warns against using relative dates:

"When setting up view definitions and filters, do not use dynamic or relative date conditions, such as current time, in the last five minutes, today, yesterday, or tomorrow"

How much of an impact, or how do we measure the impact, of using a relative date filter?

Does it break the CDN?

Is it just the initial query before data is added to the cache that is slow?

Any insights or suggestions greatly appreciated.

 

 

ChristopherCounsell

My understanding of this is that dynamic filters, such as relative date, cannot be cached and will always reach the database. This is because the underlying conditions (current time) - and thereby the cached response - is always changing. I believe this would apply to the browser cache, CDN and feature tile cache. For both view and map based filters.

You can check if the requests are going to the cache or database by monitoring the network traffic:

• x-cache: Hit from cloudfront indicates the response comes from the CDN cache (miss = did not use CDN)
• x-esri-ftiles-cache-compress: true indicates the response is cached as a feature tile on the server
• query-request units of 1-3 means it has not been cached and has gone to the database. Will count against your organization performance limits - too many requests and it'll timeout for 60 seconds.

https://www.esri.com/arcgis-blog/products/data-management/data-management/scalable-hosted-feature-layers-in-arcgis-online-tile-queries-and-response-caching

https://www.esri.com/arcgis-blog/products/js-api-arcgis/mapping/mapping-large-datasets-on-the-web

https://mediaspace.esri.com/media/t/1_oznrvijl (might be a more recent UC video available...)

I don't know much about ArcGIS Velocity but you could look to try and use a different attribute e.g. calculate a field to "current" or "archived" as part of the feature updates and use that your filter. You'd still need to consider the edit frequency impact on indexing, CDN, feature tile cache generation... but I think it'll have some capacity to succeed where relative date filters won't.