clustering 15,000+ items using javascript API and a FeatureServer or MapServer

Discussion created by sean.odin on Nov 16, 2012
Latest reply on Dec 9, 2015 by gildhoff
interesting problem:

ArcGIS server, that has a FeatureServer and a MapServer, serving layer with 15,000+ items.

problem: the service is configured to return max 1,000 items per query.

Currenty, my javascript page is using the FeatureServer, which is great, as I can use Feature Definition Expressions to filter the items.

problem: the FeatureServer only ever returns about 2,000 items, which means that when zoomed out, then whole areas of the map are empty!

note: requirement is that user should be able to view map at global scale (zoomed out) AND be able to see all 15,000 items, or at least clusters.

proposed solution:
re-configure the *MapServer* to return max 8,000 or ideally 20,000 items.
change javascript, to send query (with params to filter) to retrieve ALL matching items ( up to 15,000+ items ).
then javascript will use a custom clustering solution (taking an example from the ArcGIS help site) to cluster the items, so that we can draw a reasonable number of items in the browser, say 500 items.

PROBLEM:  the ArcGIS server is outside of my control, and the admins might NOT agree to increasing the max items returned from a query, which is currently limited to 1,000 items.   this means it would take up to 16 queries to populate the map, which is frankly silly !

QUESTION: has anyone got an alternative solution ?
for example - could the *FeatureServer* be re-configured to support clustering ?
note: we need to work with individual items, as we have a complex filtering requirement (with about 20+ different attributes that the user can filter on) so a solution with say Country layer or State layer etc is not good.

I have tried setting the 'generalising' parameters, setMaxAllowableOffset(), but this seems to have no effect.
is setMaxAllowableOffset() obsolete, since it is for 'generalising' which is supposed to be done automatically by Feature Server ?

Have also tried MODE_SNAPSHOT but found that MODE_ONDEMAND actually brings back more items, and so, is more useful.

Possible solution: just write my own web service to serve up the map points.
however, this is a *lot* of work, and there could be deployment issues, as the Production Environment is very restricted.

Any help or tips would be most appreciated ....