Retrieve Unique Values from feature layers

[RyanSmith1]

Hi Guys, i need to retrieve unique values from a set of feature layers. I've been using the following example as a guideline - Using JavaScript to populate a ComboBox with unique values | ArcGIS Blog but my problem as noted in that article is that the records i have is over 500, 2500 to be exact. I came across a certain "returnDistinctValues" property from the Query API Reference - Query | API Reference | ArcGIS API for JavaScript which if set to true will return distinct Values of the specified outfields of the query. I haven't tried out as yet since it requires "supportsAdvancedQueries" property of the feature layer to be set to true and the current configuration has it set to false. 

I do recognize that changing the maxRecord of the feature layer could severely hamper the performance, so the only other option i can think of using the queryIds method of the feature layer?

Please advise as to which approach i should take to overcome this issue.

Thanks,

Ryan

[TomSellsted]

Ryan,

Do you expect the query to return more than 500 distinct values? That would certainly be a lot for combo box.  I did a distinct query on my parcel service for land use codes.  My parcel database contains over 100,000 records. The distinct query returned 65 land use categories very quickly.

Regards,

Tom

[RyanSmith1]

Hi Tom, thanks for your response.

There's 2500 records from the feature layer but currently it's not getting all the unique values because it's only processing the first 1000 records. I'm expecting about 80 - 90 unique values for the combo box. You said that you ran a distinct query, so i need to set returnDistinctValues to true on the query?

*I am using API version 3.13

Thanks,

Ryan

[NigelDsouza]

Hi Ryan,

If you want to check for unique values in all your records (i.e >1000) then try this..

Use the queryIds(query, callback?, errback?) method on the feature layer. This method has no limit to the number of records returned from the server + it will return an array of all the object_ids present in your feature class. Once you get the length of all the records present in your feature class, use this object_id array to query for distinct values and populate your final array.

For eg: I have an array length of object_ids = 2500 and at the same time, I'm also aware of the limit of records return from the ArcGIS Server (the default is 1000).

So ill query the first 1000 then the next and then the remain... and in each query ill get the distinct values and sort it in an array.

You will have to implement a logic by which this looping query is done in packets like (1-999,1000-1999,2000-2500).

Regards,

Nigel.

[RyanSmith1]

Hi Nigel,

Thanks for this, I'm probably going to use this approach if using the returnDistinctValues property doesn't work. I thought i could avoid the array filtering/querying of object Ids which does affect performance of the application. Surprisingly this hack also didn't work ArcGIS JavaScript API Hacks: Query Return Distinct - raykendo

Thanks,

Ryan

[TomSellsted]

Ryan,

The returnDistinctValues works very well.  The parcel records service is over 100,000 records and it queries it very quickly.  I used the query without a map sample to create demo that you view.  The pertinent code for this query is:

// query distinct use code values from parcels
var queryTask = new QueryTask("url to your service");


var query = new Query();
query.where = "USE_CODE is not null";  // query for non-null values
query.orderByFields = ["USE_CODE"];    // sort it so we won't have to later
query.returnGeometry = false;          // turn geometry off, required to be false when using returnDistinctValues
query.returnDistinctValues = true;
query.outFields = ["USE_CODE"];


queryTask.execute(query, showResults);

You can view the demo of this at:

Query Distinct Values

Regards,

Tom

[NigelDsouza · Accepted solution]

Hi Tom,

Your answer is right. However if I have more than 1000 records and distinct value lies say on the 1001th<  record then that value will not show up in the result using just the code you have given cause by default the max records returned by the server will not exceed 1000(This value can be changed).

So Ryan will first have to query the objectids then pass a packet of objectids in the query example you provided for him to get all the distinct values. The query sample you provided will have to be run in a loop untill all the objectids have been queried.

I had the same issue in one of my widgets in Flex so I worked it out this way. Yes the returnGeometry must be turned to false and returnDistinct values to true to optimize the query.

Eg: Once I had to query district names of India which have over 3000 unique Values. If i just simple query on a layer specifying the outField as "dtname", returnDistinctValues =true and returnGeometry= false ill just get 1000.

Hence the above apprach is one way.

Regards,

Nigel.

[TomSellsted]

Nigel,

You are correct.  If you are expecting a list of values that exceeds the max count of the service being queried, then you would need to do multiple queries as suggested.

In the context of using these values for a combo box, it would seem having a list with that many values for a user to look through might make it difficult to use.

Regards,

Tom

[NigelDsouza]

I had used a List view and a text box so the user could enter some key words and filter the list.

Somewhat like a autocomplete prompter.

And the list control also helped to make multiple selections.

Maybe we could use a editable combobox in this case what say .Actually unless one is very sure that distinct values is going to be less than a 1000 then just your approach is fine. But incase like data just keeps flowing in and it is expected that the distinct values can go over a 1000 then a combination of the two above suggested methods would do great.

I have also tired the combined queries with records less than a 1000 and its performance is almost instantaneous.

- Nigel.

[TomSellsted]

Nigel,

Agreed!  I had used the JQuery listview with the filtering function which works very well.  You are correct, the query is almost instantaneous.

Regards,

Tom