Filter widget loses unselected values after selection is made - query logic issue or bug?

JasperRomero

I have repeatedly encountered behavior with ExB filters which I think reflects a bug, or at least a profoundly unintuitive implementation:

I have several filters on a single data source in one filter widget. Each filter operates on a different field, and most of them follow the pattern "[Field] is any of [selection]" from the Multiple source type, with "Ask for values" selected and "List values based on" = "All unique values of this field".

Most of the filters include some default values selected, and have "Apply this filter automatically" selected.

When the Experience is opened, the preconfigured filter application and value selections work as intended. When I open the dropdown for one of the filters, the "Show all" and "Show selection" buttons do what they should.

However, if I "Clear selection" for one of the filters, and then select only a couple of options, all of the other options disappear. Whether I have "Show all" or "Show selection" selected, only the selected options are available.

If I then "Clear selection", it deselects the options but the rest of the values do not reappear.
The only ways to get the full list of values back are to either use the "Turn off all filters" button or "Reset all filters" button (which returns them to their default state with multiple selections made for each filter).

I have read similar posts where the issue is the SQL query logic in the filter configuration, but I can't understand why, if all of the records are able to be displayed when the app first loads with multiple filters applied and multiple selections per filter, they would disappear when the user starts to manually change filter selections. In any case, it seems like "All unique values of this field" should ensure that all unique values of the field are indeed displayed regardless of other filter settings, otherwise I would have selected "Values filtered by previous expressions" or "Values filtered by all other expressions". The behavior seems like the filter selection is immediately applied to the data source, thereby limiting the data the filter is available to see...

I have not tried manually defining all of the options and using the Multiple (predefined) source type, but this shouldn't be necessary (especially if I have a field with 100 different values).

Any insights, explanations, configuration suggestions, or confirmation that this is strange behavior would be very much appreciated.

NicoleJohnson

I'm not sure it's a bug. That's not to stop you from submitting a ticket or an Idea though.

Group filters, data views (yes, different thing than a feature layer view): Google these (no shade, but you will get more out of the documentation than I can regurgitate to you). When you add a widget to the canvas, there's also an information icon in the upper right of the widget configuration panel. Click that and it'll take you to the documentation.

Re: multiple instances of a data source, my suggestion is to:

Add another instance of your data (e.g., the web map if that's how you brought in your data).
Re-name it (inside Experience Builder) so you can keep track of which instance holds the data you actually see in the map.
Then, in your SR Points layer in this second data instance, make a data view called "Neighborhood."
Go back to your filter and instead of basing the filter on the layer used in the map, create a group filter where the primary layer is this second instance of SR Points, specifically the "Neighborhood" data view.
Then, add your map layer instance as a secondary layer in the group filter.

There's no need for triggers/actions for this, having the map layer instance in the group filter means it'll filter when the filter is applied. Repeat at step 3 for report type, submission date, etc. The reason this works is because the filter values are coming from data views that aren't being affected by anything else.

The downside is that there's no way to convert a single-layer-based filter to a group filter, so you'd have to redo the filters. Group filters also don't currently have the ability to do multi-clause filters, if you have any of those.

All that said, there could very well be something I'm missing about your setup and the solution is easier than this. But, if you can't figure out a different way, this is an option to try.

View solution in original post

NicoleJohnson

If I'm understanding correctly, you have several filters, all/some of which have pre-selected values, but more values might be available for a given filter depending on the filter. The expected behavior is that no matter what the user selects/deselects, they're still able to see all the values that exist. Is that correct? If so, I've definitely seen this behavior before too and have handled it a few different ways depending on the data/my mood:

Like you mentioned, use the multiple (predefined) option and enter all the options. Not a great option when you have a lot of values or changing values.
You can instead create data views (comprised of all the data) for each filter you're wanting to use, then have that data view be the primary layer in a group filter.
- I've had issues before doing this with a data view created from the same layer being used in a map. I always keep my map data instance and my everything-else data instance separate.
You could also bring in the unique values through Arcade and like the 2nd option, use a group filter.

JasperRomero

@NicoleJohnson thanks for the response!

Yes, your summary of what I'm hoping for is correct.

First off, from your suggestions it sounds like this behavior is more bug to circumvent than feature, is that fair to say? i.e. there is nothing in the logic of my data or filter configuration that should necessarily force the behavior? Trying to figure this out because (i) if I'm doing something outright wrong I want to understand what it is and (ii) maybe this is worth a bug report or suggestion.

Re: your suggestions - haven't used Group filters before, could you say a little more about how to configure this? I would make a Data view in ExB (not the same as a feature layer view?) by adding the same layer that's in the map as a standalone data source, not filter anything out when making the data view, and use that as the primary layer for a group filter, then presumably would add a trigger data/action data connection to the filter so that the results of the filter are also applied to the map layer? Additionally, if I have both a map and several charts all using the same data source, is the issue you encounter specific to the map layer? i.e. is it safer to separate "filter layer =/= map and chart layers" or "map layer =/= filter and chart layers"?

Thank you again, really appreciate the suggestions and response. Happy to add more info/screenshots if anything is unclear.

NicoleJohnson

I'm not sure it's a bug. That's not to stop you from submitting a ticket or an Idea though.

Group filters, data views (yes, different thing than a feature layer view): Google these (no shade, but you will get more out of the documentation than I can regurgitate to you). When you add a widget to the canvas, there's also an information icon in the upper right of the widget configuration panel. Click that and it'll take you to the documentation.

Re: multiple instances of a data source, my suggestion is to:

Add another instance of your data (e.g., the web map if that's how you brought in your data).
Re-name it (inside Experience Builder) so you can keep track of which instance holds the data you actually see in the map.
Then, in your SR Points layer in this second data instance, make a data view called "Neighborhood."
Go back to your filter and instead of basing the filter on the layer used in the map, create a group filter where the primary layer is this second instance of SR Points, specifically the "Neighborhood" data view.
Then, add your map layer instance as a secondary layer in the group filter.

There's no need for triggers/actions for this, having the map layer instance in the group filter means it'll filter when the filter is applied. Repeat at step 3 for report type, submission date, etc. The reason this works is because the filter values are coming from data views that aren't being affected by anything else.

The downside is that there's no way to convert a single-layer-based filter to a group filter, so you'd have to redo the filters. Group filters also don't currently have the ability to do multi-clause filters, if you have any of those.

All that said, there could very well be something I'm missing about your setup and the solution is easier than this. But, if you can't figure out a different way, this is an option to try.

JasperRomero

Following up - the group filter approach seems to have resolved the issue.

I did not test the Multiple (predefined) approach, but it occurred to me that it might be possible to do programmatically via the Python API editing the item_data or config.json (i.e. pull the list of values from the layer, set the list of values for Multiple (predefined)) - haven't looked into it further but if anyone else is trying to solve the problem, that could be worth looking into.

Also haven't tried the Arcade option, but after manually selecting hundreds of checkboxes (why have a "clear selection" button but no "select all" button in the UI?!) to set up the Data Views and preselected filter values, I think if you're dealing with fields that have many values (i.e. if the list of what shouldn't be included is shorter than the list of what should) it could definitely be worthwhile to use an Arcade expression to make a feature dataset for your primary group filter field.

Thanks again for the suggestions @NicoleJohnson

NicoleJohnson

@JasperRomero wrote:
....after manually selecting hundreds of checkboxes (why have a "clear selection" button but no "select all" button in the UI?!) to set up the Data Views...

Maybe I'm misunderstanding, but setting up the separate data view is meant to keep you from having to do all this manual work. Unless you're saying your own preselected values that you chose (that you want the user to have "on" by default) are comprised of hundreds of values.

Btw, I recently contacted support about adding a "select all" action when you have a filter with the "is any of" operator. Here's what they said:

I wanted to let you know that I successfully replicated the behavior you described. During my research, I found that an active Enhancement Request already exists for this exact functionality: ENH-000167898. I've linked it to your case so you can view it directly in My Esri > Support > Bugs and Enhancements.
The request is currently Under Consideration. To give you some context, our Product Development team has noted a couple of complex technical challenges with implementing a simple "Select All" default:
App Freezing: For huge datasets, triggering a "Select All" can instantly hit performance limits and freeze the app.
Data Limits: Due to data paging, the button might only select the first page of records or get blocked by maximum record thresholds (MaxRecordCount), leading to incomplete data displays.
Our development team is actively evaluating alternative ways to improve this user experience. In the meantime, the current workaround is to manually select one by one the required options within the filter.

JasperRomero

@NicoleJohnson thanks for the follow up - yes, preselected values to have on by default (and having to manually select to create the Data view itself - is there some way around this?).

Good info re: "Select all" - I guess that makes sense. Just Kudoed and followed that ENH.

NicoleJohnson

@JasperRomero When you create the data view, the data view shouldn't have any selections, those should only happen in the filter widget. By creating the data view and using it as your primary layer (and not using that data view for anything but a universe of values), you should see all values regardless of whatever else you filter on/off. But yeah, if you want many values preselected in the filter, there's no way around that except selecting them.

If you follow up with support and see any resolution to this, please @ me! It's something I've thought was annoying but didn't have the wherewithal to send to support.

Suming

Hi @JasperRomero, the Filter widget works as follows:

Multiple Filter widgets operate independently. Therefore, when Filter widget 1 applies an SQL expression, it may affect the values available in the clause dropdown list of Filter widget 2.
Within the same Filter widget, however, individual filters do not affect one another. Therefore, the values in a clause dropdown list should remain stable and should not be affected by the Clear selection operation.

Based on the behavior described in the original post, this appears to be a bug. However, the issue could not be reproduced in ArcGIS Online with my test app:
https://experience.arcgis.com/builder/?id=d73dacead887481da37fb7175c7fb3fe&views=page

Could you help confirm whether this issue occurs in a specific version of ArcGIS Enterprise or Experience Builder? If possible, could you please share a test app that reproduces the issue?

JasperRomero

@Suming the issue described initially in this post was the result of several filters on different fields for a single layer, all within one Filter widget.

This is through AGOL, which I believe is not affected by our Enterprise version (but please correct me if I'm mistaken). To be clear [ 😉 ], the problem was not affected (for better or for worse) by Clear selection - when a selection was made, unselected fields disappeared shortly afterwards - they didn't disappear, or reappear, in response to the Clear selection button being used.

The app contains potentially sensitive info and is for internal use only, but if I find some time to reproduce the issue with something suitable for public sharing I will follow up.

I was not able to access your test app, seems like the permissions may need to be changed.

Thanks for weighing in!