GP Tools — Ignore selection but honor definition query

[Bud]

ArcGIS Pro 3.2.1:

There's a new toggle in geoprocessing tools that lets us choose to use the highlighted/selected records (toggle enabled) or all records (toggle disabled).

Disabling that toggle means the tool will ignore filters on the layer (definition query, time filter, range filter [and a selection layer's "selection definition set"]). 

I would like to have the option of ignoring the selection, but honoring filters (like a definition query). I have standard definition queries that I apply to almost all FCs/tables, such as STATUS=ACTIVE. It wouldn't make sense to ignore those definition queries in GP tools just because I have a selection.

The enhanced functionality could look something like this:  If the selection toggle is disabled, then provide an option to honor filters: yes/no.

Related: Original Idea - "The input has a selection." - slap a clear button in there!

[MErikReedAugusta]

I left my comments in more detail on the original idea that Bud linked above, but in short:

We also regularly use standard definition queries like this.  We have Assets that we can't physically delete from the database because of historic & backwards compatibility reasons in other systems, but we also don't want to have them live in the database because they're not there anymore.

Making them suddenly active just because I toggled off my selection feels really, really prone to error, at minimum.

I am heavily in favor of something along the lines of Bud's suggestion here, and am going to be in damage-control mode within our team once our IT department greenlights the update to 3.2.1, if this toggle still affects definition queries.

[AlfredBaldenweck]

Oh, bizarre. I assumed that the toggle was only for actual selections, not queries. 

Bud is right, this is completely misleading. If I'm clearing a selection, I want to be clearing a selection. I don't want to be clearing a query.

[MargaretCrawford]

I want to address some concerns about the selection, highlight and filter toggle button. Turning the toggle off will not remove your selections, highlights, or filters. Rather, it gives you the option to ignore them so you can process all records when you run the tool.

In the image below the selection was not removed from the layer even though the toggle is off:

Similarily, when you have a layer with a definition query and a selection applied, turning off the toggle will cause the tool to use all records in the table. Turning the toggle off will not remove the definition query from your layer.  If you would like to run a tool with the definiton query but no selection, you will have to remove the selection from the layer, rather than using the toggle button.

(In this image: the layer has a definition query and a selection. I can turn the toggle off in the tool, and they will still be present and active on the layer).

[Bud]

@MargaretCrawford 

If you would like to run a tool with the definiton query but no selection, you will have to remove the selection from the layer, rather than using the toggle button.

I understand what you're saying. And I understand that it would have been difficult to keep this functionality simple enough so that it doesn't clutter up the tool, while still meeting users' needs. But I feel this introduces a "gotcha" into ArcGIS Pro. As a user, I wouldn't expect ignoring highlighted/selected features would ignore the definition query.

I've been pleased to see all the enhancements Esri has made in recent years in terms of making selections/definition queries/geoprocessing tools more intuitive. But it seems to me that this toggle has added a new quirk (which is just part of life when rolling out new functionality); I think the next step should be refining the functionality to address user concerns.

For example, this wouldn't look as nice as the current toggle, but maybe a picklist would be more intuitive:

1. Use highlighted records
2. Use selected records
3. Use definition query
4. Use all records

Or something better.

[MErikReedAugusta]

I have to second Bud on this: it's absolutely a gotcha.  I get that you're not removing the selections or queries, but you are ignoring them in the calculation.  Yet the wording in the toggle switch only implies doing one of those two actions.

Bare minimum, I think that toggle should say something to indicate that it's also ignoring any query/filter that exists underneath the selection.

To reiterate a point that's been raised before in this discussion: Many of us have records that cannot be deleted for database interoperability/stability reasons (or as a matter of historical record, which is just as important for those of us working in government jobs), but which are no longer active or valid to end users.

Having those will always necessitate having baseline Definition Queries active in my teams' projects that I don't want ignored by GP Tools, and I especially don't want them ignored by the tool when they're led to believe that the tool's only going to ignore the Selection they have active on top of that Query and leave the Query active.

With the explanation @JonathanNeal gave on the other thread about what that toggle is actually doing behind the scenes, I'm willing to accept that this toggle is in practice an all-or-nothing proposition.  At this point, I'm arguing that the text next to that toggle needs to make that clear.

And I think it's also worth noting that the subset of people reading every single pop-up tooltip is likely not a high enough percentage to use what you have in that tooltip as a failsafe on this.

[MErikReedAugusta]

Thinking more about this, I think we might all be arguing past each other and overthinking this.

Why not just flip the script on that toggle?

Current Paradigm:

Text: "Use selected records: <count>"

Toggle On (default) Use Feature Layer with all active Filters/Selections/Highlights
Toggle Off Use underlying Feature Class and bypass all Filters/Selections/Highlights

Alternate Paradigm:

Text: "<Count> records selected.  Toggle to ignore all filters and selections and run this tool using the raw dataset."

Toggle On Use underlying Feature Class and bypass all Filters/Selections/Highlights
Toggle Off (default) Use Feature Layer with all active Filters/Selections/Highlights

[AlfredBaldenweck]

Rather, it gives you the option to ignore them so you can process all records when you run the tool.

Again, the point of a definition query is that I've already selected everything I want to see. If I want to ignore it, I would just turn it off. It makes sense to be able to ignore selections, but definition queries are a lot more intentional than that.

I wonder, does this also ignore the queries for query layers?

[MargaretCrawford]

Hi @AlfredBaldenweck, Definition queries are interactive, you can have multiple on a layer and turn them on and off. For query layers, the query is used to create the layer or table view, so it cannot be turned off. The toggle on the geoprocessing pane will not appear if you use a query layer as an input (without any other selection, highlight or filter). And if you have a query layer with a selection or filter that you used as an input on a tool, you can turn off the toggle and the query layer will still be honored.

So, you could try using query layers if you would like your query to always be honored by the tool: https://pro.arcgis.com/en/pro-app/latest/tool-reference/data-management/make-query-table.htm

[Bud]

I noticed that the toggle is in the Field Calculator tool. That makes sense since the Field Calculator was the subject of this implemented idea: Ability to calculate field on highlighted records only (yellow selection in attribute table) while r...

I wonder if the toggle is especially risky when it comes to the Field Calculator. If a user has a definition query and a selection on the layer and then disables the toggle without understanding that the tool will ignore the definition query, then the field calculation would get accidentally applied to the entire feature class.

That would be a significant problem. 

[AndrewDeLuca]

Hi  @MargaretCrawford, thank you for engaging on this. I just had an issue with this enhancement as described in @Bud's most recent post. I attempted to calculate field on a table with both a definition query applied and a selection applied with "use selected records" toggled off. I expected the calculate field to apply to all the visible records in the table regardless of my selection but not to the records filtered out by my definition query. When I turned off my definition query I realized that was not the case and had to discard a decent amount of work and re run my operations.

To me as a user, I have always associated "selection" and "definition queries" as different actions applied to a table. I appreciate it may be the same on the backend and that there are workarounds to avoiding this issue but I would like to reinforce the confusion in how this toggle operates from a user perspective. 

Definition queries/filters to me are a way to reduce the amount of records to interact with, not a way to select records. The fact that definition queries are then ignored when you toggle "use selected/highlighted records" off is not expected functionality from my perspective. Having edits applied to records not visible anywhere in the application due to a definition query is perplexing.