idea GP Tools — Ignore selection but honor definition query in ArcGIS Pro Ideas

GP Tools — Ignore selection but honor definition query

Bud — Fri, 19 Apr 2024 00:32:24 GMT

ArcGIS Pro 3.2.1:

There's a new toggle in geoprocessing tools that lets us choose to use the highlighted/selected records (toggle enabled) or all records (toggle disabled).

Disabling that toggle means the tool will ignore filters on the layer (definition query, time filter, range filter [and a selection layer's "selection definition set"]).

I would like to have the option of ignoring the selection, but honoring filters (like a definition query). I have standard definition queries that I apply to almost all FCs/tables, such as STATUS=ACTIVE. It wouldn't make sense to ignore those definition queries in GP tools just because I have a selection.

The enhanced functionality could look something like this: If the selection toggle is disabled, then provide an option to honor filters: yes/no.

Re: GP Tools — Ignore selection but honor definition query

MErikReedAugusta — Thu, 28 Dec 2023 16:34:43 GMT

I left my comments in more detail on the original idea that Bud linked above, but in short:

We also regularly use standard definition queries like this. We have Assets that we can't physically delete from the database because of historic & backwards compatibility reasons in other systems, but we also don't want to have them live in the database because they're not there anymore.

Making them suddenly active just because I toggled off my selection feels really, really prone to error, at minimum.

I am heavily in favor of something along the lines of Bud's suggestion here, and am going to be in damage-control mode within our team once our IT department greenlights the update to 3.2.1, if this toggle still affects definition queries.

Re: GP Tools — Ignore selection but honor definition query

AlfredBaldenweck — Thu, 28 Dec 2023 19:17:31 GMT

Oh, bizarre. I assumed that the toggle was only for actual selections, not queries.

Bud is right, this is completely misleading. If I'm clearing a selection, I want to be clearing a selection. I don't want to be clearing a query.

Re: GP Tools — Ignore selection but honor definition query

MargaretCrawford — Wed, 10 Jan 2024 22:38:16 GMT

I want to address some concerns about the selection, highlight and filter toggle button. Turning the toggle off will not remove your selections, highlights, or filters. Rather, it gives you the option to ignore them so you can process all records when you run the tool.

In the image below the selection was not removed from the layer even though the toggle is off:

Similarily, when you have a layer with a definition query and a selection applied, turning off the toggle will cause the tool to use all records in the table. Turning the toggle off will not remove the definition query from your layer. If you would like to run a tool with the definiton query but no selection, you will have to remove the selection from the layer, rather than using the toggle button.

(In this image: the layer has a definition query and a selection. I can turn the toggle off in the tool, and they will still be present and active on the layer).

Re: GP Tools — Ignore selection but honor definition query

Bud — Fri, 12 Jan 2024 19:21:02 GMT

@MargaretCrawford

If you would like to run a tool with the definiton query but no selection, you will have to remove the selection from the layer, rather than using the toggle button.

I understand what you're saying. And I understand that it would have been difficult to keep this functionality simple enough so that it doesn't clutter up the tool, while still meeting users' needs. But I feel this introduces a "gotcha" into ArcGIS Pro. As a user, I wouldn't expect ignoring highlighted/selected features would ignore the definition query.

I've been pleased to see all the enhancements Esri has made in recent years in terms of making selections/definition queries/geoprocessing tools more intuitive. But it seems to me that this toggle has added a new quirk (which is just part of life when rolling out new functionality); I think the next step should be refining the functionality to address user concerns.

For example, this wouldn't look as nice as the current toggle, but maybe a picklist would be more intuitive:

Use highlighted records
Use selected records
Use definition query
Use all records

Or something better.

Re: GP Tools — Ignore selection but honor definition query

MErikReedAugusta — Wed, 10 Jan 2024 23:22:51 GMT

I have to second Bud on this: it's absolutely a gotcha. I get that you're not removing the selections or queries, but you are ignoring them in the calculation. Yet the wording in the toggle switch only implies doing one of those two actions.

Bare minimum, I think that toggle should say something to indicate that it's also ignoring any query/filter that exists underneath the selection.

To reiterate a point that's been raised before in this discussion: Many of us have records that cannot be deleted for database interoperability/stability reasons (or as a matter of historical record, which is just as important for those of us working in government jobs), but which are no longer active or valid to end users.

Having those will always necessitate having baseline Definition Queries active in my teams' projects that I don't want ignored by GP Tools, and I especially don't want them ignored by the tool when they're led to believe that the tool's only going to ignore the Selection they have active on top of that Query and leave the Query active.

With the explanation @JonathanNeal gave on the other thread about what that toggle is actually doing behind the scenes, I'm willing to accept that this toggle is in practice an all-or-nothing proposition. At this point, I'm arguing that the text next to that toggle needs to make that clear.

And I think it's also worth noting that the subset of people reading every single pop-up tooltip is likely not a high enough percentage to use what you have in that tooltip as a failsafe on this.

Re: GP Tools — Ignore selection but honor definition query

MErikReedAugusta — Wed, 10 Jan 2024 23:31:03 GMT

Thinking more about this, I think we might all be arguing past each other and overthinking this.

Why not just flip the script on that toggle?

Current Paradigm:

Text: "Use selected records: <count>"

Toggle On (default) Use Feature Layer with all active Filters/Selections/Highlights
Toggle Off Use underlying Feature Class and bypass all Filters/Selections/Highlights

Alternate Paradigm:

Text: "<Count> records selected. Toggle to ignore all filters and selections and run this tool using the raw dataset."

Toggle On Use underlying Feature Class and bypass all Filters/Selections/Highlights
Toggle Off (default) Use Feature Layer with all active Filters/Selections/Highlights

Re: GP Tools — Ignore selection but honor definition query

AlfredBaldenweck — Thu, 11 Jan 2024 18:58:02 GMT

Rather, it gives you the option to ignore them so you can process all records when you run the tool.

Again, the point of a definition query is that I've already selected everything I want to see. If I want to ignore it, I would just turn it off. It makes sense to be able to ignore selections, but definition queries are a lot more intentional than that.

I wonder, does this also ignore the queries for query layers?

Re: GP Tools — Ignore selection but honor definition query

MargaretCrawford — Fri, 12 Jan 2024 00:06:52 GMT

Hi @AlfredBaldenweck, Definition queries are interactive, you can have multiple on a layer and turn them on and off. For query layers, the query is used to create the layer or table view, so it cannot be turned off. The toggle on the geoprocessing pane will not appear if you use a query layer as an input (without any other selection, highlight or filter). And if you have a query layer with a selection or filter that you used as an input on a tool, you can turn off the toggle and the query layer will still be honored.

So, you could try using query layers if you would like your query to always be honored by the tool: https://pro.arcgis.com/en/pro-app/latest/tool-reference/data-management/make-query-table.htm

Re: GP Tools — Ignore selection but honor definition query

Bud — Thu, 18 Jan 2024 04:44:06 GMT

I noticed that the toggle is in the Field Calculator tool. That makes sense since the Field Calculator was the subject of this implemented idea: Ability to calculate field on highlighted records only (yellow selection in attribute table) while retaining the original selection

I wonder if the toggle is especially risky when it comes to the Field Calculator. If a user has a definition query and a selection on the layer and then disables the toggle without understanding that the tool will ignore the definition query, then the field calculation would get accidentally applied to the entire feature class.

That would be a significant problem.

Re: GP Tools — Ignore selection but honor definition query

AndrewDeLuca — Wed, 21 Feb 2024 15:46:05 GMT

Hi @MargaretCrawford, thank you for engaging on this. I just had an issue with this enhancement as described in @Bud's most recent post. I attempted to calculate field on a table with both a definition query applied and a selection applied with "use selected records" toggled off. I expected the calculate field to apply to all the visible records in the table regardless of my selection but not to the records filtered out by my definition query. When I turned off my definition query I realized that was not the case and had to discard a decent amount of work and re run my operations.

To me as a user, I have always associated "selection" and "definition queries" as different actions applied to a table. I appreciate it may be the same on the backend and that there are workarounds to avoiding this issue but I would like to reinforce the confusion in how this toggle operates from a user perspective.

Definition queries/filters to me are a way to reduce the amount of records to interact with, not a way to select records. The fact that definition queries are then ignored when you toggle "use selected/highlighted records" off is not expected functionality from my perspective. Having edits applied to records not visible anywhere in the application due to a definition query is perplexing.

Re: GP Tools — Ignore selection but honor definition query

Bud — Thu, 22 Feb 2024 06:05:16 GMT

From @MitchHolley1 in Field Calculate 'Use the selected records' Does Not Honor Selection Layers When Disabled:

...Since this [selection] layer is a different layer in the map (yes, they are "pointing" to the same data) my opinion is this behavior shouldn't happen. This makes working with subsets of data extremely difficult and confusing. If I wanted to work with the non-selection dataset, I would use the drop-down in the Calculate Field tool to do so, but that is not my intention.
The 'Source' tab of the layer properties also says this:
I do not want to access all the features in the source data, and that is why the 'Clear selected features to access all features in the source data' radio button in the Selection properties is unchecked. By default, this is unchecked. I shouldn't be able to access ALL features in the source data unless I explicitly tell the software to do so.

ArcGIS Pro - Selection layers

Re: GP Tools — Ignore selection but honor definition query

Bud — Sat, 11 May 2024 05:12:37 GMT

@MargaretCrawford

you could try using query layers if you would like your query to always be honored by the tool
Also: https://community.esri.com/t5/arcgis-pro-ideas/quot-the-input-has-a-selection-quot-slap-a-clear/idc-p/1420482/highlight/true#M29961

Query layers and query tables would only be used if the user never edits the data, which isn't normally the case. Typically, we work with FCs and tables directly in the map and edit them as needed. So, we want attribute filters like definition queries (such as STATUS=ACTIVE) to be honored in GP tools, even if the user disables the selection toggle.

And of course, for GP tools that modify data like Calculate Field, we wouldn't use query layers or query tables.