Enhance Append Tool to Allow Python/Arcade Expressions for Unmapped Fields

1606
14
07-26-2022 06:33 AM
Status: Under Consideration
Labels (1)
feralcatcolonist_old
Occasional Contributor II

Enhance the Append Tool's functionality by allowing users to utilize Python or Arcade expressions to fill in output fields that do not have a 1:1 match between the two data sets.

 

Example:

Many exploratory data analysis feature classes start from source data with their own attribution. In the process of analysis, an individual may add/remove some fields but otherwise keep the data similar. One workflow is to then append additional records from the source data, selecting "use the field map to reconcile field differences" is useful to just add the new data, but then the user has to go into the table, highlight the newly added records and then run field calculations for each of the new columns their analysis dataset might have.

Or a different example, you have two datasets that are similar, but one is missing some relevant individualized fields-- --or has the same data in incompatible fields (it has a string and you need a numeric). You can create these fields easily enough with a field calculation expression and should be able to do this at the point of entry within the append tool.

The enhancement would look this, for a field(s) you could write in expressions to fill out the data programmatically as it uploads to your established dataset.

feralcatcolonist_2-1658842159966.png

 

 

14 Comments
ÁkosHalmai

Make able the Field Mapping in all Geoprocessing tools to consume constant values, not just fields. In some cases for example in the Append tool in NO_TEST scenarios it would be extremely useful to fulfill a field (or multiple fields) with a constant value. There is a quite simple workaround to achieve this: leave the field unmapped and then fill it with Field Calculator. But if this feature would be integrated into the field mapping then it would be more convenient, consistent and more logical to use (raster tools can also consume constant values).

Ákos Halmai

ChrisMathers1

I was coming to create this same idea. When appending I often wish I could put in a value for a field in the target that doesn't have something to map into it.

KubaSzostak1

It would be great to see such functionality in ArcGIS Pro Append Tool.

I have about 10 shapefiles which I import into one EGD feature class. There is attribute type which I set based on shapefile name (points_fixed.shp, points_calculated.shp, points_temp.shp, ...). Having option to set Constant value would resolve my problem.

Another use case is when there is Non-NULL field in target dataset which is absent in input dataset. You can't append such data without redesigning input dataset scheme (which not always is possible).

RichardFairhurst

I voted for your idea and would have a use for this.  Please also vote for my idea, which gives other examples of how it would be useful to be able to use Arcade for some of the parameters of other tools.

Allow Arcade Expressions in Parameters of Geoprocessing Tools

LindsayRaabe_FPCWA

When using the Append geoprocessing tool, you have the option to use the field map to reconcile field differences between the Input data and Target data. 

My idea, is that when you do use this method, you have an additional option under field sources to input a default value for those fields that don't have a match. In my example below, I'm appending data from one layer with a simple Label option, into another layer that has a corresponding "Comments" fields, but also a "Class" field with 4 class options (for basic data categorisation). It would be a good time saver, if in the Class Field, I could either type in a default vale that is added against all appended features, or alternatively, select an option from the Feature Template. Obviously, it would be fine if left blank, or would not be used if a source field is mapped. 

LindsayRaabe_FPCWA_0-1681280811343.png

 

Bud
by

Somewhat related:

In other enterprise systems like IBM Maximo, there is a concept called “Bind Variables”. Those variables can be used to get things like the system date when populating a value.

https://bportaluri.com/2012/08/variables-dynamic-queries-conditional-expressions.html
https://www.ibm.com/support/pages/username-appname-and-other-special-bind-variables-you-can-use

Would something like that be useful in ArcGIS Pro? Use the date, etc as the value for the constant/default value you’re describing? At least in some cases?

 

LindsayRaabe_FPCWA

@Bud I could see those types of options being of value in some cases (just not in my posted example). Could even be a drop-down list of options under the Default Value, or custom value. 

Bud
by

Merged from a separate idea.


When using the Append geoprocessing tool:

We have the option to use the field map to reconcile field differences between the Input data and Target data. 

It would be helpful if we had an additional option under field sources to input a system date value instead of matching a field or doing a field calculation after-the-fact.

Kind of like “Bind Variables” in other enterprise systems:

https://bportaluri.com/2012/08/variables-dynamic-queries-conditional-expressions.html
https://www.ibm.com/support/pages/username-appname-and-other-special-bind-variables-you-can-use

Could that functionality be added to the Append tool?

JonathanNeal

@Bud Tools support %t%, but looks like I can only make it stick to a double field, not a DateTime field.

JonathanNeal_0-1681781999616.png

 

Bud
by

Merged from a separate idea.

When using the Append geoprocessing tool:

We have the option to use the field map to reconcile field differences between the Input data and Target data. 

It would be helpful if we had an additional option under field sources to input a custom expression (SQL or field calculation) instead of matching a field or doing a field calculation after-the-fact. (It can be messy/error prone to go into the attribute table afterwards to find the new rows and do the field calculation.)

Example (SQL):

case when class='A' then 1 else 0 end
sysdate + 1