Select to view content in your preferred language

Feature Class to Feature Class Options to Preserve OBJECTID and GLOBALID Foreign Keys

185
3
2 weeks ago
Status: Needs Clarification
Labels (1)
RichardFairhurst
MVP Honored Contributor

Being able to retain foreign key values in any extract of data that the user wants to create that match the source primary key values is fundamental to most data administration workflows.  Extractions of records from a source that relies on its OBJECTIDs and/or its GLOBALIDs as its primary key in relationship to other records should not be relationally destroyed relative to the source that they came from if the user has an explicit requirement to preserve those values as a foreign key in their extracted output.  However, this kind of relational destruction is currently always happening whenever the Feature Class to Feature Class and Table to Table tools are used, which are the default tools for creating extractions in ArcGIS Pro.

The Feature Class to Feature Class and Table to Table tools (and possibly other tools) need to add two options that would allow the user to create new foreign key fields that preserve the OBJECTID and/or GLOBALID values of the source feature class/table in the output.  If the user checks either of these options a control in the tool should suggest a default Field Name for each which the user could override.  Validation should ensure that the source contains OBJECTID/GLOBALID fields before enabling these options, and that the user definable field names are valid and not already in the source feature class/table or duplicated if the user checks both options.  The source OBJECTID values would be preserved in a Long field and the source GLOBALID values would be preserved in a GUID field.

The use case for this is when the only field(s) that contain a unique value for each of the source features or rows is the OBJECTID and/or the GLOBALID.  Often the source table is not owned by the user and cannot be modified to store these values into a Long and/or GUID field in advance.  Using either of these tools overwrites the source OBJECTID and GLOBALID values with new values in the output, especially when the user only wants to extract a subset of records.  Having to copy the source data into a workspace the user can modify to add these fields in advance is very time consuming when the source contains large numbers of features and/or rows and they have to repeatedly copy the source every time they need a new extraction if they have any reason to believe that the source contains new records that they want. 

This Copy workaround is especially burdensome to the user who only wants a small subset of the source features or rows and not an entire copy to apply the fundamentals of relational database design with their output.  There are also many instances when the Copy function is not supported for even doing this workaround when the source feature class is part of a topology within a feature dataset.

Incorporating the preservation of the OBJECTIDs in a static foreign key field in the output of these two tools is absolutely possible, as demonstrated by numerous tools like Spatial Join, Union, Intersect, Polygon to Line, Multipart to Singlepart, etc. all of which preserve a foreign key FID field in their output.  Expending this kind of foreign key preservation behavior to also allow for the preservation GLOBALID values in a GUID foreign key output field should not be significantly more difficult.  This would be a great enhancement and time saver for users who need extractions from Parcel Fabric data sources and any other advanced feature dataset capabilities that are built around GLOBALIDs as primary keys for relationships.

And adding a GLOBALID to GUID option to many other tools that currently by default preserve OBJECTIDs in a foreign key FID field would be great as well as a time saver.  But that is less critical, since at least those outputs already preserve usable foreign key values from the source OBJECTIDs in their outputs by default.  Additionally some of those tools support field mapping, which may allow the user to accomplish the GLOBALID value preservation themselves.  But the Feature Class to Feature Class and Table to Table tools currently do not offer any option that allows the user to preserve any usable foreign key values in their output relative to any sources that rely on OBJECTIDs or GLOBALIDs as their sole unique primary keys.

If the existing Feature Class to Feature Class and Table to Table tools should not be modified to add this option because they are accessible to all license levels and intentionally provide limited functionality as a result, a new pair of Conversion tools called Feature Class to Feature Class with FIDs and Table to Table with FIDs should be added for Advanced License users at minimum.  These two new tools would definitely be a selling point for many Basic and Standard users to consider upgrading to an Advanced license, as well as two very befitting and useful additions for truly Advanced users.

3 Comments
SSWoodward
Status changed to: Needs Clarification

Thanks for the Idea @RichardFairhurst 

Feature Class to Feature Class and Table to Table are deprecated geoprocessing tools and are not eligible for additional development.  These tools, and their current equivalents, Export Features and Export Table, honor the 'Preserve Global ID' environment setting.

Give this setting a look and let us know if it meets the needs you've shared with your global ID based relationship classes. 

RichardFairhurst

I was using the ArcMap tool when I wrote this post and I am using ArcGIS Pro which apparently predates the release of the Export Features and Export Tables tools.  So at this time I cannot test the Export Features or Export Tables tools.  However, ArcGIS 2.9 does offer the "Preserve Global IDs" environmental setting and that does work with Feature Class to Feature Class and Table to Table at 2.9.  So that aspect of my request can be considered Implemented.

However, the bulk of the data I deal with only uses ObjectID as the primary key unique record value.  Preserving these values is paramount to many of my workflows.

I do note that the Export Features and Export Table tools include a field map, which may allow me to preserve the ObjectID.  However, other tools that include a field map have been notoriously unsuccessful and unreliable at preserving the ObjectID values up through ArcGIS Pro 2.9.  So, I cannot test if this is a viable solution, since the Field Map is not available with Feature Class to Feature Class or Table to Table.

I do not foresee upgrading to ArcGIS Pro 3.x any time soon in my organization, although I may push to upgrade my own ArcGIS Pro installation sooner if you can confirm the Export Features and Export Table field map offers ObjectID value preservation.  If you can confirm the Field Map capability of the Export Features and Export Table tools support ObjectID preservation, then you can mark this idea as Implemented.  But if it does not preserve the ObjectID foreign key values reliably, then consider implementing this aspect of this idea through the field map or an an additional option.

RichardFairhurst

Additionally, if the field map does support ObjectID preservation, please describe the set up steps, since just checking the ObjectID field in the field map does not reliably work in my experience. 

For example, checking the ObjectID field in the field map of the Dissolve tool completely fails to preserve the actual unique ObjectID values of the source.  Instead it behaves like I set it to use the Min or First option rather than to get the Unique values of the ObjectID.  The Dissolve tool output seems to generate multiple features as though it was trying to preserve the Unique values, but at the end multiple records have duplicate values in all of the fields because the real unique ObjectID values were not preserved.