idea Options to Remove Metadata Location URL Information in ArcGIS Pro Ideas

Options to Remove Metadata Location URL Information

StanJohnston — Fri, 10 May 2019 16:10:51 GMT

I've been using ArcGIS for 15 years and when a few years ago during the Sony database hack, I thought of a useful modification to the metadata editor in ArcGIS. We all know that there is an automatically updated location URL where our data is stored. The problem I found is that when you export your SHP file or feature class within a file geodatabase you send automatically have location information in that URL within the metadata. If it's a SHP file it's 'easy' enough to remove the location information from the XML file but, if it's within a file geodatabase it becomes more problematic. The reason for me making this observation is that I've noticed that file geodatabases that I've downloaded from government sources in the past, have the original file location URL info within the feature class metadata. This to me, in light of the Sony hack seemed to be a potential security issue.

How could this be a security issue?

In a scenario where an organization has a lot of geospatial data from a lot of different departments and they make that data available for download as SHP files or file geodatabases, either freely or through licence agreements, you could potentially if you downloaded all that free data, begin to see a partial representation of server names, file directory trees within an organization. That in my opinion should never be allowed.

A way to combat this would be by adding two elements to the metadata editing tools:

Add a check box which would allow for removal of the URL info. This would apply for geospatial data that is going to be offered up for public download.
Add a numerical entry box which would allow the metadata editor to selectively pick the directory tree depth, of the full directory to keep.

D:\Source_Data\Projects\Project_001\Industrial\2019\Spring.gdb

would become

\Spring.gdb

or alternatively

\2019\Spring.gdb

You can imagine that you don't want the public knowing or being able to know network topology by downloading enough data sources.

Some would say that you can alleviate this by loading everything into ArcGIS Online, which is not necessarily a bad idea, but some GIS practioners actually like to have data to download and use the way they want to. In such cases where companies, governments, NGOs, academia, etc. want to provide geospatial data for download, freely or through licence agreement they need to be able to protect the network topology of their organization.

I'm sure the python programmers among you would simply say, "write a script" to remove that info, however, not everyone is a programmer, nor should they need to be. If there was some additional features to the metadata editing options in ArcGIS Desktop / Pro, then this could take care of this, as I see it security concern. Yes I know Desktop will be deprecated at some point, but still in ArcGIS Pro it could be added.

In an age where even Amazon web services storage buckets are being compromised by sloppy admins not properly securing their cloud storage, and nefarious actors from all across the internet trying to compromise systems, it should be something the ArcGIS users can help address by not allowing their directory trees and server names being freely offered up to the world if they don't want them to.

Thoughts?

Re: Options to Remove Metadata Location URL Information

DerekMStrout — Tue, 10 Aug 2021 13:26:41 GMT

Hi Stan,

Found your comment after thinking the same thing. Have you received any information or devised a solution?

Derek

Re: Options to Remove Metadata Location URL Information

KoryKramer — Tue, 10 Aug 2021 17:19:40 GMT

Have you looked at the "Without sensitive info" option here https://pro.arcgis.com/en/pro-app/latest/help/metadata/save-a-copy-of-an-item-s-metadata.htm#ESRI_SECTION1_6C818AD079E645BAB9414E1D49C67F4A ? Could you export the xml without paths and import the filtered xml to overwrite the datasets' metadata?

Re: Options to Remove Metadata Location URL Information

DerekMStrout — Tue, 10 Aug 2021 17:28:15 GMT

Hi Kory. I attempted to do just that in ArcCatalog but when I imported the XML, something didn't go quite right and it cleared out info that was previously there (Summary, etc.) and the local paths remained. I'm sure it's just user error between the different metadata types, but I will try it again through Pro. Thanks for following up.

Re: Options to Remove Metadata Location URL Information

RiceAdam — Mon, 22 Nov 2021 23:49:52 GMT

@KoryKramer Was a solution ever found for this issue? We've had it flagged as a security issue and I need to resolve this for multiple layers, feature servers etc.

It's just not feasible to pull every affected layer back into Pro, save/export the XML Without Sensitive Info and then re-upload everything.

There must be a way to implement/enforce Without Sensitive Info on the entire Portal. As an admin I don't want to rely on users performing those steps every time.

Re: Options to Remove Metadata Location URL Information

Anonymous User — Thu, 06 Jan 2022 16:47:18 GMT

We have exactly the same objection. Some of our data has to be posted for download as zipped file geodatabase feature classes. There is no way in Pro, out of the box, to obliterate the internal file paths in the feature class metadata. If the metadata are saved without sensitive information from Pro, and then imported into a feature class with no metadata, the internal file path is automatically re-added to the metadata. Come on, Esri, get your act together and STOP IT!

Our work-around for now is to use ArcCatalog to accomplish what we need (not a long-term solution.) And we have a programmer working to see if we can create custom code in Pro to obliterate what we don't want to see in the metadata, as well as providing a way to move metadata around in a model. This is work we should not have to be doing - it is replacing functionality that existed in ArcCatalog.

Re: Options to Remove Metadata Location URL Information

RiceAdam — Thu, 15 Sep 2022 02:07:57 GMT

Here's the Case Resolution we received. Haven't yet attempted it. I also received locked ISO information (which i don't think will resolve the issue)...

Case Resolution

Because 'all' the metadata is stored in the layer (and the setting in Pro and Portal control the visibility) it sounds like you're looking to remove that confidential information from the metadata entirely so someone can't just change the metadata style and then lift your usernames.

We have some other script samples for this:
https://pro.arcgis.com/en/pro-app/latest/arcpy/metadata/migrating-from-arcmap-to-arcgis-pro.htm#:~:text=Remove%20content%20from%20an%20item%27s%20metadata

You would need to adjust for fields you're looking to remove.

The other option is to save a copy of the metadata (with the option 'remove_all_sensitve_info') and then reimport that exported metadata back into the layer.
https://pro.arcgis.com/en/pro-app/latest/arcpy/metadata/migrating-from-arcmap-to-arcgis-pro.htm#:~:text=XSLT%20stylesheet%20resources.-,Save%20a%20copy%20of%20an%20item%27s%20metadata,-The%20capabilities%20that

The third option would be to just go into this metadata in Pro or Portal (with item description) and manually clear all the confidential information, then save.

Enforcing an ISO

In your file (if you wanted ISO 19115-3 metadata for all instances of Pro), you would introduce the following line to the config file.

<!-- <MetadataStyle isLocked="true">ISO 19115-3 XML Schema Implementation</MetadataStyle>

Re: Options to Remove Metadata Location URL Information

Will — Wed, 14 Dec 2022 23:09:21 GMT

Is the metadata export option "without sensitive info" accessible via python or some other way to use in batch format? Thanks

Re: Options to Remove Metadata Location URL Information

Anonymous User — Thu, 15 Dec 2022 13:37:08 GMT

As far as we have been able to tell, only by creating custom script using ArcPy Metadata module: methods exportMetadata, deleteContent, and copy.

Re: Options to Remove Metadata Location URL Information

RiceAdam — Thu, 15 Dec 2022 20:19:00 GMT

I've been following this up, hopefully this helps 🙂

Disable metadata logging
The Esri geoprocessing tools insert the metadata into the enterprise geodatabases by default. This behavior can be disabled in ArcGIS Desktop and ArcGIS Pro, or by using a Python script.

Python:
For script tools and standalone scripts (scripts that run outside of an ArcGIS application), you can enable or disable history logging using the SetLogHistory function. We recommend disabling logging whenever possible and especially for compress and analysis scripts by adding this line to the top of your script: arcpy.SetLogHistory(false)

ArcGIS Desktop/Pro:
In ArcGIS Pro, logging behavior can be disabled by unchecking the "Write geoprocessing operations to dataset metadata" option under Project > Options > Geoprocessing.

In ArcCatalog/ArcMap, logging behavior can be disabled by unchecking the Logging option in the Geoprocessing > Geoprocessing Options dialog.

GeoDatabase:
How To: Delete geoprocessing history from a geodatabase in ArcGIS Pro using Python (esri.com)

Re: Options to Remove Metadata Location URL Information

PaulHoefflerGISS — Tue, 16 Dec 2025 19:46:53 GMT

Cleaning up metadata within ArcGIS Pro has been a problem for years. There have been some major improvements, but it seems as though there are still issues.

I can export the metadata from a file geodatabase and a number of feature classes to XML files using REMOVE_ALL_SENSITIVE_INFO and all of the sensitive info is scrubbed. Along the way I also set deleteContent('GPHISTORY') and deleteContent('ENCLOSED_FILES') for good measure. Great. Inspecting the XML files directly shows that they are clean.

I then use the ArcPy importMetadata method to read the XML files back in and the save method to save them to the original feature class.

Viewing the metadata in ArcGIS Pro, using the FGDC CSDGM Metadata style and either the classic or non-classic metadata editor, I find a Resource Details section, ArcGIS item properties, Location, storing a file:// UNC-ish path that has my local machine name and also my username stored in the path. I can move where I'm working, but the machine name/UNC path will be embedded in the metadata and still present a problem. This section isn't shown at all in either metadata editor.

Using the third-party add-in for Pro, Clear Geoprocessing History (ArcGIS Pro 3.x), doesn't address this, as it's not actually geoprocessing history.

As a long shot, I also set arcpy.SetLogHistory(False). That didn't help.

How can we completely remove sensitive information from metadata in ArcGIS Pro? It doesn't seem that we can do this within the application, in either metadata editor, or using an add-in or API?

Re: Options to Remove Metadata Location URL Information

RiceAdam — Wed, 17 Dec 2025 00:42:34 GMT

@PaulHoefflerGISS I'm not sure if it's actually possible. I have also had pentests performed years ago that informed us that metadata can also be retrieved via changing the REST call output response (i.e. - &f=JSON/HTML etc.). I'm not sure exactly how to replicate it now.

Regardless, afaik you can't. There was a tool that removed DIR/folder paths but I believe that was "REMOVE_ALL_SENSITIVE_INFO". We do get flagged though since exposing DIRs is a security vector for penetration, currently we've disabled the REST service 'view' to help mitigate this issue, but fundamentally someone could still loop through these services exposing potentially personal or system info.

Re: Options to Remove Metadata Location URL Information

PaulHoefflerGISS — Wed, 17 Dec 2025 14:21:08 GMT

It seems that some of this comes down to format/delivery, which is inconsistent when it comes to metadata (apparently).

I'm looking at the metadata for feature classes in a file geodatabase. The file geodatabase-level metadata does not contain the sensitive path, but the metadata for each feature class does.

Reviewing a hosted feature layer in ArcGIS Online that was published using this same data, some of the metadata is published through, but much of it is not.

It's possible that this is a setting or configuration in ArcGIS Pro, in the map, in ArcGIS Online, in the item, or elsewhere - or that other pieces of metadata are lurking somewhere less obvious that can be exposed through the REST API.

The inconsistency across formats and levels within formats complicates it all.