idea Geoprocessing History in Metadata in ArcGIS Pro Ideas

Geoprocessing History in Metadata

BillMyers — Tue, 08 Nov 2011 22:14:44 GMT

I recommend 2 changes related to the geoprocessing history stored in the metadata:

1. Allow users to enable/disable storing geoprocessing history in the metadata
2. In the metadata editor, allow the geoprocessing history items to be deleted in mass and one by one

Re: Geoprocessing History in Metadata

RayChilcote — Wed, 16 Nov 2011 16:11:20 GMT

Agree this needs to be implemented as a normal tool, rather than...

Workaround for removing gp history from metadata:
http://support.esri.com/index.cfm?fa=knowledgebase.techarticles.articleShow&d=31620

Re: Geoprocessing History in Metadata

GregBonynge — Thu, 01 Dec 2011 16:28:20 GMT

Thanks for posting the workaround, Captain.

Re: Geoprocessing History in Metadata

NathanShaub — Tue, 15 Jan 2013 19:21:17 GMT

We have several Python scripts that perform daily geoprocessing tasks. Since geoprocessing history is automatically logged, the GDB_ITEMS table in our geodatabases bloats considerably and affects performance significantly. I discovered the effect on performance by first taking the workaround code suggested by ESRI and modifying it so that I could run it at the geodatabase level to affect all objects in the geodatabase (the original workaround would take a VERY long time to actually perform if I had to go to each feature class individually). Once I had cleaned up over two years of geoprocessing history from the GDB_ITEMS table of all of my geodatabases, I found that:

1) The reserved disk space for all of the SDE databases on my SQL Server dropped by nearly 11 Gb.
2) My automation scripts run MUCH faster. For instance, one job that performs a DB compression and updates statistics went from taking 4 hours and 13 minutes to taking 8 minutes! Another script that deletes several point feature classes and recreates them using X/Y coordinates downloaded from a business database went from taking 2 hours and 25 minutes to taking 15 minutes.

Having some method to manage logging to the GDB_ITEMS table would be an excellent enhancement!

Re: Geoprocessing History in Metadata

KarenFolger — Wed, 27 Feb 2013 22:17:54 GMT

I would also recommend that there be a way to programmatically turn off logging to metadata of scripts doing things like Analyze to the entire SDE set of features. It's huge for us to have all this garbage in the metadata.

Re: Geoprocessing History in Metadata

DuncanHornby — Tue, 25 Mar 2014 11:17:37 GMT

ArcGIS is now 10.2.1 and there is still no way of deleting geo-processing logs from the metadata when in the Data Source Item Description editor.

I do think that when a geo-processing tool is run and metadata is updated its a great way of keeping track of what is done but sometimes you do something that is logged and you simply want to remove it but not the rest of the history.

For example you add a field and do a calculation on it, this is logged. You realise the calculation was incorrect so you re-run the calculation, this gets logged. I would want to go back into the metadata editor and remove the first calculation so the values in the field are synchronised with what the geo-processing log is showing.

You can edit/remove other items in the metadata (usually by a red cross button) why can't you make the Geoprocessing History do the same?

Re: Geoprocessing History in Metadata

AaronCohen — Thu, 01 May 2014 14:14:06 GMT

This definitely should be a property of a featureclass.
GP History ON or OFF.

Re: Geoprocessing History in Metadata

curtvprice — Fri, 08 Aug 2014 22:09:15 GMT

There is a file that can be used with the XSLT Transformation tool to zap the GP history:

C:\ArcGIS\Desktop10.1\Metadata\Stylesheets\gpTools\remove geoprocessing history.xslt

So you could build a model build and script to do this for you on a regular basis when your use-case demands it. This unfortunately is not Hornbydd's use case where he wants to just delete a selected few.

Re: Geoprocessing History in Metadata

ThomasWhitfield — Tue, 07 Apr 2015 13:54:59 GMT

This is most annoying especially when having to release data to the public. My metadata document is 15 pages, with an additional 24 pages of geoprocessing steps that users do not need or care about. All they want is the data with a concise metadata document.
.

Re: Geoprocessing History in Metadata

FatihDur — Mon, 14 Nov 2016 22:26:35 GMT

Having realised a need for relatively opposite case, as mentioned in Allow IN_MEMORY datasets to store metadata generated by geo-processing tools , I think there could be an interim option instead of on/off switch which let users to control the amount of history to keep with the data items, say the last 20 geoprocessing actions. This history info is just auxiliary most of the time but sometimes it may become critical, particularly if a data item has a lineage related to the source items and their locations (say intersect operation applied and accompanying datasets to control source data versions, instead of rerunning a long intersect operation to make sure you used correct source datasets).

Re: Geoprocessing History in Metadata

DanaNolan — Fri, 24 Mar 2017 15:28:00 GMT

Great idea. I believe the default should be on, though. I have figured out the what and where hundreds of files just from the GP history. Most of the field calc history is a waste of time to read, but the appends are useful. None of these files had real metadata (documentation), and you can't count on most people to create any metadata, especially non-GIS non-IT people.

Re: Geoprocessing History in Metadata

Natalie_Runyan — Wed, 23 Aug 2017 00:04:42 GMT

In addition, I would appreciate the option to add the geoprocessing step that was used to generate the initial feature class. This has been requested in another form, as in to carry forward geoprocessing history from input feature classes that result in new features. In my example, we'd like to see the configuration parameters for our annotation feature classes show up in the metadata without having to manually transfer them. The create annotation from feature labels tool runs, and provides the details on reference scale, cardinality, etc in the annotation layer's properties, but none of this info is included in the metadata, nor is it formatted in way that is easy to cleanly copy/paste from the feature class Properties.

Re: Geoprocessing History in Metadata

RyanDavis1 — Thu, 20 Feb 2020 17:45:11 GMT

I'll add that this could be considered a security issue by some organizations. Full file paths, including network drives, are included in metadata and some groups might not be keen on inadvertently publicizing this.

Re: Geoprocessing History in Metadata

KoryKramer — Fri, 26 Jun 2020 12:45:09 GMT

1. Allow users to enable/disable storing geoprocessing history in the metadata

In ArcGIS Pro, see Geoprocessing history—ArcGIS Pro | Documentation

2. In the metadata editor, allow the geoprocessing history items to be deleted in mass and one by one

It looks like in the metadata's Geoprocessing History, one could individually choose to export (or not) processes.

If the main use case is when sharing exported metadata, see Export metadata to a standard format—ArcGIS Pro | Documentation

Re: Geoprocessing History in Metadata

PaulHoefflerGISS — Tue, 08 Sep 2020 15:54:04 GMT

Kory Kramer‌, unfortunately we can't stop geoprocessing history from writing to the metadata when using Python packaged with Pro due to #BUG-000128506 Arcpy.SetLogHistory(False) is inconsistent when running in a python script in a Python 3 environment.

On the second point, based on testing reported by Esri Technical Support, exporting the metadata from Pro without Sensitive Information or Machine Names and then importing it back into Pro will not remove this information from the source metadata. We need to redact the embedded metadata.

Please let me know if you find either of these not to be the case, or have additional thoughts on point #2.

Thanks!

Re: Geoprocessing History in Metadata - Status changed to: Under Consideration

NaicongLi — Fri, 29 Oct 2021 15:14:44 GMT

Re: Geoprocessing History in Metadata - Status changed to: In Product Plan

NaicongLi — Fri, 01 Dec 2023 18:05:28 GMT

Re: Geoprocessing History in Metadata - Status changed to: Implemented

KoryKramer — Fri, 10 May 2024 15:03:48 GMT

This is implemented in ArcGIS Pro 3.3. See Your Ideas in ArcGIS Pro 3.3 for a full list of all user ideas you can look forward to! And be sure to check out the release blog and what’s new help topic and video to learn more about all the new features and functionality in the release.