topic Re: List metadata with Python in Python Questions

List metadata with Python

JoshThompson — Mon, 20 Jun 2011 20:00:18 GMT

I have a list of feature classes from a geodatabase I created from a Python script and would like to add their respective metadata or a portion of (such as the description portion of the ArcGIS metadata style) to this list by feature class. Is this possible with Python or am I asking too much?

Thanks,
Josh

Re: List metadata with Python

DarrenWiens2 — Sat, 11 Dec 2021 10:50:16 GMT

I don't know how to get at metadata directly, but you can export the metadata to xml (I believe this tool only works FROM INSIDE PYTHON in SP1 and beyond) and then read through the xml with an ElementTree. I can't remember if the translator doesn't export all FGDC tags or doesn't import all FGDC tags correctly, but it's one of the two, so beware.

import arcpy, sys
from xml.etree.ElementTree import ElementTree
from xml.etree.ElementTree import Element, SubElement
from arcpy import env
env.overwriteOutput = True
fcpath = r"X:\working\metadata\BlankGDB.gdb\FullMetaFC" # the input feature class
translatorpath = r"D:\Program Files (x86)\ArcGIS\Desktop10.0\Metadata\Translator\ARCGIS2FGDC.xml" # the translator file to use - it should be in an installed folder like this
xmlfile = r"X:\working\metadata\FullMetaFC.xml" # the output xml file
arcpy.ExportMetadata_conversion(fcpath, translatorpath, xmlfile) # export the metadata to xml

tree = ElementTree() # make an ElementTree object
tree.parse(xmlfile) # read the xml into the ElementTree

spot = tree.find("idinfo/citation/citeinfo/title") # find whatever tag you want
print spot.text # print the text between the tags

edit: updated paths as Curtis advises below.

Re: List metadata with Python

curtvprice — Sat, 11 Dec 2021 10:50:19 GMT

Danger, the code above uses normal strings with backslashes. These are escape characters in Python and will not work. Easily fixed by making them "raw" strings as follows:

fcpath = r"X:\working\metadata\BlankGDB.gdb\FullMetaFC"

Also you may want to read up on the Arc 10 metadata story, as both FGDC and ArcGIS 10 metadata may be in the metadata file and you will want to decide how you can handle that.

A New Approach To Metadata at ArcGIS 10 (Part 1)
http://blogs.esri.com/Dev/blogs/arcgisdesktop/archive/2011/01/03/A-new-approach-for-Metadata-with-ArcGIS-10.aspx

Re: List metadata with Python

DarrenWiens2 — Mon, 20 Jun 2011 21:23:25 GMT

Oh, I can never remember which is which. It works fine as is, I suppose because there were no characters to be escaped...? In any case, you should probably fill in your own paths, as I doubt you have access to my X:/.

Re: List metadata with Python

curtvprice — Sat, 11 Dec 2021 10:50:22 GMT

It works fine as is, I suppose because there were no characters to be escaped...?

Just don't ever name any folders or files starting with n, r, or t, etc....:

>>> "a\z" # "\z" is not a special string literal, so OK
'a\\z'
>>> "a\t"  # this is "a" followed by <tab>
'a\t'
>>>

See:
http://docs.python.org/reference/lexical_analysis.html#string-literals

Re: List metadata with Python

JoshThompson — Tue, 21 Jun 2011 17:37:13 GMT

Hey Guys,

Thanks for the help. So far, all the info you have given me has worked very well. I am able to list the "Description" portion of the metadata for an individual feature class.

Is there a way to integrate this code with the "Listfc" script? My ultimate goal is to list all feature classes, by dataset, and their respective metadata from a file GDB to give to staff in various dept. I am a beginner and am just not able to make this work on my own. I appreciate any help you can give.

Josh

Re: List metadata with Python

curtvprice — Tue, 21 Jun 2011 18:38:07 GMT

I am able to list the "Description" portion of the metadata for an individual feature class.

Cool! Please post your working script either in full or as an attachment. Wrapping this in a loop should be pretty straighforward.

Re: List metadata with Python

JoshThompson — Tue, 21 Jun 2011 19:53:44 GMT

Here is the script I used (per dkwiens) to extract and print the "Description" portion of the metadata for an individual feature class:

import arcpy, sys
from xml.etree.ElementTree import ElementTree
from xml.etree.ElementTree import Element, SubElement
from arcpy import env
env.overwriteOutput = True
fcpath = r"D:\GIS\GDB\Lakeland\Lakeland.gdb\AddPoints\AddressPts" # the input feature class
translatorpath = r"C:\Program Files (x86)\ArcGIS\Desktop10.0\Metadata\Translator\ARCGIS2FGDC.xml" # the translator file to use - it should be in an installed folder like this
xmlfile = r"D:\GIS\metadata\test.xml" # the output xml file
arcpy.ExportMetadata_conversion(fcpath, translatorpath, xmlfile) # export the metadata to xml

tree = ElementTree() # make an ElementTree object
tree.parse(xmlfile) # read the xml into the ElementTree

spot = tree.find("idinfo/descript/abstract") # find whatever tag you want
print spot.text # print the text between the tags

If you can help me tie this into the fcList script, I will be indebted to you.

Josh

Re: List metadata with Python

DarrenWiens2 — Sat, 11 Dec 2021 10:50:24 GMT

Rather than call one feature class (fcpath), you'll call each feature class, one by one, from the fcList. This is totally untested, so you might run into errors (tricky areas might include overwriting the xml file, or reusing the ElementTree). You could also dynamically name the xml (e.g. with a similar name to the feature class name) inside the loop if you wanted separate xml files, rather than just one overwritten each time.

import arcpy, sys
from xml.etree.ElementTree import ElementTree
from arcpy import env
env.overwriteOutput = True
env.workspace = r"D:\GIS\GDB\Lakeland\Lakeland.gdb\AddPoints" # set the workspace for use in  ListFeatureClasses
fcList = arcpy.ListFeatureClasses() # lists all feature classes from inside env.workspace

translatorpath = r"C:\Program Files (x86)\ArcGIS\Desktop10.0\Metadata\Translator\ARCGIS2FGDC.xml" # the translator file to use
xmlfile = r"D:\GIS\metadata\test.xml" # the output xml file

for fc in fcList:
    arcpy.ExportMetadata_conversion(fc, translatorpath, xmlfile) # export the current fc's metadata to xml
    tree = ElementTree() # make an ElementTree object
    tree.parse(xmlfile) # read the xml into the ElementTree
    spot = tree.find("idinfo/descript/abstract") # find whatever tag you want
    print spot.text # print the text between the tags

Re: List metadata with Python

curtvprice — Tue, 21 Jun 2011 22:25:54 GMT

I tried today to use the XSLTransform_conversion tool to just extract out everything (avoiding a transform) but I am getting an error message that appears to be a bug in tool validation (which I sent to ESRI support). I've attached this python script that is a function to pull out the ArcGIS metadata elements more directly (no need for the FGDC transform).

The Arc 10 metadata schema is documented in the file:

\ArcGIS\Desktop10.0\Metadata\Translator\Rules\ArcGIS metadata v1.dtd

though I have found it easier to just look in an existing ".shp.xml" file.

Another approach to this problem is to create an xslt stylesheet and run the XSL Transformation tool (conversion toolbox) to pull out the information you want into an XML file.

Re: List metadata with Python

DarrenWiens2 — Wed, 22 Jun 2011 04:40:39 GMT

I tried today to use the XSLTransform_conversion tool to just extract out everything (avoiding a transform) but I am getting an error message that appears to be a bug in tool validation (which I sent to ESRI support).

I believe this was fixed in SP1. I can't remember the exact error I was getting, but it's gone now.

Re: List metadata with Python

JoshThompson — Wed, 22 Jun 2011 18:40:02 GMT

Darren,
When I attempted to use the code you supplied, it simply moves to a new prompt. No errors, no output, nothing. Just a new prompt.

Curtis,
I am getting the same error.

Re: List metadata with Python

curtvprice — Sat, 11 Dec 2021 10:50:27 GMT

The following attached script works as a script tool. I get the error when I try to run it from IDLE or the command line... apparently there is something in the environment that is messing me up. I'm running 10.0 SP 2 on XP 64.

import sys, os, arcpy

def getMetadataText(Dataset,Element="dataIdInfo/idPurp"):
    """Extracts metadata contents from ArcGIS metadata
  arguments
    Dataset - Path to an ArcGIS dataset
    Element - XML path in ArcGIS metadata. 
    Default: "metadata/dataIDInfo/idPurp" ("Summary")
"""
    from xml.etree.ElementTree import ElementTree
    try:
        # Get arcgis to fgdc translator path
        ARCGISHOME = arcpy.GetInstallInfo("desktop")["InstallDir"]
        XSLT = os.path.normpath(os.path.join(
            ARCGISHOME,
            "Metadata/Stylesheets/gpTools/exact copy of.xslt"))
        xmlTemp =  os.path.join(os.environ["TEMP"],"xx_xmltemp.xml")
        # export the metadata to xml   
        arcpy.env.overwriteOutput = True       
        arcpy.XSLTransform_conversion(Dataset,XSLT,xmlTemp)
        # parse out Description element from ArcGIS 10 metadata
        tree = ElementTree() # make an ElementTree object
        tree.parse(xmlTemp) # read the xml into the ElementTree
        Data = tree.find(Element) # find whatever tag you want
        del tree
        arcpy.Delete_management(xmlTemp)
        return Data.text # return the contents as text
    except Exception, xmsg:
        raise Exception, str(xmsg)

# You could put a loop here around this to print out Summary for a list of datasets
dataset = r"E:\work\work.gdb\BeetleSprayArea.shp"
text = getMetadataText(dataset)
arcpy.AddMessage(text)

Re: List metadata with Python

JoshThompson — Thu, 23 Jun 2011 18:03:18 GMT

Thanks Curtis, but I'm afraid this code is beyond my skill level. I think I have gotten in over my head with this and need some more practice time with Python before trying this again. Thanks again for your help.

Josh

Re: List metadata with Python

JoshThompson — Fri, 29 Jul 2011 18:30:03 GMT

All,
I brushed up on Python a little and attempted to tackle this script again. I have had some success in that I can now generate a list of feature classes and the "Summary" component of the metadata for all features in a feature dataset. The script below will produce that information.

import arcpy, sys
from xml.etree.ElementTree import ElementTree
from arcpy import env
env.overwriteOutput = True
env.workspace = r"D:\Lakeland_GIS\GDB\Lakeland\Lakeland.gdb\AddPoints"

translatorpath = r"C:\Program Files (x86)\ArcGIS\Desktop10.0\Metadata\Translator\ARCGIS2FGDC.xml"
xmlfile = r"U:\my docs\GIS Projects\Python\Scripts\List Metadata\working_test.xml"

datasetList = arcpy.ListDatasets()
datasetList.sort()
for dataset in datasetList:
    print dataset
fcList = arcpy.ListFeatureClasses()
fcList.sort()
for fc in fcList:
    arcpy.ExportMetadata_conversion(fc, translatorpath, xmlfile)
    tree = ElementTree()
    tree.parse(xmlfile)
    spot = tree.find("idinfo/descript/purpose")
    print fc
    print spot.text

What I am attempting to do is generate a list of all feature classes, by dataset, for an entire geodatabase. Ex. - (Dataset>Feature Class>Metadata). As of now I can only get FeatureClass>Metadata for one feature dataset at a time.

Help! If anyone could help me complete this I would be truly appreciative.

Thanks,
Josh

Re: List metadata with Python

curtvprice — Fri, 29 Jul 2011 19:27:34 GMT

You really need to put these things in code blocks so we can see your loops.

bbcode formatting: http://forums.arcgis.com/misc.php?do=bbcode

Re: List metadata with Python

JoshThompson — Fri, 29 Jul 2011 20:03:06 GMT

Sorry, Curtis. I didn't realize the format doesn't carry over into the message. Lesson learned. Sending the code as an attachment.

Thanks,
Josh

Re: List metadata with Python

curtvprice — Sat, 11 Dec 2021 10:50:30 GMT

You were very close! I didn't handle any metadata on the feature dataset itself; you could add that.

import arcpy, sys
from xml.etree.ElementTree import ElementTree
from arcpy import env
env.overwriteOutput = True
env.workspace = r"D:\Lakeland_GIS\GDB\Lakeland\Lakeland.gdb\AddPoints"

AGSHOME = arcpy.GetInstallInfo("Desktop")["InstallDir"]
translatorpath = AGSHOME + r"Metadata\Translator\ARCGIS2FGDC.xml" 
xmlfile = r"U:\my docs\GIS Projects\Python\Scripts\List Metadata\working_test.xml" 
  
# list any standalone feature classes
fcList = arcpy.ListFeatureClasses()
fcList.sort()
for fc in fcList:
    arcpy.ExportMetadata_conversion(fc, translatorpath, xmlfile) 
    tree = ElementTree() 
    tree.parse(xmlfile) 
    spot = tree.find("idinfo/descript/purpose") 
    print fc
    print spot.text
        
# list feature datasets
datasetList = arcpy.ListDatasets()
datasetList.sort()
for dataset in datasetList:
    print dataset
    # list feature classes inside the dataset
    fcList = arcpy.ListFeatureClasses("","",dataset)
    fcList.sort()
    for fc in fcList:
        arcpy.ExportMetadata_conversion(fc, translatorpath, xmlfile) 
        tree = ElementTree() 
        tree.parse(xmlfile) 
        spot = tree.find("idinfo/descript/purpose") 
        print "  " + fc
        print "  " + spot.text

Re: List metadata with Python

JoshThompson — Tue, 02 Aug 2011 14:10:06 GMT

Beautiful! That did it. Thanks Curtis for all your help and everyone else who contributed. Also, the indention to distinguish datasets from feature classes was a nice touch. Hadn't thought of that...

Josh

Re: List metadata with Python

JoshThompson — Mon, 08 Aug 2011 18:49:44 GMT

All,

I just wanted to address two small issues with this script I have encountered:

First, the script works great in the Python window in ArcMap/ArcCatalog, but produces errors in Pythonwin:
(ExecuteError: Failed to execute. Parameters are not valid.
ERROR 000816: The tool is not valid.
Failed to execute (ExportMetadata).

Second, if the name of your feature class has the same name as the feature dataset it resides in, it will cause the script to fail. (ex. - Data.gbd>Contours>Contours) Will also terminate if no text is found in the metadata component it is looking for.

I just wanted to let anyone who may use this script to be aware of these issues.

Josh