nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

List metadata with Python

JoshThompson

I have a list of feature classes from a geodatabase I created from a Python script and would like to add their respective metadata or a portion of (such as the description portion of the ArcGIS metadata style) to this list by feature class. Is this possible with Python or am I asking too much?

Thanks,
Josh

Encontre mais publicações marcadas com

Accepted answers

All comments

blake.terhune

It's also better to include the specific error type you're expecting so you don't end up passing unexpected errors too. I don't know what type of error this is supposed to produce but here's an example.

<SPAN class="keyword token">try</SPAN><SPAN class="punctuation token">:</SPAN>  
    Abstract <SPAN class="operator token">=</SPAN> tree<SPAN class="punctuation token">.</SPAN>find<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"idinfo/descript/abstract"</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">.</SPAN>text  
<SPAN class="keyword token">except</SPAN> AttributeError<SPAN class="punctuation token">:</SPAN>  
    Abstract <SPAN class="operator token">=</SPAN> <SPAN class="string token">"No abstract"</SPAN>
<SPAN class="keyword token">except</SPAN><SPAN class="punctuation token">:</SPAN>
    <SPAN class="comment token"># Catch for all other unhandled exceptions</SPAN>
    <SPAN class="keyword token">raise</SPAN><SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

AndresCastillo

I believe the first way with the if else is similar to Duck Typing and the "look before you leap" lbyl non-pythonic methods.

As Curtis said, the latter with the try/except is pythonic:

https://www.youtube.com/watch?v=x3v9zMX1s4s

AndresCastillo

relevant resources:

What is the Metadata module—ArcGIS Pro | Documentation

arcpy-metadata · PyPI

GitHub - ucd-cws/arcpy_metadata: Python metadata editing classes for ArcGIS feature classes

using arcpy.da.walk to inventory data and export metadata to csv

Python Errors in IDLE when processing metadata

arcpy - Creating table containing all filenames (and possibly metadata) in File Geodatabase? - Geographic Information Sy…

Access and update map and layer metadata (arcpy.mp)

python - updating metadata for feature classes programatically using arcpy - Stack Overflow

Edit metadata for many ArcGIS items—ArcMap | Documentation

BrianColson

Curtis,

This appears to be really helpful code, but I am certainly still learning Python. I am trying to use it on multiple datasets, and loop through with multiple metadata xml files, and to pull out only three parts (Abstract, Summary, and Credits). The code appears to be working but it never pulls the xml file to get any information out of it. If you wish I can send you a sample xml file as well so you can see how it�??s formatted.

My ultimate goal was output a csv file that gives a dataset name, abstract, summary and credits for everything within a folder or geodatabase. I was going to set this up as a Toolbox Script, so the end user doesn't have to work with the python code. I don't think I am quiet there, but that�??s why there is a bunch of stuff commented out at the top.

Any suggestions you can provide would be greatly appreciated.

Thank you.

Brian

# Create the Geoprocessor object 
import arcpy, sys, os
from xml.etree.ElementTree import ElementTree
from arcpy import env

# Get input parameters
##env.workspace = arcpy.GetParameterAsText(0)
##xmlfile = arcpy.GetParameterAsText(1)
##outfile = arcpy.GetParameterAsText(2)
env.overwriteOutput = True
##AGSHOME = arcpy.GetInstallInfo("Desktop")["InstallDir"]
##translatorpath = AGSHOME + r"Metadata\Translator\ARCGIS2FGDC.xml"

#Output file
##outfile=open(outfile,'w')

#Dataset list function

# list any standalone feature classes
env.workspace = r"C:\projects\Vicinity.gdb"
AGSHOME = arcpy.GetInstallInfo("Desktop")["InstallDir"]
translatorpath = AGSHOME + r"Metadata\Translator\ARCGIS2FGDC.xml" 
fcList = arcpy.ListFeatureClasses()
fcList.sort()
for fc in fcList:
    fcname = str(fc)
##    outfile.write(fcname + "\n" + ",")
    print fc
    xmlfile = r"C:\projects\Testing\ " + fcname + ".xml"
    arcpy.UpgradeMetadata_conversion (fc, "ESRIISO_TO_ARCGIS")
    arcpy.ExportMetadata_conversion(fc, translatorpath, xmlfile) 
    tree = ElementTree() 
    tree.parse(xmlfile)
    spot = tree.find("abstract")
    if spot: # if spot is True (not None) 
        print spot.text 
    else:
        print "No Abstract"
    spot = tree.find("purpose") 
    if spot: # if spot is True (not None)
        print spot.text 
    else:
        print "No Summary"
    spot = tree.find("datacred") 
    if spot: # if spot is True (not None)
        print spot.text
    else:
        print "No Credits"
# list feature datasets
datasetList = arcpy.ListDatasets("","Feature")
datasetList.sort()
for dataset in datasetList:
    print dataset
    # list feature classes inside the dataset
    arcpy.env.workspace = dataset
    fcList = arcpy.ListFeatureClasses()
    fcList.sort()
    for fc in fcList:
        print "  " + fc
        arcpy.ExportMetadata_conversion(fc, translatorpath, xmlfile) 
        tree = ElementTree() 
        tree.parse(xmlfile) 
        spot = tree.find("descript/abstract") 
        if spot: # if spot is True (not None)
            print "  " + spot.text
        else:
            print "  No abstract"

curtvprice

If you're using 10.1 SP1 or later, there's a new approach for inventorying workspaces: arcpy.da.walk.

It could be very handy for the application discussed in this thread.

http://arcpy.wordpress.com/2012/12/10/inventorying-data-a-new-approach/

curtvprice

It's hard to tell because of the spurious "..." prefixes and that [post=166129]you didn't use a [noparse]

[/noparse] block,[/post] but I think you're missing an indent. 

Hopefully these fixes address both bugs [post=124021]noted above:[/post]

if the name of your feature class has the same name as the feature dataset it resides in, it will cause the script to fail. (ex. - Data.gbd>Contours>Contours) Will also terminate if no text is found in the metadata component it is looking for.
 

See changes below in red

import arcpy, sys
from xml.etree.ElementTree import ElementTree
from arcpy import env
env.overwriteOutput = True
env.workspace = r"G:\sdeConnections\win_auth\sde_published.sde"
AGSHOME = arcpy.GetInstallInfo("Desktop")["InstallDir"]
translatorpath = AGSHOME + r"Metadata\Translator\ARCGIS2FGDC.xml" 
xmlfile = r"C:\Users\mtenold\Downloads\test.xml" 
# list any standalone feature classes
fcList = arcpy.ListFeatureClasses()
fcList.sort()
for fc in fcList:
    print fc
    arcpy.ExportMetadata_conversion(fc, translatorpath, xmlfile) 
    tree = ElementTree() 
    tree.parse(xmlfile)
    spot = tree.find("idinfo/descript/abstract") 
    if spot: # if spot is True (not None)
        print spot.text 
    else:
        print "No abstract" 
# list feature datasets
datasetList = arcpy.ListDatasets("","Feature")
datasetList.sort()
for dataset in datasetList:
    print dataset
    # list feature classes inside the dataset
    arcpy.env.workspace = dataset
    fcList = arcpy.ListFeatureClasses()
    fcList.sort()
    for fc in fcList:
        print "  " + fc
        arcpy.ExportMetadata_conversion(fc, translatorpath, xmlfile) 
        tree = ElementTree() 
        tree.parse(xmlfile) 
        spot = tree.find("idinfo/descript/abstract") 
        if spot: # if spot is True (not None)
            print "  " + spot.text
        else:
            print "  No abstract"

MattTenold

Thank you for the help with the if exception part.   I just have one last question.   I run the script and it returns a list of my feature datasets only. Can you look at my script and see what I am doing wrong. Sorry I am more of a vb.net C#/ArcObjects programmer.   Python is a bit strange for me.

import arcpy, sys
... from xml.etree.ElementTree import ElementTree
... from arcpy import env
... env.overwriteOutput = True
... env.workspace = r"G:\sdeConnections\win_auth\sde_published.sde"
... AGSHOME = arcpy.GetInstallInfo("Desktop")["InstallDir"]
... translatorpath = AGSHOME + r"Metadata\Translator\ARCGIS2FGDC.xml"
... xmlfile = r"C:\Users\mtenold\Downloads\test.xml"
... # list any standalone feature classes
... fcList = arcpy.ListFeatureClasses()
... fcList.sort()
... for fc in fcList:
...     arcpy.ExportMetadata_conversion(fc, translatorpath, xmlfile)
...     tree = ElementTree()
...     tree.parse(xmlfile)
    spot = tree.find("idinfo/descript/abstract")
    if spot: # if spot is True (not None)
        print spot.text
    else:
        print "No abstract"
... # list feature datasets
... datasetList = arcpy.ListDatasets()
... datasetList.sort()
... for dataset in datasetList:
...     print dataset
... # list feature classes inside the dataset
... fcList = arcpy.ListFeatureClasses("","",dataset)
... fcList.sort()
... for fc in fcList:
...     arcpy.ExportMetadata_conversion(fc, translatorpath, xmlfile)
...     tree = ElementTree()
...     tree.parse(xmlfile)
    spot = tree.find("idinfo/descript/abstract")
    if spot: # if spot is True (not None)
...         print " " + fc
...         print " " + spot.text
    else:
            print "No abstract"

curtvprice

The thing is I keep getting an error since about half of the feature classes do not have text in the description field, if I add a description then the script continues. Here is the error I receive when it finds a feature class with no text in the description of the metadata. Runtime error <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'text' Isn't there a way to add an if else statement?

Here's an approach that may help:

spot = tree.find("idinfo/descript/abstract") 
if spot: # if spot is True (not None)
    print spot.text 
else:
    print "No abstract"

A more pythonesque approach tries and asks forgiveness:

try:
    Abstract = tree.find("idinfo/descript/abstract").text
except:
    Abstract = "No abstract"

MattTenold

The thing is I keep getting an error since about half of the feature classes do not have text in the description field, if I add a description then the script continues. Here is the error I receive when it finds a feature class with no text in the description of the metadata. Runtime error <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'text' Isn't there a way to add an if else statement?

curtvprice

You'd need to parse the metadata [post=121311]as shown above[/post] and work with what you find

        tree = ElementTree()
        tree.parse(xmlfile)
        spot = tree.find("idinfo/descript/purpose")
[/post]

MattTenold

I have your script working Curtis, but how do you write an if then statement for feature classes without a purpose(description) filled in?

Thanks

curtvprice

if arcpy.Exists(existing_xml):
    # do the conversion
    arcpy.ESRITranslator_conversion (existing_xml, translator, r'\\export\outputfile.xml') # this fails if the MD is pre ArcGIS10

The Python Way is to save yourself trouble by taking the approach of instead of asking for permission, asking for forgiveness later. It sounds messy, but the approach has an elegance that (I think) is really remarkable.

For example (note, totally untested code):

import os
import arcpy
from arcpy import conversion as CV
# get translator path
InstallDir = arcpy.GetInstallInfo("Desktop")["InstallDir"]
translator = os.path.join(InstallDir,r"\Metadata\Translator",
                 "ESRI_ISO2ISO19139.xml")

# input and output
docThing = r"C:\work\myfile.gdb\featureclass"
outXML = os.path.join(r"\\export",os.path.basename(docThing)) + ".xml"
try:
    CV.ESRITranslator(docThing, translator, outXML) 
except:
    # export failed
    try:
        # perhaps this metadata just needs upgrade?
        print "upgrading %s to 10..." % docThing
        CV.UpgradeMetadata(docThing,"ESRIISO_TO_ARCGIS")
    finally:
        # in any case, if upgrade fails or not, create or update 
        # metadata off the dataset properties
        CV.SynchronizeMetadata(docThing,"ALWAYS")
    try:
        # one more time try to export
        CV.ESRITranslator(docThing, translator, outXML)
    except:
        print "Metadata export failed for " + docThing

stevepeedell

Hi

I have written some python code that goes recursively through a filesystem, lists all datasets and either a) generates a stub metadata file if none exists (ISO19139), or b) converts the existing MD file to ISO19139. I then put all the 19139 metadata into GeoNetwork.

So far, so good. Only now I have encountered some old metadata that is in the pre ArcGIS10 format and needs to be upgraded. How can I test for this in code and perform the upgrade?

       existing_xml = os.path.join(dirname, dataset + '.xml') #this checks that a dataset has an xml file, not very elegant, as it could be any xml and not necessarily a metadata file, and I have no way of distinguishing between ESRI-ISO, FGDC and other schema
       if arcpy.Exists(existing_xml):
             # do the conversion
             arcpy.ESRITranslator_conversion (existing_xml, translator, r'\\export\outputfile.xml') # this fails if the MD is pre ArcGIS10

Thanks

Steve

KyleChouinard

For those who cant get metadata conversions to work outside of the python window in arcgis check this thread post #5, you may have to adjust your commands if you are running 64bit windows

http://forums.arcgis.com/threads/15433-Python-Errors-in-IDLE-but-Works-in-ArcCatalog

David_JAnderson

I banged my head against the 000816 error for a while until reading this thread. I could not get the exportmetadata to work in a Python IDLE window or even as a script file run from IDLE. I could get the command to work from the Python window in ArcCatalog and I could run the script so long as it was a script tool in a toolbox.
Definitely something odd about how the parameters are being handled internally versus externally.

JoshThompson

All,

I just wanted to address two small issues with this script I have encountered:

First, the script works great in the Python window in ArcMap/ArcCatalog, but produces errors in Pythonwin:
(ExecuteError: Failed to execute. Parameters are not valid.
ERROR 000816: The tool is not valid.
Failed to execute (ExportMetadata).

Second, if the name of your feature class has the same name as the feature dataset it resides in, it will cause the script to fail. (ex. - Data.gbd>Contours>Contours) Will also terminate if no text is found in the metadata component it is looking for.

I just wanted to let anyone who may use this script to be aware of these issues.

Josh

JoshThompson

Beautiful! That did it. Thanks Curtis for all your help and everyone else who contributed. Also, the indention to distinguish datasets from feature classes was a nice touch. Hadn't thought of that...

Josh

curtvprice

You were very close! I didn't handle any metadata on the feature dataset itself; you could add that.

import arcpy, sys
from xml.etree.ElementTree import ElementTree
from arcpy import env
env.overwriteOutput = True
env.workspace = r"D:\Lakeland_GIS\GDB\Lakeland\Lakeland.gdb\AddPoints"

AGSHOME = arcpy.GetInstallInfo("Desktop")["InstallDir"]
translatorpath = AGSHOME + r"Metadata\Translator\ARCGIS2FGDC.xml" 
xmlfile = r"U:\my docs\GIS Projects\Python\Scripts\List Metadata\working_test.xml" 
  
# list any standalone feature classes
fcList = arcpy.ListFeatureClasses()
fcList.sort()
for fc in fcList:
    arcpy.ExportMetadata_conversion(fc, translatorpath, xmlfile) 
    tree = ElementTree() 
    tree.parse(xmlfile) 
    spot = tree.find("idinfo/descript/purpose") 
    print fc
    print spot.text
        
# list feature datasets
datasetList = arcpy.ListDatasets()
datasetList.sort()
for dataset in datasetList:
    print dataset
    # list feature classes inside the dataset
    fcList = arcpy.ListFeatureClasses("","",dataset)
    fcList.sort()
    for fc in fcList:
        arcpy.ExportMetadata_conversion(fc, translatorpath, xmlfile) 
        tree = ElementTree() 
        tree.parse(xmlfile) 
        spot = tree.find("idinfo/descript/purpose") 
        print "  " + fc
        print "  " + spot.text

JoshThompson

Sorry, Curtis. I didn't realize the format doesn't carry over into the message. Lesson learned. Sending the code as an attachment.

Thanks,
Josh

ListMetadata.py.zip

curtvprice

You really need to put these things in code blocks so we can see your loops.

bbcode formatting: http://forums.arcgis.com/misc.php?do=bbcode

JoshThompson

All,
I brushed up on Python a little and attempted to tackle this script again. I have had some success in that I can now generate a list of feature classes and the "Summary" component of the metadata for all features in a feature dataset. The script below will produce that information.

import arcpy, sys
from xml.etree.ElementTree import ElementTree
from arcpy import env
env.overwriteOutput = True
env.workspace = r"D:\Lakeland_GIS\GDB\Lakeland\Lakeland.gdb\AddPoints"

translatorpath = r"C:\Program Files (x86)\ArcGIS\Desktop10.0\Metadata\Translator\ARCGIS2FGDC.xml"
xmlfile = r"U:\my docs\GIS Projects\Python\Scripts\List Metadata\working_test.xml"

datasetList = arcpy.ListDatasets()
datasetList.sort()
for dataset in datasetList:
    print dataset
fcList = arcpy.ListFeatureClasses()
fcList.sort()
for fc in fcList:
    arcpy.ExportMetadata_conversion(fc, translatorpath, xmlfile)
    tree = ElementTree()
    tree.parse(xmlfile)
    spot = tree.find("idinfo/descript/purpose")
    print fc
    print spot.text

What I am attempting to do is generate a list of all feature classes, by dataset, for an entire geodatabase. Ex. - (Dataset>Feature Class>Metadata). As of now I can only get FeatureClass>Metadata for one feature dataset at a time.

Help! If anyone could help me complete this I would be truly appreciative.

Thanks,
Josh

JoshThompson

Thanks Curtis, but I'm afraid this code is beyond my skill level. I think I have gotten in over my head with this and need some more practice time with Python before trying this again. Thanks again for your help.

Josh

curtvprice

The following attached script works as a script tool. I get the error when I try to run it from IDLE or the command line... apparently there is something in the environment that is messing me up. I'm running 10.0 SP 2 on XP 64.

import sys, os, arcpy

def getMetadataText(Dataset,Element="dataIdInfo/idPurp"):
    """Extracts metadata contents from ArcGIS metadata
  arguments
    Dataset - Path to an ArcGIS dataset
    Element - XML path in ArcGIS metadata. 
    Default: "metadata/dataIDInfo/idPurp" ("Summary")
"""
    from xml.etree.ElementTree import ElementTree
    try:
        # Get arcgis to fgdc translator path
        ARCGISHOME = arcpy.GetInstallInfo("desktop")["InstallDir"]
        XSLT = os.path.normpath(os.path.join(
            ARCGISHOME,
            "Metadata/Stylesheets/gpTools/exact copy of.xslt"))
        xmlTemp =  os.path.join(os.environ["TEMP"],"xyz_xmltemp.xml")
        # export the metadata to xml   
        arcpy.env.overwriteOutput = True       
        arcpy.XSLTransform_conversion(Dataset,XSLT,xmlTemp)
        # parse out Description element from ArcGIS 10 metadata
        tree = ElementTree() # make an ElementTree object
        tree.parse(xmlTemp) # read the xml into the ElementTree
        Data = tree.find(Element) # find whatever tag you want
        del tree
        arcpy.Delete_management(xmlTemp)
        return Data.text # return the contents as text
    except Exception, xmsg:
        raise Exception, str(xmsg)

# You could put a loop here around this to print out Summary for a list of datasets
dataset = r"E:\work\work.gdb\BeetleSprayArea.shp"
text = getMetadataText(dataset)
arcpy.AddMessage(text)

JoshThompson

Darren,
When I attempted to use the code you supplied, it simply moves to a new prompt. No errors, no output, nothing. Just a new prompt.

Curtis,
I am getting the same error.

DarrenWiens2

I tried today to use the XSLTransform_conversion tool to just extract out everything (avoiding a transform) but I am getting an error message that appears to be a bug in tool validation (which I sent to ESRI support).

I believe this was fixed in SP1. I can't remember the exact error I was getting, but it's gone now.

curtvprice

I tried today to use the XSLTransform_conversion tool to just extract out everything (avoiding a transform) but I am getting an error message that appears to be a bug in tool validation (which I sent to ESRI support). I've attached this python script that is a function to pull out the ArcGIS metadata elements more directly (no need for the FGDC transform).

The Arc 10 metadata schema is documented in the file:

\ArcGIS\Desktop10.0\Metadata\Translator\Rules\ArcGIS metadata v1.dtd

though I have found it easier to just look in an existing ".shp.xml" file.

Another approach to this problem is to create an xslt stylesheet and run the XSL Transformation tool (conversion toolbox) to pull out the information you want into an XML file.

GetMetadataDescription.py.txt.zip

DarrenWiens2

Rather than call one feature class (fcpath), you'll call each feature class, one by one, from the fcList. This is totally untested, so you might run into errors (tricky areas might include overwriting the xml file, or reusing the ElementTree). You could also dynamically name the xml (e.g. with a similar name to the feature class name) inside the loop if you wanted separate xml files, rather than just one overwritten each time.

import arcpy, sys
from xml.etree.ElementTree import ElementTree
from arcpy import env
env.overwriteOutput = True
env.workspace = r"D:\GIS\GDB\Lakeland\Lakeland.gdb\AddPoints" # set the workspace for use in  ListFeatureClasses
fcList = arcpy.ListFeatureClasses() # lists all feature classes from inside env.workspace

translatorpath = r"C:\Program Files (x86)\ArcGIS\Desktop10.0\Metadata\Translator\ARCGIS2FGDC.xml" # the translator file to use
xmlfile = r"D:\GIS\metadata\test.xml" # the output xml file

for fc in fcList:
    arcpy.ExportMetadata_conversion(fc, translatorpath, xmlfile) # export the current fc's metadata to xml
    tree = ElementTree() # make an ElementTree object
    tree.parse(xmlfile) # read the xml into the ElementTree
    spot = tree.find("idinfo/descript/abstract") # find whatever tag you want
    print spot.text # print the text between the tags

JoshThompson

Here is the script I used (per dkwiens) to extract and print the "Description" portion of the metadata for an individual feature class:

import arcpy, sys
from xml.etree.ElementTree import ElementTree
from xml.etree.ElementTree import Element, SubElement
from arcpy import env
env.overwriteOutput = True
fcpath = r"D:\GIS\GDB\Lakeland\Lakeland.gdb\AddPoints\AddressPts" # the input feature class
translatorpath = r"C:\Program Files (x86)\ArcGIS\Desktop10.0\Metadata\Translator\ARCGIS2FGDC.xml" # the translator file to use - it should be in an installed folder like this
xmlfile = r"D:\GIS\metadata\test.xml" # the output xml file
arcpy.ExportMetadata_conversion(fcpath, translatorpath, xmlfile) # export the metadata to xml

tree = ElementTree() # make an ElementTree object
tree.parse(xmlfile) # read the xml into the ElementTree

spot = tree.find("idinfo/descript/abstract") # find whatever tag you want
print spot.text # print the text between the tags

If you can help me tie this into the fcList script, I will be indebted to you.

Josh

curtvprice

I am able to list the "Description" portion of the metadata for an individual feature class.

Cool! Please post your working script either in full or as an attachment. Wrapping this in a loop should be pretty straighforward.

JoshThompson

Hey Guys,

Thanks for the help. So far, all the info you have given me has worked very well. I am able to list the "Description" portion of the metadata for an individual feature class.

Is there a way to integrate this code with the "Listfc" script? My ultimate goal is to list all feature classes, by dataset, and their respective metadata from a file GDB to give to staff in various dept. I am a beginner and am just not able to make this work on my own. I appreciate any help you can give.

Josh

curtvprice

It works fine as is, I suppose because there were no characters to be escaped...?

Just don't ever name any folders or files starting with n, r, or t, etc....:

>>> "a\z" # "\z" is not a special string literal, so OK
'a\\z'
>>> "a\t"  # this is "a" followed by <tab>
'a\t'
>>>

See:
http://docs.python.org/reference/lexical_analysis.html#string-literals

DarrenWiens2

Oh, I can never remember which is which. It works fine as is, I suppose because there were no characters to be escaped...? In any case, you should probably fill in your own paths, as I doubt you have access to my X:/.

Participe da Conversa

Membros conectados podem postar, seguir atualizações e mais. Novo aqui? Registre uma conta gratuita.

Entrar
Registrar

Links Rápidos

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.