Select to view content in your preferred language

List Broken Data Source's Path

10771
46
08-07-2013 09:49 AM
LauraMiles1
Frequent Contributor
Hi everyone, I'm a very amateur Python user. I've managed to get the following, which I pieced together from here and there, working for the most part. I need to create a csv file which has a column showing the path to the data source which is broken. Everything works fine and dandy if I take out the dataSource part; when I add it in, it will run for a while and then fail. It seems to be getting tripped up on some mxd or data source it doesn't like perhaps? I have no clue. Here's my code:

import arcpy, os
path = r"H:\Plans\GIS Plans\2003"
f = open('BrokenMXD2003.csv', 'w')
f.write("Type, File Path, Layer, Broken Path" + "\n")
for root, dirs, files in os.walk(path):
    for fileName in files:
        basename, extension = os.path.splitext(fileName)
        if extension == ".mxd":
            fullPath = os.path.join(root, fileName)
            mxd = arcpy.mapping.MapDocument(fullPath)
            brknMXD = arcpy.mapping.ListBrokenDataSources(mxd)
            for brknItem in brknMXD:
                lyrList = arcpy.mapping.ListLayers(mxd)
                f.write("MXD, " + fullPath + ", " + brknItem.name)
                if brknItem.supports("dataSource"):
                    f.write(", " + brknItem.dataSource + "\n")
                else:
                    f.write("\n")

f.close()

print "Script Completed"


And here's the error I get:

Traceback (most recent call last):
  File "X:\Documents\Working Files\Broken Data Sources\ListBrokenMXD.py", line 11, in <module>
    brknMXD = arcpy.mapping.ListBrokenDataSources(mxd)
  File "D:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\utils.py", line 181, in fn_
    return fn(*args, **kw)
  File "D:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\mapping.py", line 1465, in ListBrokenDataSources
    result = mixins.MapDocumentMixin(map_document_or_layer).listBrokenDataSources()
  File "D:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\arcobjects\mixins.py", line 832, in listBrokenDataSources
    broken_sources = [l for l in self.layers if not l._arc_object.valid]
  File "D:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\arcobjects\mixins.py", line 683, in layers
    for frame in reversed(self.dataFrames):
  File "D:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\arcobjects\mixins.py", line 695, in dataFrames
    return map(convertArcObjectToPythonObject, self.pageLayout.dataFrames)
AttributeError: 'NoneType' object has no attribute 'dataFrames'

It does run and create several lines of output, but this error appears when it gets near to the end of the mxd's in the directory. I haven't a clue what the problem could be, as I said I'm quite amateur. If anyone can see what it is, I'd be greatly appreciative. Thank you!
Tags (2)
0 Kudos
46 Replies
lelaharrington
Regular Contributor

This works great. the only question i have and since I am new at this is have you found a way to get this script to walk thru all sub folders and look or will i have to go thru each folder in the directory with this script? 

Thank you

Lela Harrington

0 Kudos
LauraMiles1
Frequent Contributor

Hi Lela, if I recall it does go through all the subfolders. So just change the path where indicated in the script and it will go through all folders/subfolders etc.

0 Kudos
lelaharrington
Regular Contributor

I tried to just do like H: and to pool thru sub folders and it sadly won't talk thru them all

Lela Harrington

0 Kudos
LauraMiles1
Frequent Contributor

What happened, did it freeze? I just tried running it on one of our main folders and it worked. It stopped when it hit a table view data source, but it was running through subfolders...

0 Kudos
lelaharrington
Regular Contributor

yup i got the table view error that is what happened 

but it doesn't tell me where the table was 

"Traceback (most recent call last):

  File "H:/PythonTesting/OS_Walk_Testing/List_Broken_Data_Sourse_Path.py", line 18, in

    if brknItem.supports("dataSource"):

AttributeError: 'TableView' object has no attribute 'supports'"

Thank you

Lela Harrington

0 Kudos
LauraMiles1
Frequent Contributor

Yes now that I recall, and looking through the rest of the thread, I sort of stopped working on the script without solving this error. The test for the table view actually causes the script to crash when it does encounter a table view (or so the error suggests).

I tried modifying things so that it only writes the dataSource if the file extension is not .dbf, but it still crashes with the same tableview error. I inspected the .mxd it crashed halfway through and there aren't actually any tableviews in it. So there is something else within the .mxd that's causing this to happen. Testing for the .dbf extension is not weeding out the problem...I ran the .mxd through MXD Doctor and there seems to be nothing wrong with it. I'm stumped.

0 Kudos
BlakeTerhune
MVP Regular Contributor

I'm running into this issue to (started with AttributeError: 'NoneType' object has no attribute 'dataFrames'). From my testing, it looks like it has to do with older MXD's (mine was from around 2010) that have something that's somehow not compatible with the arcpy stuff. I tried Save As and Save a Copy and the problem persisted. Copying and pasting the dataframe doesn't work either. However, if I create a new dataframe in the MXD and copy the layers into it and delete the old data frame, it works fine; so it can't be something with the MXD in general.

I think the solution is to print the MXD path and try to manually fix the dataframe(s) in there before continuing. If you have a lot of MXD's and touching each one isn't reasonable, the next best solution is probably what Stacy Rendall​ mentioned about using a try, except block to catch the AttributeError exception and skip that MXD if there's an error. Not the cleanest, but it works.

try:
     # Do stuff with MXD using arcpy.mapping
except AttributeError:
     print "This MXD has unreadable dataframes\n{}".format(mxd.filePath)
     pass
0 Kudos
RebeccaStrauch__GISP
MVP Emeritus

I was getting this error "AttributeError: 'NoneType' object has no attribute 'dataFrames'" today in a script that I was running against all STARTED services in AGS (10.2.2).  I'm looping thru all those services, finding the MXDs and searching to see if the FGDB I'm needing to update is in the MXD, then creating a list of the services I need to STOP so I could replace the FGDB and then restart.  This error was showing up on only one of my services (and therefore crashing).  It turns out the for some reason, that mxd must be saved as 10.3.x, or so the error seems to indicate if I try to open with ArcMap 10.2.2. 

The error occurred  with "lyrList = arcpy.mapping.ListLayers(mxd)"  so I wrapped in in a ListDataFrames loop, with no luck.  Since there is no easy or reliable way to check for the version of an .mxd (from several hours of searching and testing a half dozen suggestions), I found the try: my best option, and I'll just include this service in my STOP list, just in case.

Thought I would give this thumbs up on the try: solution for anyone that might see this....we do what we need to get things done sometimes  Snippet of my code, still under development for anyone interested. (will not run on its own)

# ....
def myMsgs(message):
    arcpy.AddMessage(message)
    print(message)
    
updateServices = []
for startedService in myServices:
    mxdFound = False
    update = False
    #myMsgs("Reviewing service: {0}".format(startedService))
    theServicePath = os.path.join(servicesDir, startedService)
    if not arcpy.Exists(theServicePath):
        myMsgs("...we have a problem")
        exit
    else:
        myMsgs("Reviewing service: {0}".format(startedService))
        for root, dirs, files in os.walk(theServicePath):
            for fn in files:
                fullPath = os.path.join(root, fn)
                basename, extension = os.path.splitext(fn)
                if not extension == ".mxd":
                    pass
                else:
                    #myMsgs("  mxd file found")
                    mxd = arcpy.mapping.MapDocument(fullPath)
                    try:  
                        # Do stuff with MXD using arcpy.mapping  
                        for df in arcpy.mapping.ListDataFrames(mxd):
                            lyrList = arcpy.mapping.ListLayers(mxd)
                            mxdFound = True
                            for lyr in lyrList:
                                if lyr.isFeatureLayer and update == False:
                                    if lyr.supports("dataSource") and update == False:
                                        theSource = lyr.dataSource
                                        if updateDS[0] in theSource:
                                            update = True
                                            myMsgs("   update {0}".format(startedService))    
                                            updateServices.append(startedService)
                    
                    except AttributeError:  
                        myMsgs("!!!!  WARNING: This MXD has unreadable dataframes\n{}  !!!!".format(mxd.filePath))
                        myMsgs("    -> will stop as a precaution....")
                        updateServices.append(startedService)   # edit....forgot this on original post
                        pass                          
                    del mxd

for a in updateServices:
    myMsgs("will stop {}".format(a))
# ....
RichardFairhurst
MVP Honored Contributor
Python Beginner - Using version 10.1 and not an SDE install I have an atribute table with firehydrant inspection data. I am trying to create a layer that will only show the records grouped by fire hydrant number and then only display the most recent inspection date. When I execute the following code it runs fine and does in fact create the layer but it doesn't select the Max Date. If I have a record for Hydrant 100 with two dates I only want the most recent date to be created within the new layer.
Appreciate any assistance on this.

import arcpy
from arcpy import env
env.workspace=r"Z:\GIS_Data\gdb\GIS_Test.gdb"

fc="COA_FH_Inspections"
cursor = arcpy.UpdateCursor(fc)
idlist = []
datelist = []

for row in cursor:
    idlist.append(row.getValue('HydrID'))
    datelist.append(row.getValue('TestDate'))
    cursor.updateRow(row)

print idlist
print datelist

idunique = set(idlist)

#Make a layer from the feature class
arcpy.MakeFeatureLayer_management(fc, "lyr")

for i in idunique:
    if i == idlist:
        arcpy.SelectLayerByAttribute_management("lyr", "New_Selection", '"HydrID" = i')
        arcpy.SelectLayerByAttribute_management("lyr", "subset_Selection",'"TestDate" = (Select Max("TestDate") from "lyr"')


Again thank you - Terry


SDE and personal geodatabases supposedly support subqueries, but they are not well documented and not easy to troubleshoot.  The Subquery has to return a single result and be in the format:

RelateField2 IN (SELECT RelateField1 FROM Table1 WHERE SQLWhereClause1)

I also do not think a subquery for a Max value can come from the same table that you are making a selection on (at least I know they fail for an fgdb, but those are much more limited than SDE subqueries, so that may not be a restriction for your SDE database)

To do this I use the Summary Statistics.  Unfortunately dates do not allow you to perform summary functions in any ESRI tools.  The work around steps I have had to use are listed here.  The advantage of the Summary Statistics approach over the approach you are attempting is that it can summarize thousands of max dates for thousands of unique IDs in under 2 minutes and do the selection of every Max date for every unique ID in the entire tables in one selection operation.  A Feature Class to Feature Class output of that selection can make a 1:1 joinable table for other summary or data transfer operations.  You can make the summary outputs to in memory tables if you don't want them written to disk.

Processing each ID with individual queries can take up to 100 times longer to process, since that causes a new disk read from the start of the table for each new query.
0 Kudos
TerryHiggins1
Occasional Contributor
Perhaps I did not express myself clearly - I do not have SDE. I am dealing with a file gdb. When I export the table to Access then I can easily extract what I want but within Arc and python I am totally lost.
0 Kudos