replaceDataSource failing when iterating through multiple MXDs

Anonymous User — Fri, 10 Dec 2021 23:29:05 GMT

UPDATE: This woks fine for updating the source to Shapefiles but not for SDE feature classes!

I am trying to use replaceDataSource by iterating through multiple map documents. My script updates the first map document but always fails on the second. So if I have map1.mxd, map2.mxd and map3.mxd it updates dataSources in map1 and fails on map2. Even if I put map2 through first (map1 2nd and map3 3rd) it will still fail on the 2nd iteration, so map1 fails in this instance.

I am updating the source to SDE. The input csv is the output from another script that gets unique broken datasources. I then add a second column that matches the broken datasource to a new one

import arcpy, csv, os
import arcpy.mapping as m

def updateDataSource(datapath, broken_lyr):
    if datapath.endswith(".shp"):
        workspace_path = datapath.rsplit("\\", 1)[0]
        workspace_type = "SHAPEFILE_WORKSPACE"
        dataset_name = datapath.rsplit("\\", 1)[-1][0:-4]
    elif datapath.find(".sde") != -1:
        workspace_path = datapath.rsplit(".sde", 1)[0] + ".sde"
        workspace_type = "SDE_WORKSPACE"
        dataset_name = datapath.rsplit("\\")[-1]
    elif datapath.find(".gdb") != -1:
        workspace_path = datapath.rsplit(".gdb", 1)[0] + ".gdb"
        workspace_type = "FILEGDB_WORKSPACE"
        dataset_name = datapath.rsplit(".gdb\\")[-1]
    print "\n"
    print workspace_path
    print workspace_type
    print dataset_name
    print "\n"
    if workspace_path:
        broken_lyr.replaceDataSource(workspace_path, workspace_type, dataset_name, True)

    else:
        print "Failed for {0} \n workspace is not FILEGDB, SDE, or SHAPEFILE".format(broken_lyr.name)

input_csv = r"C:\Users\glen.bambrick\Documents\csv\broken_links_20180523.csv"
input_folder = r"C:\Users\glen.bambrick\Documents\mxd"

broken_dict = {}
broken_mxd_list = []

with open(input_csv, 'rb') as read_csv:
    reader = csv.reader(read_csv)
    next(reader, None)
    for row in reader:
        if row:
            broken_dict[row[0]] = row[1]
            broken_mxd_list.append(row[3])

mxd_set = set([])
for value in broken_mxd_list:
    for item in value.split(", "):
        mxd_set.add("{0}\\{1}".format(input_folder, item))

for key, value in broken_dict.iteritems():
    print key, value

print "\n"
print mxd_set

for mapdoc in mxd_set:
    print "\n"
    print mapdoc
    mxd = m.MapDocument(mapdoc)
    list_broken = m.ListBrokenDataSources(mxd)
    if len(list_broken) == 0:
        continue
    else:
        for broken in list_broken:
            if broken.supports("DATASOURCE") and broken_dict[broken.dataSource]:
                print "Updating: {0}".format(broken.dataSource)
                print "\tTo: {0}".format(broken_dict[broken.dataSource])
                updateDataSource(broken_dict[broken.dataSource], broken)

    mxd.save()
    del mxd
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

The output and the error. As you can see the same parameters are going in but it always fails on the first replaceDataSource() of the second iteration no matter what map document it is. If there is one mxd in the list it will do it just fine, but I want to loop through hundreds eventually.

C:\Users\user1\AppData\Roaming\ESRI\Desktop10.4\ArcCatalog\N6.sde\JN233985_GCOB.DBO.GCOB_03391_Beneficial_Deposition_1 Database Connections\Connection to GALSQL01.sde\JN233985_GCOB.DBO.GCOB_03_Environmental\JN233985_GCOB.DBO.GCOB_03391_Beneficial_Deposition
C:\Users\user2\AppData\Roaming\ESRI\Desktop10.4\ArcCatalog\Connection to GALSQL01.sde\JN233985_GCOB.DBO.GCOB_03391_Beneficial_Deposition_1 Database Connections\Connection to GALSQL01.sde\JN233985_GCOB.DBO.GCOB_03_Environmental\JN233985_GCOB.DBO.GCOB_03391_Beneficial_Deposition


set(['C:\\Users\\glen.bambrick\\Documents\\mxd\\map2.mxd', 'C:\\Users\\glen.bambrick\\Documents\\mxd\\map1.mxd'])


C:\Users\glen.bambrick\Documents\mxd\map2.mxd
Updating: C:\Users\user1\AppData\Roaming\ESRI\Desktop10.4\ArcCatalog\N6.sde\JN233985_GCOB.DBO.GCOB_03391_Beneficial_Deposition_1
     To: Database Connections\Connection to GALSQL01.sde\JN233985_GCOB.DBO.GCOB_03_Environmental\JN233985_GCOB.DBO.GCOB_03391_Beneficial_Deposition


Database Connections\Connection to GALSQL01.sde
SDE_WORKSPACE
JN233985_GCOB.DBO.GCOB_03391_Beneficial_Deposition


Updating: C:\Users\user2\AppData\Roaming\ESRI\Desktop10.4\ArcCatalog\Connection to GALSQL01.sde\JN233985_GCOB.DBO.GCOB_03391_Beneficial_Deposition_1
     To: Database Connections\Connection to GALSQL01.sde\JN233985_GCOB.DBO.GCOB_03_Environmental\JN233985_GCOB.DBO.GCOB_03391_Beneficial_Deposition


Database Connections\Connection to GALSQL01.sde
SDE_WORKSPACE
JN233985_GCOB.DBO.GCOB_03391_Beneficial_Deposition


C:\Users\glen.bambrick\Documents\mxd\map1.mxd
Updating: C:\Users\user1\AppData\Roaming\ESRI\Desktop10.4\ArcCatalog\N6.sde\JN233985_GCOB.DBO.GCOB_03391_Beneficial_Deposition_1
     To: Database Connections\Connection to GALSQL01.sde\JN233985_GCOB.DBO.GCOB_03_Environmental\JN233985_GCOB.DBO.GCOB_03391_Beneficial_Deposition


Database Connections\Connection to GALSQL01.sde
SDE_WORKSPACE
JN233985_GCOB.DBO.GCOB_03391_Beneficial_Deposition



Traceback (most recent call last):
  File "C:\Users\glen.bambrick\Desktop\test.py", line 65, in <module>
    updateDataSource(broken_dict[broken.dataSource], broken)
  File "C:\Users\glen.bambrick\Desktop\test.py", line 23, in updateDataSource
    broken_lyr.replaceDataSource(workspace_path, workspace_type, dataset_name, True)
  File "C:\Program Files (x86)\ArcGIS\Desktop10.5\ArcPy\arcpy\utils.py", line 182, in fn_
    return fn(*args, **kw)
  File "C:\Program Files (x86)\ArcGIS\Desktop10.5\ArcPy\arcpy\_mapping.py", line 682, in replaceDataSource
    return convertArcObjectToPythonObject(self._arc_object.replaceDataSource(*gp_fixargs((workspace_path, workspace_type, dataset_name, validate), True)))
ValueError: Layer: Unexpected error‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: replaceDataSource failing when iterating through multiple MXDs

DanPatterson_Retired — Fri, 25 May 2018 13:43:48 GMT

Glenn Code Formatting .... would help and throw in a print statement before the line 45 since what going in to the first process doesn't seem like it was expected

Re: replaceDataSource failing when iterating through multiple MXDs

Anonymous User — Fri, 25 May 2018 14:49:45 GMT

Thanks for looking Dan, I was looking for the code formatting and in my haste missed it. I have updated the code and the output/error. Cheers

Re: replaceDataSource failing when iterating through multiple MXDs

DanPatterson_Retired — Fri, 25 May 2018 16:42:46 GMT

Glen with all the nesting it isn't clear to me if the mxd.save() is the right place.

I would suggest collecting all the mxds with broken links first, then process them separately, rather than nesting, then performing another check and finally doing something.

Re: replaceDataSource failing when iterating through multiple MXDs

forestknutsen1 — Fri, 25 May 2018 17:38:57 GMT

 broken_lyr.replaceDataSource(workspace_path, workspace_type, dataset_name, True

You could try to set this validation to false. I have had some weird behavior around this and sde datasouces in the past.

Re: replaceDataSource failing when iterating through multiple MXDs

Anonymous User — Mon, 28 May 2018 10:11:22 GMT

I tried this and while it allows the looping to continue without failing and saving the mxd it does not update the sources after the first mxd

Re: replaceDataSource failing when iterating through multiple MXDs

Anonymous User — Mon, 28 May 2018 13:38:08 GMT

The mxd.save() is in the correct place, the mxds with broken links are already collected and used from the input_csv, within the nesting it is getting the broken layers and updating using the function updateDataSource(). The code works perfect for Shapefiles so SDE is the issue. I just need to find a workaround or is this a possible bug?

Re: replaceDataSource failing when iterating through multiple MXDs

ThibautDelaere1 — Mon, 15 Apr 2019 10:03:01 GMT

UPDATE (WORKAROUND FOUND😞 adding arcpy.ClearWorkspaceCache_management() before every replaceDatasource seems to solve the issue.

I'm having the same issue right now, so I was wondering if you ever solved it?

I run a Python Toolbox and noticed the error when rerunning the tool replacing datasources. So first run everything works fine, second run (and further) it fails on a certain SDE-connection with the ValueError mentioned.

Closing ArcCatalog and reopening makes the tool run again succesfully. But just once. Then the same error appears.

So, it seems related to certain SDE-connections and something in memory/cached/… of the Python environment.

Re: replaceDataSource failing when iterating through multiple MXDs

JosephJose — Tue, 16 Apr 2019 01:07:13 GMT

Anyone found a solution for this? I am facing the same problem

Re: replaceDataSource failing when iterating through multiple MXDs

JosephJose — Tue, 16 Apr 2019 01:10:46 GMT

Yes you were correct, if it's open in Catalog or ArcMap, it creates the error

topic Re: replaceDataSource failing when iterating through multiple MXDs in ArcMap Questions

replaceDataSource failing when iterating through multiple MXDs

Re: replaceDataSource failing when iterating through multiple MXDs

Re: replaceDataSource failing when iterating through multiple MXDs

Re: replaceDataSource failing when iterating through multiple MXDs

Re: replaceDataSource failing when iterating through multiple MXDs

Re: replaceDataSource failing when iterating through multiple MXDs

Re: replaceDataSource failing when iterating through multiple MXDs

Re: replaceDataSource failing when iterating through multiple MXDs

Re: replaceDataSource failing when iterating through multiple MXDs

Re: replaceDataSource failing when iterating through multiple MXDs