topic Re: Arcpy: replaceDataSource (from gdb to SDE w/ change in data source name) in Python Questions

Arcpy: replaceDataSource (from gdb to SDE w/ change in data source name)

CrystalSchiffbauer-Bowles1 — Fri, 31 Jul 2015 22:13:43 GMT

Hello all. We have a large task of replacing datasources for MANY feature classes and rasters (we received data from a consultant, and in order to place the data on the Client's enterprise server, all data has to conform to schema requirements). I have saved all of the data in their respective receptacles on the enterprise SDE environment. Now, I need to change the datasources within each of the mxds - this means every feature class in every mxd has to be relinked to the new name and location on the SDE. I want to do this with Python - naturally, otherwise task is beyond tedious.

So, I have my list of broken links, I have a table with name and location crosswalk for reference. I am not at all a Python guru and could really use some help in figuring out what the error below means (I feel like it has something to do with the enterprise SDE connection properties, but honestly don't know, especially since it is referencing my storage properties):

I have my broken list already, so am referencing that:

>>> for lyr in brokenlist:

... if lyr.supports ("DATASOURCE"):

... if lyr.dataSource == "Y:\00726_11_BDCP\data\DHCCP_ENG_Rev10b\DHCCP_Eng_Rev10b.gdb\Option_Point_Features":

... lyr.replaceDataSource ("Y:\00726_11_BDCP\data\DHCCP_ENG_Rev10b\DHCCP_Eng_Rev10b.gdb\Option_Point_Features", "SDE_WORKSPACE", "C:\Users\cbowles\AppData\Roaming\ESRI\Desktop10.2\ArcCatalog\dhccp_eng.SDE.ER10b_Option_Point_Features")

... mapdoc.save ()

... del mapdoc

Runtime error

Traceback (most recent call last):

File "<string>", line 5, in <module>

File "c:\program files (x86)\arcgis\desktop10.2\arcpy\arcpy\utils.py", line 181, in fn_

return fn(*args, **kw)

File "c:\program files (x86)\arcgis\desktop10.2\arcpy\arcpy\_mapping.py", line 801, in save

return convertArcObjectToPythonObject(self._arc_object.save(*gp_fixargs((), True)))

IOError: MapDocObject: Unable to save. Check to make sure you have write access to the specified file and that there is enough space on the storage device to hold your document.

I appreciate all helpful comments.

Thanks,

Crystal

Re: Arcpy: replaceDataSource (from gdb to SDE w/ change in data source name)

IanMurray — Sat, 11 Dec 2021 06:43:57 GMT

Do you have a list of broken links for all map documents, or just for one? You really need to handle the broken links on a map by map basis, since each one will have its own list of layers to be fixed.

Here is a script I used, walks a workspace makes a list of maps, for each map lists layers and fixes the path designated, you should be able to modify it for replaceDataSource instead of findAndReplaceWorkspacePath

#Importing Modules  
import arcpy, os   
from arcpy import env
#Put file path for the folder containing mxds, where I have my sample file path  
env.workspace = r"S:\TS Projects\Basemaps"
#Workspaces I want replaced
find_workspace_path = r"C:\Users\iamurray\Desktop\Export_Output"
find_workspace_path2 = r"C:\Users\iamurray\Desktop\ED_Remainder"
find_workspace_path3 = r"C:\Users\iamurray\Desktop\ED_Remainder2"
find_workspace_path4 = r"C:\Users\iamurray\Desktop\Export_Output2"
replace_workspace_path  = r"C:\Test\NewWorkspace" 
#Recursively walking workspace
for root, dirs, files in os.walk(env.workspace):
        for f in files:
                if f.endswith(".mxd"):
                        mxd = arcpy.mapping.MapDocument(os.path.join(root , f))
                        Layers = arcpy.mapping.ListLayers(mxd) 
                        for Layer in Layers:
                                if Layer.dataSource ==  find_workspace_path:
                                        Layer.findAndReplaceWorkspacePath (find_workspace_path, replace_workspace_path)
                                elif Layer.dataSource ==  find_workspace_path2:
                                        Layer.findAndReplaceWorkspacePath (find_workspace_path2, replace_workspace_path)
                                elif Layer.dataSource ==  find_workspace_path3:
                                        Layer.findAndReplaceWorkspacePath (find_workspace_path3, replace_workspace_path)
                                elif Layer.dataSource ==  find_workspace_path4:
                                        Layer.findAndReplaceWorkspacePath (find_workspace_path4, replace_workspace_path)
                        mxd.save()
                        del mxd

Re: Arcpy: replaceDataSource (from gdb to SDE w/ change in data source name)

CrystalSchiffbauer-Bowles1 — Fri, 31 Jul 2015 23:09:56 GMT

Thank you, Ian. I am currently working on a map-to-map basis, since yes, as you mentioned, each one contains different layers.

I will certainly try yours. I had seen a similar post to yours on another site, but this one certainly seems easier to understand. Thank you for your help. Will let you know how it works.

Crystal.

Re: Arcpy: replaceDataSource (from gdb to SDE w/ change in data source name)

MichaelVolz — Mon, 03 Aug 2015 13:52:47 GMT

If your layers have broken links then how does the code

if lyr.dataSource == "Y:\00726_11_BDCP\data\DHCCP_ENG_Rev10b\DHCCP_Eng_Rev10b.gdb\Option_Point_Features":

work, as I would expect lyr.dataSource to be NULL

Re: Arcpy: replaceDataSource (from gdb to SDE w/ change in data source name)

RebeccaStrauch__GISP — Mon, 03 Aug 2015 18:15:50 GMT

Typically the dataSource will not be NULL, but will be a location that it can no longer find and therefore will be broken.

arcpy.mapping.ListBrokenDataSources(mxd)

can give you a list of brokenlinks within an MXD. I'm working on a toolbox (python addin) to help with this process since I have many that need to be updated, but it's a secondary project so I haven't finished polishing it yet. I'm building in the ability to list and fix most if not all the data types since I have some very old mxd's that still point to coverages or did point to coverages and now need to point to FGDB, etc.

Anyway, try the ListBrokenDatatSources command to see what it thinks the data sources are.

Re: Arcpy: replaceDataSource (from gdb to SDE w/ change in data source name)

CrystalSchiffbauer-Bowles1 — Wed, 05 Aug 2015 20:31:09 GMT

Hello Ian. So, I have been troubleshooting the reason for my now WAY simplified code (I've gone to trying to get just 1 feature class w/in 1 mxd) runs, but the data source does not get updated. This is the code (I have gone to just trying to get the data source to change between geodatabases because I was getting no where with SDE):

mapdoc = arcpy.mapping.MapDocument (r"Y:\HCP\_Template_HCP_Portrait_11x17_20120710.mxd")

lyr in arcpy.mapping.ListBrokenDataSources (mapdoc):

... if lyr.supports ("DATASOURCE"):

... if lyr.dataSource == r"Y:\00726_11_BDCP\data\BDCP_BaseMap2.gdb\ICF_Cities":

... lyr.replaceDataSource (r"T:\WGI-38\GIS_33_00\GISRequest_11F\Deliverable_10F\MGT096_13\GISFiles\BDCP_EIRS_HCP_15.gdb", "FILEGDB_WORKSPACE", "i13_BDCP_Cities_09")

... mapdoc.saveACopy (r"Y:\HCP\_Template_HCP_Portrait_11x17_20120710s.mxd")

As I said, the code runs, no errors, but none of the data is updated in the new or current mxd. Any ideas why this might be happening? I appreciate everyone's help.

Re: Arcpy: replaceDataSource (from gdb to SDE w/ change in data source name)

MichaelVolz — Thu, 06 Aug 2015 11:51:19 GMT

Crystal:

Are you saving a new mxd whether or not any datasources actually get updated? I ask because I made sure that at least one (1) datasource was updated before I executed the SaveACopy command in a script I wrote that also replaced datasources. I would add a counter to make sure you were actually updating a datasource before saving a new file. Your if conditions might not be working and you are saving a new mxd even though no changes are being made to the mxd.

I would add some print statements to test whether your if conditions were actually being met. I would try to verify that the paths to the files were being interpreted correctly as that always seem to be a troublesome area with python coding.

Re: Arcpy: replaceDataSource (from gdb to SDE w/ change in data source name)

RandyKreuziger — Thu, 06 Aug 2015 18:20:11 GMT

I don't think the replace datasource works when going from a GDB to SDE. You can do replacements from PGDB to FGDB, FGDB to FGDB, FGDB to PGDB and PGDB to PGDB when the feature class has the same name for example "county." But at least in SQL Server the name in SDE is seen as "databasename.schema.county." In our case GeoLib.DBO.County so the repair broken links in MXDs doesn't work.

Re: Arcpy: replaceDataSource (from gdb to SDE w/ change in data source name)

MichaelVolz — Thu, 06 Aug 2015 18:29:28 GMT

I would think you could put conditional statements in so if a feature class has a name such as Roads it would go into the Transportation schema in SDE so the replaceDataSource line which currently is:

lyr.replaceDataSource ("Y:\00726_11_BDCP\data\DHCCP_ENG_Rev10b\DHCCP_Eng_Rev10b.gdb\Option_Point_Features", "SDE_WORKSPACE", "C:\Users\cbowles\AppData\Roaming\ESRI\Desktop10.2

could be

lyr.replaceDataSource ("SDE connection string\"Schema from conditional statement".Option_Point_Features", "SDE_WORKSPACE", "C:\Users\cbowles\AppData\Roaming\ESRI\Desktop10.2

so for Roads it would be

lyr.replaceDataSource ("SDE connection string\Transportation.Roads", "SDE_WORKSPACE", "C:\Users\cbowles\AppData\Roaming\ESRI\Desktop10.2

Re: Arcpy: replaceDataSource (from gdb to SDE w/ change in data source name)

IanMurray — Thu, 06 Aug 2015 18:37:57 GMT

According to the help, you can switch workspace types with replaceDataSource. The third parameter is the dataset name to be replaced.

"A string that represents the name of the dataset the way it appears in the new workspace (not the name of the layer in the TOC). If dataset_name is not provided, the replaceDataSource method will attempt to replace the dataset by finding a table with a the same name as the layer's current dataset property."

So yes you would need to provide the script with the databasename and schema for replaceDataSource to work properly.

Re: Arcpy: replaceDataSource (from gdb to SDE w/ change in data source name)

CrystalSchiffbauer-Bowles1 — Wed, 12 Aug 2015 18:11:48 GMT

Hello Michael. I have run some updated code. The first 3 layers were updated however, everything else after only had the lyr.name within TOC updated, where the actual properties of the data still shows the original data source. I am not sure why. I tried just updating single layers at a time, and I got the same result. I find it strange that their are no syntax errors and that the datasource is correct (since it's changing the name in the TOC), but not actually updating the properties. Any comments/ideas are much appreciated. Thank you, Crystal Bowles

Example of portion of code attached:

Re: Arcpy: replaceDataSource (from gdb to SDE w/ change in data source name)

CrystalSchiffbauer-Bowles1 — Wed, 19 Aug 2015 20:24:17 GMT

UPDATE:

So, there apparently is known (though not-well documents) issues with arcpy and the replaceDataSource from one workspace to an SDE Workspace. ESRI customer support is trying to figure out the cause and perhaps answer. So... this issue has not been solved for replacing all Data Sources within to start with, a single mxd. I have been able to get a single feature class to update, sometimes, until it stops working... weird issues.

Anyway, here is the code that does work... sometimes:

>>> #import arcpython module
>>> import arcpy
>>> #define mapdocument
>>> mxd = arcpy.mapping.MapDocument (r"CURRENT")

>>> #ListLayers function to update DataSource

>>> for lyr in arcpy.mapping.ListLayers(mxd):

... if lyr.supports ("DATASOURCE"):

... if lyr.dataSource == r"full fc file path":

... lyr.replaceDataSource (r"Database Connections\connection name.sde", "SDE_WORKSPACE", "name.sde.SDE.newFCname")

... lyr.name = "FC name in TOC"

... del mxd

So, good luck and happy GIS-ing.

Re: Arcpy: replaceDataSource (from gdb to SDE w/ change in data source name)

CrystalSchiffbauer-Bowles1 — Wed, 23 Dec 2015 18:59:42 GMT

Happy Holidays all! So, this issue has not been resolved. We have a programmer working on the process - utilizing other code to see if it will work. We have yet to get any resolution. ESRI did put some effort into finding a solution, but to no avail. So... we are waiting to see what our programmer comes up with - to be honest, I have little hope it will work, as I have gone through and manually updated the data sources for more than 400 map documents since Sept. Sorry for not having a solution, but wanted to give an update anyway.

Happy Christmas and New Year!

Re: Arcpy: replaceDataSource (from gdb to SDE w/ change in data source name)

RebeccaStrauch__GISP — Wed, 23 Dec 2015 19:31:37 GMT

If you programmer hasn't looked at this already, you might want them to see how I did it. Not perfect and I'll be making additional updates at some point (off that project right now).

Python addin for data inventory and “broken-link” repair.