topic Re: layer workspace path points to a .sde file that doesn't exist, yet the layer still draws in Python Questions

layer workspace path points to a .sde file that doesn't exist, yet the layer still draws

TheodoreRakel — Mon, 24 Aug 2015 17:22:12 GMT

I've been working with a .mxd that has some broken links. I have been using the following python to print the workspace path for each layer:

>>> for lyr in arcpy.mapping.ListLayers(mxd):

... if lyr.supports("workspacePath"):

... source = lyr.workspacePath

... print "%s -> %s" % (lyr, source)

I found that layers do successfully get drawn and their data source is correct in arc map even if the workspacePath points to a .sde file that doesn't exist on the machine. If the links aren't broken, then there's no need to fix them; but I'm curious how ArcMap is able to find the data in sde if the layer workspacePath is not correct.

Re: layer workspace path points to a .sde file that doesn't exist, yet the layer still draws

WesMiller — Mon, 24 Aug 2015 17:25:05 GMT

Are they ".lyr" files that are drawing? I haven't looked this up yet, but I've been suspicious for a while that ".lyr" files are saved as part of the map document.

Re: layer workspace path points to a .sde file that doesn't exist, yet the layer still draws

TheodoreRakel — Mon, 24 Aug 2015 17:28:22 GMT

No, they are not .lyr files. They are feature classes in SDE.

Re: layer workspace path points to a .sde file that doesn't exist, yet the layer still draws

RebeccaStrauch__GISP — Sat, 11 Dec 2021 13:57:14 GMT

I've been running in to the same thing/question. I've been finding that the credentials to the sde are stored in the mxd and it seems to ignore the actual .sde file. It makes it a little more difficult to update the sde connection with python, but it is possible (I'm working on it myself, and have had some success).

Just fyi - AGS services can also have unique needs when replacing connections.

Theodore Rakel Here is a test you can try to test what I'm trying to say above. First, make sure you have you mxd backed up. Open the mxd, and to make it easier, copy the sde layer (in the TOC) and move it to the top....this will make the layer in the list [0]. Then go thru the commands listed below one by one ....make sure to note the place you need to "fix" the source manually. This is one of the processes I have bee using to help figure out the issue. I hope this helps.

import arcpy
mxd = arcpy.mapping.MapDocument("CURRENT")
dfs = arcpy.mapping.ListDataFrames(mxd)
lyrList = arcpy.mapping.ListLayers(mxd)
lyr = lyrList[0]
lyr.isBroken
lyr.dataSource
lyr.name
lyr.isBroken

# manually fix the source now

lyr.dataSource
lyr.name
lyr.name = lyr.datasetName
lyr.name

Re: layer workspace path points to a .sde file that doesn't exist, yet the layer still draws

JoshuaBixby — Mon, 24 Aug 2015 18:38:38 GMT

What you are running into is expected behavior. Prior to ArcGIS 10.1, how an SDE connection file was accessed when first loading data determined whether the connection information was embedded in the map document or referenced in the map document: Bug: SDE connection information is not preserved in an MXD if a Universal Naming Convention (UNC) path is used. The issue was "addressed" in ArcGIS 10.0 SP5 and ArcGIS 10.1.

Personally, I disagreed and continue to disagree that it was a bug that needed to be fixed. The embedded vs. referenced behavior was around since the ArcGIS 8.x days, even though the aforementioned KB only covers back to 9.1. Allowing for SDE connections to be referenced instead of embedded allowed for flexibility in managing large swathes of SDE layers, but I guess it was too complicated for most users to get their head around. Alas, everything was changed to embedded.

Now that SDE connection properties are embedded, it is important to always use/check the embedded properties in the map document and not use a Describe object to look up the connection properties of an SDE connection file referenced with the workspacePath. It is completely possible that an SDE connection file referenced with workspacePath could have different connection properties than when it was used to load data into a map document.

Re: layer workspace path points to a .sde file that doesn't exist, yet the layer still draws

TheodoreRakel — Mon, 24 Aug 2015 19:21:43 GMT

Thanks. How do you check the embedded properties in a mxd? Can I do that with arcpy?

Re: layer workspace path points to a .sde file that doesn't exist, yet the layer still draws

MichaelVolz — Mon, 24 Aug 2015 19:30:53 GMT

if layr.supports("WORKSPACEPATH"):

# logfile.write("The datasetName before replacement is " + layr.datasetName + '\n')

# logfile.write("The old connection string before replacement is " + layr.workspacePath + '\n')

if layr.supports("DATASOURCE"):

if layr.supports("SERVICEPROPERTIES"):

print("The layer name in mxd in Prod is " + layr.name)

servProp = layr.serviceProperties

user = str(servProp.get('UserName', 'N/A'))

serverID = str(servProp.get('Server', 'N/A'))

serverInstance = str(servProp.get('Service', 'N/A'))

serverVersion = str(servProp.get('Version', 'N/A'))

Re: layer workspace path points to a .sde file that doesn't exist, yet the layer still draws

JoshuaBixby — Sat, 11 Dec 2021 13:57:17 GMT

To print out the embedded connection properties for all SDE layers in an open map document:

mxd = arcpy.mapping.MapDocument("CURRENT")
for lyr in arcpy.mapping.ListLayers(mxd):
    if lyr.supports("SERVICEPROPERTIES"):
        if lyr.serviceProperties["ServiceType"] == "SDE":
            for k, v in lyr.serviceProperties.iteritems():
                print "Property: {:<30}Value:{}".format(k,v)

Re: layer workspace path points to a .sde file that doesn't exist, yet the layer still draws

RebeccaStrauch__GISP — Mon, 24 Aug 2015 20:18:37 GMT

Joshua, at least for some of my really old SDE connections (still sees a version that uses port 5151), and testing using 10.2.2, sde connections do not support "SERVICEPROPERTIES" So, that will not work. just fyi. May work on other broken SDE connections.

I have found a way to find them and list properties, but it's a process of elimination....not quite as straight forward as the above. fwiw.

Re: layer workspace path points to a .sde file that doesn't exist, yet the layer still draws

MichaelVolz — Mon, 24 Aug 2015 20:29:01 GMT

Rebecca:

Do you really need to restore such old port 5151 connections? In order to make your life easier (less coding) you might think about just resourcing direct connections and let endusers resource the older port 5151 connections themselves with the Set Data Source tool available in ArcCatalog.

Re: layer workspace path points to a .sde file that doesn't exist, yet the layer still draws

RebeccaStrauch__GISP — Mon, 24 Aug 2015 20:44:46 GMT

Actually I'm creating a toolbox/Python-addin that makes it pretty easy. I have several tools

1-find GDB in a folder (recursive....each tool does a walk, so includes subfolders)

2-list all feature classes in the folder

3a-list all the (unique) broken links, by type, within all the MXDs in a folder....writting a .txt, .csv, and .xls as output

3b- does same as 3a, but lets you replace all the top level (i.e drive->drive, drive->UNC, etc) before runing the inventory

I use the output of 3a/b to add a column of all the newpaths I need, and new type in the case of changing shape/covers/sde -> FGDB, sde->service etc.

4 - takes modified csv as input .and fixes all the links.

Although I ran 3a on a directory with about 1500+ mxd, 50k+ individual broken links, and about 3k unique broken links, I'm now repairing project by project so I can make sure it works before moving to next project.

To answer your question....30 years of project and data I need to make sure is working after many changes. This is an easy way to do it, and I'm trying to get all the connections to UNC so connections don't break do to different mapping by different users. I think I now have it working for almost every combination...but just starting to put into actual use (vs testing), so may still hit a road block somewhere down the road. and that's why I make sure to back up the mxd's (to a different folder, of course) before running #4.

Re: layer workspace path points to a .sde file that doesn't exist, yet the layer still draws

JoshuaBixby — Mon, 24 Aug 2015 20:56:17 GMT

What you are running into is a bug that was addressed in ArcGIS 10.3, i.e., prior to ArcGIS 10.3 the embedded service properties were not available when an SDE data source was broken. If you try on a 10.3 machine, I would be interested to see the results. Normally I would provide links to documentation, but the Esri Support and Development documentation on this bug is a mess in terms of consistency and accuracy. There is still an open bug with Esri Support even though Esri Development fixed it already.

Re: layer workspace path points to a .sde file that doesn't exist, yet the layer still draws

RebeccaStrauch__GISP — Mon, 24 Aug 2015 21:05:29 GMT

Thanks Joshua, yes it does work in 10.3. I'm trying to keep my addin working 10.2.2 since I have users that have not upgraded yet. Keeping a toolbox from not automatically thinking it's 10.3 means I need to make sure I do all my "touching of the toolbox" and addin builds on the 10.2.2 machine. The toolbox actually seems to get corrupted for 10.2.2 if I just open it in 10.3....but that is a topic for another day.

Good to know that works though!