layer.dataSource-- did it or did it not use to return a file extension?

AlfredBaldenweck

Can someone please confirm if I'm going crazy? 

I have a script that checks either a layer's dataSource or catalog path, if it doesn't support dataSource, so then I can repair broken links.

1. Find dataSource/catalogpath
2. Check spreadsheet for it
3. Replace source with corresponding value from the spreadsheet

Here's the thing.

As recently as the 9th of April, is run on a shapefile, it would return "folder\fc.shp"

This morning, it's returning "folder\fc"

So, my question is: Am I going crazy? It worked literally last week. My version of Pro has not changed. My test data has not changed. I'm editing in a copy of the script, and the original that I used to test last week has not changed.

This morning I had to actually add in ".shp" to make the script work on the replacement side, since "folder\fc" doesn't exist but "folder\fc.shp" does.

What is going on?

 

AlfredBaldenweck

Well, the good news is that the behaviour of layer.dataSource hasn't changed and I am not, in fact, going crazy. 

The bad news is the behaviour of something else changed and I am going crazy.

 

So, the script does a find and replace for the data source using CIM.

I've done extensive testing in the last two hours and have determined it's something in that part of the code that's causing the issue, and have confirmed it using the Python Window.

Basically, when you create a layer using MakeFeatureLayer() and then steal its CIM properties, the dataset doesn't come across correctly.

My test data is three distinct shapefiles in three folders:

• Downloads\testFR.shp
• Downloads\test\testFR.shp
• Downloads\testmdb\testFR.shp

They are named the same for this particular workflow, but they have different records.

 

The code checks a dictionary of folders to show which datasource I'm trying to switch these things to.

It then does a find/replace on the layer's current datasource, checks to see if that new file path actually exists, then loads it as a layer into memory to steal from its CIM .

The problem that I'm having is that this layer that is created from MakeFeatureLayer() doesn't specify the file extension in "dataset", so instead of returning "dataset: testFR.shp", it returns "dataset: testFR", which doesn't exist.

Please note that changing the CIM version from 'V2' to 'V3' makes no difference.

The weird thing is, again, I didn't have this problem last week but I do this week. In fact, I was able to run this code over and over and watch the three files change places each time. Now I can't chain it anymore.

I'm very confused, especially given that the data and the version of Pro have not changed. 

(stripped down code in spoilers)

Spoiler

import os
# a dictionary of folders, origin and destination.
sourceDict = {r"\C:\Downloads\test" : r"\C:\Downloads",
              r"\C:\Downloads\testmdb" : r"\C:\Downloads\test" ,
              r"\C:\Downloads" : r"\C:\Downloads\testmdb"}
              
lays = arcpy.mp.ArcGISProject("CURRENT").activeMap.listLayers("testFR*")
ver = 'V2'
for lay in lays:
    print(f"Original source: {lay.getDefinition(ver).featureTable.dataConnection.__dict__}")
    # Look for the folder that we're trying to replace here.
    layPathR ='' #Blank to start, gets populated with the right folder.
    layPath = lay.dataSource
    print(f"Original Data Source: {layPath}")
    checkLen = 0
    # Check for closest folder match in the dictionary
    #   "Downloads\test" is closer to "Downloads\testmdb" than 
    #   "Downloads" is.
    for key in sourceDict:
        if key in os.path.dirname(layPath):
            # print("hi")
            if len(key) > checkLen:
                check = key
                checkLen = len(key)
    layPathR = check
    
    if layPathR in sourceDict:
        # Get the layer's CIM definition
        layCIM = lay.getDefinition(ver)
        if lay.isFeatureLayer:
            print(f"We are replacing this: {layPathR}")
            print(f"with this {sourceDict[layPathR]}")
            # Replace part of the path of the current dataSource with 
            # the value of that part in the dictionary
            # (e.g. replace "Downloads" with "Downloads\testmdb")
            testlay = layPath.replace(layPathR, 
                                       sourceDict[layPathR])

            print(f"New Data Source: {testlay}")
            if arcpy.Exists(testlay):
                # Only do anything if the other file actually exists.
                # You need the [0] in here or else you a Results Object,
                # which is pretty dumb.
                tempLay = arcpy.management.MakeFeatureLayer(testlay, "tempLay")[0]
            else:
                print("Didn't Work")
                continue

            tLCIM = tempLay.getDefinition(ver).featureTable.dataConnection
            print(f"I am making the layer's source the following {tLCIM.__dict__}")
            # Set the new data connection to the layer's CIM featureTable
            layCIM.featureTable.dataConnection = tLCIM
            lay.setDefinition(layCIM)
            print(f"New source: {lay.getDefinition(ver).featureTable.dataConnection.__dict__}")
    else:
        print("Nope")
print("done")

'''Results'''
'''
Original source: {'customParameters': [], 
                  'workspaceConnectionString': 'DATABASE=C:\\Downloads\\test', 
                  'workspaceFactory': 'Shapefile', 
                  'customWorkspaceFactoryCLSID': '', 
                  'dataset': 'testFR.shp', 
                  'datasetType': 'esriDTFeatureClass'}
Original Data Source: C:\Downloads\test\testFR.shp

We are replacing this: C:\Downloads\test

with this C:Downloads

New Data Source: C:Downloads\testFR.shp

I am making the layer's source the following 
                 {'customParameters': [], 
                  'workspaceConnectionString': 'DATABASE= C:\\Downloads', 
                  'workspaceFactory': 'Shapefile', 
                  'customWorkspaceFactoryCLSID': '', 
                  'dataset': 'testFR', 
                  'datasetType': 'esriDTFeatureClass'}
                  
New source: {'customParameters': [], 
             'workspaceConnectionString': 'DATABASE=C:\\Downloads', 
             'workspaceFactory': 'Shapefile', 
             'customWorkspaceFactoryCLSID': '', 
             'dataset': 'testFR', 
             'datasetType': 'esriDTFeatureClass'}
'''