Change layer source using CIM

[JohnMcGlynn]

I am trying to change the data source geodatabase of ArcGIS Pro layers. The feature class name is unchanged.

It is the general consensus to use connectionProperties for this, however I have a number of feature classes  that do not have connectionProperties so I've elected to use CIM for these.

if hasattr(layr,'connectionProperties'):

Returns False.

For the CIM, If I just write:

lyrCIM = layr.getDefinition('V2')
layr.setDefinition(lyrCIM)

It throws an error! The error message is the line number. Ugh.

What I want to do is:

lyrCIM = layr.getDefinition('V2')
dc = lyrCIM.featureTable.dataConnection
dc.workspaceConnectionString = f"DATABASE={newdb}"
layr.setDefinition(lyrCIM)

Which throws the same (useless) error.

My ArcGIS Pro version is 2.9.5

[DuncanHornby]

You do not show how you get `layr`, so its impossible for anyone to answer this question. My immediate thoughts are maybe you have grabbed a group layer ...

[mody_buchbinder]

Also check if you have joined table. It change the dataConnection value.

[JohnMcGlynn]

The layr object is just the result of the listLayer() function:

for layr in thismap.listLayers():
    if layr.isGroupLayer:
        continue

None of the layers are joined.

[DuncanHornby]

So you are filtering out grouplayers and processing only layers (which have no joins); again because you are not showing the full script its not possible to determine certain actions, such as saving the change you have made via CIM? I suggest you post up a minimum script that reproduces the problem because posting fragments of code is unhelpful in spotting the issue.

[Brian_Wilson]

Interesting --- I think doing a "set" should never throw errors even on a group layer?? Huh. Well, I can just suggest wrapping the calls in a try block so that you see a (possibly USELESS) error message instead of it breaking on a line number that means nothing to you.

Here  is a code fragment for you...

i = 0
for l in layers:
    name = 'noname!'
    try:
        name = l.name
        cim = l.getDefinition('V2')
        l.setDefinition(cim)
        print(i, name)
    except Exception as e:
        print(i, name, e)
    i += 1

Here is the output my map produces. There are a bunch of broken layers in there (broken data source) but you can see it has no problems doing get and set.

0 New Group Layer
1 Taxlot Number
2 Situs Address
3 Mailing Address
4 Taxlots Polygons
5 Unimproved Taxlots
6 Vector_Tile_Labels
7 Taxlots Test Hosted
8 noname! The attribute 'name' is not supported on this instance of Layer.
9 noname! The attribute 'name' is not supported on this instance of Layer.
10 noname! The attribute 'name' is not supported on this instance of Layer.
11 noname! The attribute 'name' is not supported on this instance of Layer.
12 noname! The attribute 'name' is not supported on this instance of Layer.
13 Taxlots Test Registered and Feature
14 noname! The attribute 'name' is not supported on this instance of Layer.
15 noname! The attribute 'name' is not supported on this instance of Layer.
16 noname! The attribute '

 Note that the first line is a group and it contains the next 5 layers.

[JohnMcGlynn]

Thanks Guys. The process will update the gdb of a layer source. The feature class name is the same.

Here is the full code.

   def execute(self, parameters, messages):
        gdbs = parameters[0].ValueAsText
        gdbs = gdbs.replace("'", "")
        source_gdbs = gdbs.split(';')

        completecount = 0
        missedcount = 0

        try:
            arpx = arcpy.mp.ArcGISProject("CURRENT")
            thismap = arpx.activeMap

            tot = len(thismap.listLayers())
            arcpy.SetProgressor("step", "Layers", 0, tot, 1)
            n = 0
            messages.addMessage(f"Total Layers {str(tot)}")

            # Scan the geodatabases
            gdblist = self.scanGeodatabases(source_gdbs)

            for layr in thismap.listLayers():
                # Name and longName. Note that not all layers support name.
                # If os.path.basename is used to get name, then it returns 2022 for a layer named 'The Layer (13/12/2022)'
                longname = layr.longName
                if '\\' in longname:
                    lset = longname.split('\\')
                    index = len(lset) - 1
                    lname = lset[index]
                else:
                    lname = longname

                hasdesc = False
                hastype = False
                hascatpath = False
                try:
                    desc = arcpy.Describe(layr)
                    hasdesc = True
                    ltype = desc.dataType
                    hastype = True
                    catalogpath = ''
                    if hasattr(desc,'catalogPath'):
                        catalogpath = desc.catalogPath
                        hascatpath = True
                    #messages.addMessage(f"{ltype} {longname}")
                except:
                    pass

                arcpy.SetProgressorLabel(lname)
                arcpy.SetProgressorPosition(n)
                n += 1

                isGroup = layr.isGroupLayer
                if not isGroup and hastype and  ltype != 'GroupLayer':
                    if ltype == "FeatureLayer" or ltype == "FeatureClass" or ltype == "RasterLayer" or ltype == "RasterDataset":
                        if hasattr(layr,'connectionProperties'):
                            connProp = layr.connectionProperties
                            # messages.addMessage(f"Type = {type(connProp)} End Type")
                            nType = False
                            try:
                                tst = connProp['dataset']
                            except Exception as ex:
                                nType = True

                            if nType:
                                messages.addMessage(f"Error: Layer {longname} connection contains a NoneType object")
                                missedcount += 1
                            else:
                                ##connPropDict = {k: v for k, v in connProp.items() if k is not None and v is not None}

                                fcname = connProp['dataset']
                                # fcname = layr.connectionProperties['dataset']
                                # messages.addMessage(f"Layer FC: {fcname}")
                                # FC in gdblist
                                ingdb = [g for g in gdblist if g['FC'] == fcname and g['FORT'] == 'F']
                                if len(ingdb) > 0:
                                    newconnprop = layr.connectionProperties
                                    db = ingdb[0]['GDB'].strip()
                                    newconnprop['connection_info']['database'] = db

                                    layr.updateConnectionProperties(connProp, newconnprop)
                                    messages.addMessage(f"{lname} Source updated {db}")
                                    completecount += 1
                                else:
                                    messages.addMessage(f"Layer {longname} not in databases")
                                    missedcount += 1
                        elif hascatpath:
                            fcname = os.path.basename(catalogpath)
                            ingdb = [g for g in gdblist if g['FC'] == fcname and g['FORT'] == 'F']
                            if len(ingdb) > 0:
                                db = ingdb[0]['GDB'].strip()
                                dbname = os.path.basename(db)
                                currentgdb = os.path.dirname(catalogpath)
                                currentname = os.path.basename(currentgdb)
                                # messages.addMessage(f"Current GDB: {currentname}\nNew GDB: {dbname}")
                                lyrCIM = layr.getDefinition('V2')
                                dc = lyrCIM.featureTable.dataConnection
                                # layr.setDefinition(lyrCIM) <- Testing

                                dc.workspaceConnectionString = f"DATABASE={db}"
                                messages.addMessage(f"New Wkspace {dc.workspaceConnectionString}")
                                layr.setDefinition(lyrCIM)
                                messages.addMessage(f"{longname} Source updated {db}")
                                completecount += 1
                            else:
                                messages.addMessage(f"Layer {longname} not in databases")
                                missedcount += 1
                        else:
                            messages.addMessage(f"{ltype} Layer {longname} has no catalogPath or connection properties")
                            missedcount += 1

                    else:
                        messages.addMessage(f"Cannot process {ltype} layer {longname}")
                        missedcount += 1

            messages.addMessage(f"Completed: {str(completecount)}. Missed {str(missedcount)}")

        except Exception as ex:
            exc_type, exc_obj, exc_tb = sys.exc_info()
            err = f"Error: Line {exc_tb.tb_lineno} {ex}"
            messages.addErrorMessage(f"{err}")
            return
        return

[mody_buchbinder]

Hi

First, the simple way to filter un relevant layers is to Describe and then check desc.dataType

It exists for any layer and should be FeatureLayer - no need to check if this property exists, it always does.

Then the best way to understand what you have in the CIM is first to run line by line in Python window in Pro.

The Python window will list the properties on the CIM dynamically and it is very easy to print the values so it much easier to understand how is the CIM build and what properties you have

Have Fun   

[DuncanHornby]

Alongside @mody_buchbinder advice I would recommend highly that you install the CIM viewer, this allows you to view the full tree of properties of a layer in your map. I find it invaluable for drilling down through the insane number of properties a layer has. You can download a compiled version for ArcPro 3.0 here.

[JohnMcGlynn]

It turns out the problem is with the layer grouping.

I ran the following code:

arpx = arcpy.mp.ArcGISProject("CURRENT")
thismap = arpx.activeMap

            for layr in thismap.listLayers():
                try:
                    longname = layr.longName
                    desc = arcpy.Describe(layr)
                    dty = desc.dataType
                    messages.addMessage(f"{longname} Type: {dty}")
                    lyrCIM = layr.getDefinition('V2')
                    layr.setDefinition(lyrCIM)
                    messages.addMessage(f"CIM saved")
                except Exception as ex10:
                    messages.addMessage(f"Error: {ex10}")

Which produced the (partial) output:

Flooding\Flooding Information from Council\Upper South Creek Flood Study 2017 - Camden Council\20 Year ARI Type: FeatureLayer
Error:
Flooding\Flooding Information from Council\Upper South Creek Flood Study 2017 - Camden Council\50 Year ARI Type: FeatureLayer
Error:
Flooding\Flooding Information from Council\Upper South Creek Flood Study 2017 - Camden Council\100 Year ARI Type: FeatureLayer
Error:
Flooding\Flooding Information from Council\Upper South Creek Flood Study 2017 - Camden Council\200 Year ARI Type: FeatureLayer
Error:
Flooding\Flooding Information from Council\Upper South Creek Flood Study 2017 - Camden Council\500 Year ARI Type: FeatureLayer
Error:
Flooding\Flooding Information from Council\Upper South Creek Flood Study 2017 - Camden Council\PMF Type: FeatureLayer
CIM saved

As you can see, only one layer (PMF) was able to save the CIM. The rest produced a non-existent error.

I had a look at the 100 Year ARI .lyrx file and there didn't seem to be anything wrong with it.

I removed the above layers from their groups and re-ran the code:

100 Year ARI Type: FeatureLayer
CIM saved
50 Year ARI Type: FeatureLayer
CIM saved
200 Year ARI Type: FeatureLayer
CIM saved
500 Year ARI Type: FeatureLayer
CIM saved
PMF Type: FeatureLayer
CIM saved

All the CIM data was able to be saved.

This does pose some interesting questions though. I wonder if ESRI could shed some light on it.