using arcpy to change data sources in aprx files

JoshuaChan2

I am about to break a lot of data sources in our team's .aprx files because we are overhauling the places we are storing our data. Our data is stored in shapefile and file geodatabase formats.

Is there a way to programmatically do this? I have found snippets online that answer slightly different questions, like changing a gdb or SDE connection. But in my case it might be

• changing  shapefile/feature class name,
• moving a GDB or shapefile to a different folder, or
• changing to a different GDB, different feature class,
• or some combo of the above.

the examples I found online only go down to the gdb level, as in changing to a new GDB and all the other features inside will magically get their links fixed.  Trying that method with the full path name to the feature class level didn't work for me.

I have so far got a script that

• goes thru folders to look for .aprx files
• uses the aprx object to return broken links eg. my_broke_list = my_aprx.listBrokenDataSources()
• and I have stored my data changes in a spreadsheet. I use pandas to read the columns for old and new. and was hoping to use that info to change the data source.

The missing piece is updating the data source. I've tried it different ways but each time the result is a new aprx file with the same old broken links. I've tried using the aprx property and the layer properties.

So far my only conclusion is that lyr.updateConnectionProperties(old,new) doesn't do what it is supposed to be doing...

I'm just testing on a code snippet trying to figure out how to do the layer connection fix and I've tried a few things. Latest version is:

import arcpy

arcpy.env.overwriteOutput = True

aprx = r"Q:\blah\testBrokenLinks\small_test.aprx"
aprxNew = r"Q:\blah\testBrokenLinks\small_test4.aprx"

arpxObj = arcpy.mp.ArcGISProject(aprx)
old = r'Q:\blah\testBrokenLinks\lfsa000b21a_e.shp'
new = r'Q:\blah\testBrokenLinks\postal_codes.shp'

for m in arpxObj.listMaps():
    for lyr in m.listLayers():
        if lyr.isBroken:
            print('broke')
            if lyr.supports("DATASOURCE"):
                if lyr.dataSource == old:
                    print(old)
                    lyr.updateConnectionProperties(old,new)
                    print(new)

arpxObj.saveACopy(aprxNew)
print ('finito')

any help or examples on how to do this? 

JoshuaChan2

I find the ESRI help links confusing and unfortunately I cannot find the one I'm looking for. But basically there is a layer property connection that uses a python dictionary to store old/new connection info.  That is the one you need. not the aprx properties method.  I am not very good with python nor dictionaries so I had a colleague help me decipher all this.  

I have an excel sheet that stores the old & new fullpath names. It goes thru the aprx looking for broken links. If found it compares it with the spreadsheet to grab the new connection properties and uses the layer.updateConnectionProperties method to change it. 

I have some extra lines to document where I have a broken layer but it's not in the spreadsheet - just to alert myself that there's some other layers that probably warrant future investigation. There's also a bit about "bcgw.bcgov" which is our SDE. I didn't want to record those as I was running from an IDE and not connected to our database. I just wanted to ignore those. 

# go thru a folder and look for aprx

import os
import arcpy
import pandas as pd

# !!! change this for your folder and your Excel that shows your old and new link
my_workspace = r"W:\blah\temp\test3"
my_changes_excel = r"W:\blah\temp\test3\test_data_changes.xlsx"

# !!! this excel stores the broken links that were not found in the spreadsheet above
new_excel = my_workspace + '/broken_layers.xlsx'
if os.path.exists(new_excel):
    os.remove(new_excel)

def make_aprx_list(wksp):
    # search the main directory for the APRX files
    aprx_list = []  # resetting the list
    for root, dirs, files in os.walk(my_workspace):
        for f in files: 
            if f.endswith(".aprx"): 
                print(os.path.join(root, f))
                aprx_list.append(os.path.join(root, f))
    return(aprx_list)

if __name__ == "__main__":
    mylist = make_aprx_list(my_workspace)
    df = pd.read_excel(my_changes_excel)
    for aprx_name in mylist: 
        # create pandas dataframe to store broke lyrs with no fix
        brokenlayers = pd.DataFrame(columns = ['broken layer'])
        #print(aprx_name)
        aprx = arcpy.mp.ArcGISProject(aprx_name) 
        for m in aprx.listMaps():  
            for l in m.listLayers():
                if l.isGroupLayer == False:
                    if l.isBroken == True:
                        if l.dataSource in df['old'].values:
                            new_lyr =  df.loc[df["old"]== l.dataSource, "new"].values[0]
                            #print(new_lyr)
                            new_connection = l.connectionProperties.copy()
                            old_connection = l.connectionProperties
                            new_connection['dataset'] = new_lyr.rsplit('\\', 1)[1] # new layer name
                            new_connection['connection_info']['database'] = new_lyr.rsplit('\\', 1)[0] #new layer location (i.e. gdb or shapefile in folder)
                            l.updateConnectionProperties(old_connection, new_connection)
                            print('Connection updated: ' + new_lyr.rsplit('\\', 1)[1])
                        else:
                            if l.dataSource.find('bcgw.bcgov') == -1:
                                print('Broken layer not found: ' + l.dataSource)
                                brokenlayers.loc[len(brokenlayers.index)] = l.dataSource
        split_name = aprx_name.rsplit('.',1) 
        new_name = split_name[0] + '_brokenLayersFixed'
        #aprx.saveACopy(new_name)
        aprx.save()
        # append record to Excel if the excel exists. if not create it then write to it. 
        if len(brokenlayers)>0:
           if os.path.exists(new_excel): 
                with pd.ExcelWriter(new_excel, mode="a") as writer:
                    brokenlayers.to_excel(writer, sheet_name = aprx_name.rsplit('\\', 1)[1])
           else: 
                with pd.ExcelWriter(new_excel, mode="w") as writer:
                    brokenlayers.to_excel(writer, sheet_name = aprx_name.rsplit('\\', 1)[1])
    print('finito')