topic Re: using arcpy to change data sources in aprx files in ArcGIS Pro Questions

using arcpy to change data sources in aprx files

JoshuaChan2 — Thu, 14 Mar 2024 17:48:14 GMT

I am about to break a lot of data sources in our team's .aprx files because we are overhauling the places we are storing our data. Our data is stored in shapefile and file geodatabase formats.

Is there a way to programmatically do this? I have found snippets online that answer slightly different questions, like changing a gdb or SDE connection. But in my case it might be

changing shapefile/feature class name,
moving a GDB or shapefile to a different folder, or
changing to a different GDB, different feature class,
or some combo of the above.

the examples I found online only go down to the gdb level, as in changing to a new GDB and all the other features inside will magically get their links fixed. Trying that method with the full path name to the feature class level didn't work for me.

I have so far got a script that

goes thru folders to look for .aprx files
uses the aprx object to return broken links eg. my_broke_list = my_aprx.listBrokenDataSources()
and I have stored my data changes in a spreadsheet. I use pandas to read the columns for old and new. and was hoping to use that info to change the data source.

The missing piece is updating the data source. I've tried it different ways but each time the result is a new aprx file with the same old broken links. I've tried using the aprx property and the layer properties.

So far my only conclusion is that lyr.updateConnectionProperties(old,new) doesn't do what it is supposed to be doing...

I'm just testing on a code snippet trying to figure out how to do the layer connection fix and I've tried a few things. Latest version is:

import arcpy arcpy.env.overwriteOutput = True aprx = r"Q:\blah\testBrokenLinks\small_test.aprx" aprxNew = r"Q:\blah\testBrokenLinks\small_test4.aprx" arpxObj = arcpy.mp.ArcGISProject(aprx) old = r'Q:\blah\testBrokenLinks\lfsa000b21a_e.shp' new = r'Q:\blah\testBrokenLinks\postal_codes.shp' for m in arpxObj.listMaps(): for lyr in m.listLayers(): if lyr.isBroken: print('broke') if lyr.supports("DATASOURCE"): if lyr.dataSource == old: print(old) lyr.updateConnectionProperties(old,new) print(new) arpxObj.saveACopy(aprxNew) print ('finito')

any help or examples on how to do this?

Re: using arcpy to change data sources in aprx files

KimOllivier — Fri, 15 Mar 2024 09:43:23 GMT

line 19. You are changing Connections, not featureclass paths. (I do not know why a shapefile exists in your system!!) I could not find one to test out your script!

line 19. The updateConnectionProperties is a search and replace function, so you need to have a string replace option, not a full path option

line 9 . If you are comparing paths it is safer to normalize the paths for string comparisons using old = os.path.normpath(old_path).

Maybe updateFolderConnections would work better for shapefiles

Also you are changing the wrong object source. The project has a separate connections. You should be changing the Layer connection properties.

Re: using arcpy to change data sources in aprx files

JoshuaChan2 — Fri, 15 Mar 2024 15:22:01 GMT

for line 19 re: shapefiles - actually we also have KMZs too which I'm not going to talk about...

also line 19: I was hoping the search and replace function would have worked for the full path like searching for an entire sentence vs just replacing a single word.

line 9; in the "real" script I'm getting the list of broken layers to review from the aprx.ListBrokenDatasources() and then comparing those to my excel sheet where I hold the old vs new. So the old/new will be a text string

I'll try layer connections and let you know how it goes.

Re: using arcpy to change data sources in aprx files

JoshuaChan2 — Mon, 18 Mar 2024 22:11:23 GMT

so far no luck... I've tried ChatGPT too but it pointed me to a function that existed in ArcMap 10.8... not pro.

anyone else?

Re: using arcpy to change data sources in aprx files

HollyTorpey_LSA — Wed, 17 Apr 2024 00:34:58 GMT

I am in the same boat. Did you ever figure out a way to update paths in an .aprx file? In our case, really all we need to do is find every reference to an old server name in UNC paths (in hundreds of projects) and replace it with the new server name. Everything else stays the same. If you have any advice from your recent trials, I'm all ears.

Thanks!

Re: using arcpy to change data sources in aprx files

KimOllivier — Wed, 17 Apr 2024 03:07:53 GMT

Good question, I dropped the issue for myself so I never finished debugging for Joshua. Maybe I will have another look to develop more tips with an example.

Re: using arcpy to change data sources in aprx files

SimonSchütte_ct — Wed, 17 Apr 2024 06:29:58 GMT

Here are some links that may be helpful:
Replacing datasource does not work through arcpy - Esri Community
Updating and fixing data sources—ArcGIS Pro | Documentation
Python Scripting: Repairing Data Sources | Esri Training Web Course

Re: using arcpy to change data sources in aprx files

JoshuaChan2 — Wed, 17 Apr 2024 15:31:04 GMT

I find the ESRI help links confusing and unfortunately I cannot find the one I'm looking for. But basically there is a layer property connection that uses a python dictionary to store old/new connection info. That is the one you need. not the aprx properties method. I am not very good with python nor dictionaries so I had a colleague help me decipher all this.

I have an excel sheet that stores the old & new fullpath names. It goes thru the aprx looking for broken links. If found it compares it with the spreadsheet to grab the new connection properties and uses the layer.updateConnectionProperties method to change it.

I have some extra lines to document where I have a broken layer but it's not in the spreadsheet - just to alert myself that there's some other layers that probably warrant future investigation. There's also a bit about "bcgw.bcgov" which is our SDE. I didn't want to record those as I was running from an IDE and not connected to our database. I just wanted to ignore those.

# go thru a folder and look for aprx import os import arcpy import pandas as pd # !!! change this for your folder and your Excel that shows your old and new link my_workspace = r"W:\blah\temp\test3" my_changes_excel = r"W:\blah\temp\test3\test_data_changes.xlsx" # !!! this excel stores the broken links that were not found in the spreadsheet above new_excel = my_workspace + '/broken_layers.xlsx' if os.path.exists(new_excel): os.remove(new_excel) def make_aprx_list(wksp): # search the main directory for the APRX files aprx_list = [] # resetting the list for root, dirs, files in os.walk(my_workspace): for f in files: if f.endswith(".aprx"): print(os.path.join(root, f)) aprx_list.append(os.path.join(root, f)) return(aprx_list) if __name__ == "__main__": mylist = make_aprx_list(my_workspace) df = pd.read_excel(my_changes_excel) for aprx_name in mylist: # create pandas dataframe to store broke lyrs with no fix brokenlayers = pd.DataFrame(columns = ['broken layer']) #print(aprx_name) aprx = arcpy.mp.ArcGISProject(aprx_name) for m in aprx.listMaps(): for l in m.listLayers(): if l.isGroupLayer == False: if l.isBroken == True: if l.dataSource in df['old'].values: new_lyr = df.loc[df["old"]== l.dataSource, "new"].values[0] #print(new_lyr) new_connection = l.connectionProperties.copy() old_connection = l.connectionProperties new_connection['dataset'] = new_lyr.rsplit('\\', 1)[1] # new layer name new_connection['connection_info']['database'] = new_lyr.rsplit('\\', 1)[0] #new layer location (i.e. gdb or shapefile in folder) l.updateConnectionProperties(old_connection, new_connection) print('Connection updated: ' + new_lyr.rsplit('\\', 1)[1]) else: if l.dataSource.find('bcgw.bcgov') == -1: print('Broken layer not found: ' + l.dataSource) brokenlayers.loc[len(brokenlayers.index)] = l.dataSource split_name = aprx_name.rsplit('.',1) new_name = split_name[0] + '_brokenLayersFixed' #aprx.saveACopy(new_name) aprx.save() # append record to Excel if the excel exists. if not create it then write to it. if len(brokenlayers)>0: if os.path.exists(new_excel): with pd.ExcelWriter(new_excel, mode="a") as writer: brokenlayers.to_excel(writer, sheet_name = aprx_name.rsplit('\\', 1)[1]) else: with pd.ExcelWriter(new_excel, mode="w") as writer: brokenlayers.to_excel(writer, sheet_name = aprx_name.rsplit('\\', 1)[1]) print('finito')