topic Re: Python script for mass find and replace of workspace path? in Python Questions

Python script for mass find and replace of workspace path?

KevinBladon — Wed, 23 Jan 2013 19:20:10 GMT

Hi folks,

As our research group has run out of space on a network drive, our IT department just moved all of our folders/data to a new network drive. However, in doing this all of the folder structures for the data locations have changed. As a result, none of our project files (e.g., .mxd) are functioning. As we have ~150-200 .mxd project files does anyone know if there is a piece of code which can automatically work through all of the ArcGIS related folders and find and replace the mapped network drive info or the folder structure.

I found this piece of code, but this only works for a single file. What I would like to be able to do is a batch process to look through all folders/subfolders/files and find and replace the workspace path (Note: we are working with ArcGIS 10.1).

import arcpy
mxd = arcpy.mapping.MapDocument(r"C:\Project\Project.mxd")
mxd.findAndReplaceWorkspacePaths(r"C:\Project\Data", r"D:\Project\Data")
mxd.saveACopy(r"D:\Project\Project.mxd")
del mxd

Thanks.

Re: Python script for mass find and replace of workspace path?

ChrisPedrezuela — Sat, 11 Dec 2021 13:39:30 GMT

This might help you,

import arcpy, os

path = os.getcwd() # or path = r"C:\Documents"

for fileName in os.listdir(path):
    fullPath = os.path.join(path, fileName)
    if os.path.isfile(fullPath):
        basename, extension = os.path.splitext(fullPath)
        if extension == '.mxd':
            mxd = arcpy.mapping.MapDocument(fullPath)

            mxd.findAndReplaceWorkspacePaths(r"oldfilepath1", r"newfilepath1")
            mxd.findAndReplaceWorkspacePaths(r"oldfilepath2", r"newfilepath2")
                      
            mxd.save()

del mxd

Hi folks,

As our research group has run out of space on a network drive, our IT department just moved all of our folders/data to a new network drive. However, in doing this all of the folder structures for the data locations have changed. As a result, none of our project files (e.g., .mxd) are functioning. As we have ~150-200 .mxd project files does anyone know if there is a piece of code which can automatically work through all of the ArcGIS related folders and find and replace the mapped network drive info or the folder structure.

I found this piece of code, but this only works for a single file. What I would like to be able to do is a batch process to look through all folders/subfolders/files and find and replace the workspace path (Note: we are working with ArcGIS 10.1).

import arcpy
mxd = arcpy.mapping.MapDocument(r"C:\Project\Project.mxd")
mxd.findAndReplaceWorkspacePaths(r"C:\Project\Data", r"D:\Project\Data")
mxd.saveACopy(r"D:\Project\Project.mxd")
del mxd

Thanks.

Re: Python script for mass find and replace of workspace path?

Luke_Pinner — Sat, 11 Dec 2021 13:39:33 GMT

Tweaked a bit to walk through subfolders.

import os,fnmatch,arcpy
def find(pattern, path=os.getcwd()):
    for path, dirs, files in os.walk(path):
        for filename in files:
     if fnmatch.fnmatch(filename, pattern):yield os.path.abspath(os.path.join(path, filename))
   
for f in find('*.mxd', 'C:/Data'):   
    mxd = arcpy.mapping.MapDocument(f)
    mxd.findAndReplaceWorkspacePaths(r"oldfilepath1", r"newfilepath1")
    mxd.findAndReplaceWorkspacePaths(r"oldfilepath2", r"newfilepath2")
   
    mxd.save()

Re: Python script for mass find and replace of workspace path?

CarrieDavis — Tue, 05 Feb 2013 17:04:23 GMT

This script should work to list folders in a workspace then list .mxd documents in those folders. Then you can update the sources using arcpy.mapping.
[ATTACH=CONFIG]21457[/ATTACH]

Re: Python script for mass find and replace of workspace path?

JoshJones4 — Wed, 29 Jul 2015 17:10:43 GMT

Hey Kevin.

I was just curious if either of these answers worked for you. I am about to be in the same boat, with somewhere near a thousand MXD's. I'm going to have to put something together over the next couple of days.

Thanks Kevin.

-Josh

Re: Python script for mass find and replace of workspace path?

RebeccaStrauch__GISP — Wed, 29 Jul 2015 21:08:26 GMT

This addin/toolbar (for ArcCatalog) is still in my final developement.....but I'm getting close to getting my addin that will do several things, including listing all your broken links, and then give the option to update. I'll attache the documentation, which is all over the place as far as instructions, and the addin, which has been tested in 10.3.x (but should work in 10.2). Instruction for loading (from an email I sent to test staff earlier this morning)

Save the ChkandFixLinks.esriaddin file to anywhere handy. Double click, it will ask if you want to install…click Install Add-in.

Then open or reopen ArcCatalog. If you don’t see the toolbar, or not sure, go to Customize (on top menu bar)

Click on that and you will see Toolbars> …hover to the right and make sure the toolbar is checked. If you don’t see it, click it off/on until you see where it is.

Click on the third icon over (for the one that will give a unique list of all broken links in a folder)

In the dialog box, use the folder icon and then navigate to the folder where all you mxd’s are located (it will go thru (walk) all the subfolders underneath too) and leave the other two as they are

Then click ok, and it will run. When completed, it will have created a couple files…they are the same, just one is and Excel file, and the other is comma delimited….I suggest opening the Excel (.xls) file to review. They will be called BrokenSrc with the date/time it was run.

Sorry this is pretty messy....when it's completed with the additional scripts, I will post (probably to ArcScripts 2.0). The 4th icon over will do a bulk change from drive->drive or drive->UNC look at the Samples folder (if you unzip the esriaddin ("after installing) and there is a samples folder for format. After the bulk updates, it will run a new unique list. The next script which is almost complete, will allow for changing longer paths, shapes or coverages to FGDB, etc. Feedback is welcome and appreciated (especially if it doesn't work at all for you). BTW - the first three tools don't change anything, just create lists/reports so you can see what you need to fix....i.e. other than createing .txt, .xls, and .csv files (all with unique name_date_time for file name, they can not do any damage (but you must have write access to the folder you are analyzing. (although setting the path might work (haven't tested).

I hope these tools help.

Re: Python script for mass find and replace of workspace path?

JaredPilbeam1 — Sat, 11 Dec 2021 13:39:36 GMT

@Rebecca

This is a great addin. I just tried it and It works fine for listing broken source data. But, my problem is, I need to list broken picture elements. That's something it didn't do?

I've been trying to use this code (below) from Esri. But, it's not working due to the fact, I think, that the mxd it's referencing is missing the pictures (our file directory was recently changed, so the picture path is broken). It runs with no errror and 1st_District_B3.mxd is saved in the folder, but no changes are made.

import arcpy

mxd = arcpy.mapping.MapDocument(r'R:\AtlasMaps\ATLAS_MAPS_17\New folder\1st_District_B2.mxd')
for elm in arcpy.mapping.ListLayoutElements(mxd, "GRAPHIC_ELEMENT", "*logo*"):
    if elm.name == 'County-logo-1x1Small':
        elm.sourceImage = r'P:\Images\County-logo.JPG'
        print '{} changed'.format(elm.sourceImage)
mxd.saveACopy(r'R:\AtlasMaps\ATLAS_MAPS_17\New folder\1st_District_B3.mxd')
del mxd‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Below's a printscn of the document with the broken picture element source. It's giving you the path where the JPEG once lived. It's now in the P:\Images folder.

Are you aware of a way to list and repair broken source picture elements?

Thanks in advance.

Re: Python script for mass find and replace of workspace path?

RebeccaStrauch__GISP — Tue, 24 Jan 2017 17:30:53 GMT

I haven't tried to test for graphic elements, but I can take a look at this later (probably not until later this week). I know I have a few more types of broken links I wanted to test for, but have moved off of that project for the time being, and since it does about 98%.

I may have a newer version and better description of my addin on this page, fwiw.

/blogs/myAlaskaGIS/2015/08/31/python-addin-for-data-inventory-and-broken-link-repair

Can you zip you mxd and attach it? all items will be broken of course for me, but it will give me something to test. Thanks.

Re: Python script for mass find and replace of workspace path?

IanMurray — Tue, 24 Jan 2017 17:37:05 GMT

Hi Jared,

Can you open you map and check to make sure your picture element is still named?

Alternatively, instead of checking against the name property of the picture element, you could always check against the current imageSource property is "" \\Jackson\gis_dept\Data\Images\County-logo-1x1Small.JPG" and if so then change to r'P:\Images\County-logo.JPG'.

Unfortunately there is no method in arcpy for checking if Picture Elements to see if the link is no longer valid. There is probably some python module somewhere that could do this and could be used in conjunction with arcpy to check and update picture element paths

Re: Python script for mass find and replace of workspace path?

JaredPilbeam1 — Tue, 24 Jan 2017 19:11:07 GMT

http://www.willcogis.org/website2014/gis/Data/1st_District_B2.zip

Here's a link to the mxd. Thanks for taking a look.

I checked your page and downloaded the newer version you had there. I ran it again and pictures (.JPG in my case) are still not included in the list of broken sources. You have 'raster.jpg' included in the valid extension list, but this is apparently different than a picture element (.JPG).

Re: Python script for mass find and replace of workspace path?

JaredPilbeam1 — Tue, 24 Jan 2017 19:29:44 GMT

Hi Ian,

Thanks. Basically, all my map documents have a broken link now for that picture element. Since the directory was changed, \\Jackson\gis_dept\Data\Images\County-logo-1x1Small.JPG" is a dead path. As you said, I could update the path manually, but it's too time consuming.

If and when I find a method to fix broken picture links and then repair them I'll post it.

Re: Python script for mass find and replace of workspace path?

IanMurray — Tue, 24 Jan 2017 19:35:48 GMT

Is the path for all the broken pictures the one you noted above? It would be fairly easy to have a script walk through all the map documents in a directory and replace all the picture elements with that specific path with a new path.

Also I noted something fairly interesting, when a link is working to an picture element, the element name is just the name of the image itself without the file extension, whereas when I opened your map with broken links, the name was now the filepath to where the image was, instead of just the file name.

Re: Python script for mass find and replace of workspace path?

JaredPilbeam1 — Sat, 11 Dec 2021 13:39:38 GMT

Yes.

Not working/old path: \\Jackson\gis_dept\Data\Images\County-logo-1x1Small.JPG

Working/new path: P:\Images\County-logo.JPG

I thought it wouldn't be bad either? The script I posted above, and again here, seems nice. But it doesn't want to work in a case where the path is broken. So, line 3 is calling the mxd I zipped above. And, as noted, because of the broken path, the pictures aren't in the mxd. That's why I think this script isn't working. In other words, I think the pictures have to be there for the script to work.

Description: find a picture element using a wildcard and then change the picture's data source

import arcpy

mxd = arcpy.mapping.MapDocument(r'R:\AtlasMaps\ATLAS_MAPS_17\New folder\1st_District_B2.mxd')
for elm in arcpy.mapping.ListLayoutElements(mxd, "GRAPHIC_ELEMENT", "*logo*"):
    if elm.name == 'County-logo-1x1Small':
        elm.sourceImage = r'P:\Images\County-logo.JPG'
        print '{} changed'.format(elm.sourceImage)
mxd.saveACopy(r'R:\AtlasMaps\ATLAS_MAPS_17\New folder\1st_District_B3.mxd')
del mxd‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Taken from here: ListLayoutElements—Help | ArcGIS for Desktop

Hm, that is interesting. I'll keep that in mind.

Re: Python script for mass find and replace of workspace path?

IanMurray — Sat, 11 Dec 2021 13:39:41 GMT

The reason your script wasn't changing anything was because as I noted above the image name was no longer just the file name of the picture, but the full path to the picture. I used the python window in the map document and managed to change your picture element to a site plan jpg on my desktop. Your code would have worked fine, but since the elm.name wasn't what you thought it was, it actually didn't change anything.

If that is the particular path you need to replace on all your mxds, here is some starter script to get you going on editting them all. This should walk through all the .mxd files in a workspace and any .mxds in any folders within that workspace, check for picture elements with the broken file name and fix to your new image. I would take 2 or 3 map documents into a separate folder to test this. Only thing you need to do is change the workspace to the folder where the map documents are.

import arcpy
from arcpy import env

#Change to Directory with you map documents, 
env.workspace = r"YourDirectoryHere"

#Recursively walking workspace
for root, dirs, files in os.walk(env.workspace):
     for f in files:
          if f.endswith(".mxd"):
               mxd = arcpy.mapping.MapDocument(os.path.join(root , f))
               print "Current map being checked is " + mxd.filepath
                    for elem in arcpy.mapping.ListLayoutElements(mxd, "PICTURE_ELEMENT", "*logo")):
                         if elem.name == r"\\Jackson\gis_dept\Data\Images\County-logo-1x1Small.JPG":
                              elem.sourceImage = r"P:\Images\County-logo.JPG"
                    del elem
               del mxd‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Python script for mass find and replace of workspace path?

JaredPilbeam1 — Tue, 24 Jan 2017 21:34:57 GMT

That looks great. But, it's giving me an error. Also, i'm curious why your line 13 is indented? That was giving me an error before i dedented.

Re: Python script for mass find and replace of workspace path?

IanMurray — Tue, 24 Jan 2017 21:50:26 GMT

apologies, it should be filePath, not filepath. I didn't test it, just cobbled it together from the python interpreter and some old code I had to check for broken data links.

I added the print statements so you knew what was going on with the program. They are also a great way to see what values are actually being used in python and make sure they are doing what you want. Thats how I figured out your elm.name value earlier wasn't the right one for your code to work.

Not sure about line 13, whitespace can be annoying to work with, especially when cutting/pasting code, like I said I cobbled it together quickly.

Re: Python script for mass find and replace of workspace path?

JaredPilbeam1 — Sat, 11 Dec 2021 13:39:44 GMT

Ok, thanks. I fixed 'mxd.filePath'. The script ran with no errors and printed out "Current map being checked is..." for all the documents in the workspace, but with no results.

Above in your test script you used \\Jackson\gis_dept\Data\Images\County-logo-1x1Small.JPG as your elem.name path and it worked. Does this mean it doesn't have to be a working file path? Of course, since that's a path to our directory here you can't access it. But, at the same time the Jackson network was discontinued a few days ago (hence the broken links). Or is the script recognizing that file path as the element name?

This is the message from Windows when I search for it:

So, I'm wondering if that's causing a problem in my script?

Besides that I set the workspace to a working folder. And I used the whole path, as you did, for the elem.name:

import arcpy
import os
from arcpy import env

# change to directory with your map documents
env.workspace = r'R:\AtlasMaps\ATLAS_MAPS_17\New folder'

# recursively walking workspace
for root, dirs, files in os.walk(env.workspace):
     for f in files:
          if f.endswith(".mxd"):
               mxd = arcpy.mapping.MapDocument(os.path.join(root , f))
               print "Current map being checked is " + mxd.filePath
               for elem in arcpy.mapping.ListLayoutElements(mxd, "PICTURE_ELEMENT", "*logo"):
                   if elem.name == r"\\Jackson\gis_dept\Data\Images\County-logo-1x1Small.JPG":
                       elem.sourceImage = r"P:\Images\County-logo.JPG"

del elem
del mxd‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Python script for mass find and replace of workspace path?

IanMurray — Wed, 25 Jan 2017 17:34:39 GMT

The filepath doesn't have to exist, it just had to be the value of the element name in the map drawing. If you looked at the image I posted earlier using the mxd you posted, I used print statements to determine what the actual name of the picture element was in the map drawing.

Also I noticed another bug that might be causing the issue, line 13 needs an asterisk on the other side of logo, so it should be:

for elem in arcpy.mapping.ListLayoutElements(mxd, "PICTURE_ELEMENT", "*logo*"):

If it doesn't have a wildcard on both sides, it might not be pulling any picture elements at all, so it has no elements to even check(Since *logo would only find names ending with logo, instead of *logo* which looks to find names that contain the character string logo). If that doesn't work, could you add a few print statements(even if it does work it helps show what is going on). After line 13, could you add:

print "current element name is " + elem.name; print "current element source is " + elem.sourceImage

(same indent as line 14 currently).

Then after the current line 15 add:

print "new element source is " + elem.sourceImage

(same indent as line 15 currently)

Re: Python script for mass find and replace of workspace path?

JaredPilbeam1 — Sat, 11 Dec 2021 13:39:47 GMT

Alright, great. At some point during testing I did try the asterisk on both sides of logo.

I added the print statements and the asterisk. It ran with no errors, but no results to the map document again. I double checked and both instances of the JPG in the maps of the workspace are exactly: \\Jackson\gis_dept\Data\Images\County-logo-1x1Small.JPG.

import arcpy
import os
from arcpy import env

# change to directory with your map documents
env.workspace = r'R:\AtlasMaps\ATLAS_MAPS_17\New folder'

# recursively walking workspace
for root, dirs, files, in os.walk(env.workspace):
    for f in files:
        if f.endswith(".mxd"):
            mxd = arcpy.mapping.MapDocument(os.path.join(root, f))
            print "current map being checked is " + mxd.filePath
            for elem in arcpy.mapping.ListLayoutElements(mxd, "PICTURE_ELEMENT", "*logo*"):
                print "current element name is " + elem.sourceImage
                if elem.name == r"\\Jackson\gis_dept\Data\Images\County-logo-1x1Small.JPG": #\\Jackson\gis_dept\Data\Images\County-logo-1x1Small.JPG
                    elem.sourceImage = r"P:\Images\County-logo-1x1Small.JPG"
                    print "new element source is " + elem.sourceImage

del elem
del mxd‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

I'm going to try with some different maps that have the have the same broken link to Jackson.

Re: Python script for mass find and replace of workspace path?

IanMurray — Wed, 25 Jan 2017 19:36:47 GMT

Okay, so I'm a bit of an idiot, I didn't bother to have the script actually save the map after the changes were made. According to the the print statements the script is changing the source of your image, which is what we wanted, but the map document is not being saved with the changes. Since I originally did this in ArcMap python window, I didn't need to save the map, I just refreshed the view to fix the changes.

Just add mxd.save() to line 18 with the same indent as line 11 and that should save the changes......