topic Re: replaceDataSource not working in Python Questions

replaceDataSource not working

ChristopherGraff — Wed, 20 Jul 2011 15:50:24 GMT

I'm trying to redirect a raster layer in an mxd from one SDE to another.

the relavent Code:
sdeFile = r"G:\DEPARTMENTS\GIS\Chris_Graff\temp\Aerial.sde"
...
lyr.replaceDataSource(sdeFile, "SDE_WORKSPACE", r"AERIAL.DATALOADER.AP2008", True)

when this runs I don't get any errors. When I open the mxd it changed the workspace to the correct SDE but the raster's dataset_name is still the old name ("STATIC.DATALOADER.AP2008") rather than the one I provided.

Anyone have any ideas? I'm running arcgis 10sp2
thanks,
-chris

Re: replaceDataSource not working

ChristopherGraff — Sat, 11 Dec 2021 09:16:06 GMT

If helpful here's the full code....

import os, arcpy
start = r"G:\DEPARTMENTS\GIS\Chris_Graff\temp\scketchup"
sdeFile = r"G:\DEPARTMENTS\GIS\Chris_Graff\temp\Aerial.sde"

for root, dirs, files in os.walk(start):
    for file in files:
        if file.endswith(".mxd"):
            path = os.path.abspath(os.path.join(root,file))
            mxd = arcpy.mapping.MapDocument(path)
            print path
            for lyr in arcpy.mapping.ListLayers(mxd):
                if lyr.supports("SERVICEPROPERTIES") and lyr.serviceProperties["ServiceType"] == "SDE":
                    if lyr.datasetName.upper() == "STATIC.DATALOADER.AP2008":
                        print "!   " + lyr.datasetName
                        lyr.replaceDataSource(sdeFile, "SDE_WORKSPACE", r"AERIAL.DATALOADER.AP2008", True)
                        print "NEW " + lyr.datasetName
                    else:
                        if lyr.isRasterLayer:
                            print "!!!!" + lyr.datasetName
                        else:
                            print "    " + lyr.datasetName
                else:
                    print "    " + lyr.name
            try:
                mxd.save()
            except IOError as e:
                print "Save failed!"

del mxd, lyr, file, path

Re: replaceDataSource not working

JakeSkinner — Sat, 11 Dec 2021 09:16:09 GMT

You can update the layer name by searching for a keyword and then replacing the text. I was working on a similar script earlier this morning. I added this functionality to it.

import arcpy, os
from arcpy import env
from arcpy import mapping
env.workspace = workspace = arcpy.GetParameterAsText(0)

env.overwriteOutput = True

for (path, dirs, files) in os.walk(workspace):
    for file in files:
        if ".mxd" in file.lower():
            mxd = os.path.join(path, file)
            print mxd + " is being processed"
            mxd = mapping.MapDocument(mxd)
            for df in mapping.ListDataFrames(mxd, "*"):
                for lyr in mapping.ListLayers(mxd, "*", df):
                    lyr.replaceDataSource(arcpy.GetParameterAsText(1), "SDE_WORKSPACE", "")
                    print "Successfully updated data sources"
                    if "vector" in lyr.name:
                        lyr.name = "RASTER.DATALOADER." + lyr.name.split(".")[2]
                        print "Succesfully changed layer name"
            mxd.save()

del mxd

The previous database name was "vector", so the feature classes were listed as "vector.dataloader.parcels". I created and IF statement to find all layer names containing vector and renamed the layers to the new database raster:

 if "vector" in lyr.name:
      lyr.name = "raster.dataloader." + lyr.name.split(".")[2]

Re: replaceDataSource not working

ChristopherGraff — Thu, 21 Jul 2011 12:23:17 GMT

My problem is not with the lyr.name, its the lyr.datasetName. I need to change the datasetName as the lyr.replaceDataSource class is not honoring any string I try to use. Whatever I use lyr.replaceDataSource always returns the original datasetName. I've tried all the below but always the datasetName remains unchanged.

lyr.replaceDataSource(sdeFile, "SDE_WORKSPACE", "AERIAL.DATALOADER.AP2008", True)

lyr.replaceDataSource(sdeFile, "SDE_WORKSPACE", "AP2008", True)

lyr.replaceDataSource(sdeFile, "SDE_WORKSPACE", "", True)

lyr.replaceDataSource(sdeFile, "SDE_WORKSPACE", "newName", True)

Since the the datasetName contains a referance to the old database, even though the SDE data source changes, the layer in the mxd is broken. Does lyr.replaceDataSource not honor raster datsets in SDE?

Re: replaceDataSource not working

JakeSkinner — Thu, 21 Jul 2011 15:34:26 GMT

Looks like this may be a bug for the replaceDataSource method and raster datasets. However, this works with Mosaic Datasets. Have you considered migrating your large raster mosaics to Mosaic Datasets? This is a much more efficient way to manage your imagery at ArcGIS 10.

The mosaic dataset makes it easy for you to manage, search, and discover imagery in collections of any size. The mosaic dataset catalogs image and raster data sources, stores detailed metadata, and defines how imagery should be transformed into different products on the fly without the need
to preprocess. The mosaic dataset includes:

�?� Dynamic mosaicking for efficient handling of overlapping imagery

�?� On-the-fly processing to create multiple products from a single source

�?� Management and accessibility to metadata

�?� Integration for multiple disparate image sensors and formats

�?� Streamlined data maintenance and updates of new imagery

The mosaic dataset also saves time and storage space. The mosaic dataset simply reads the images from their native file locations and format. Since it reads the data from their native file locations it is extremely fast to create mosaic datasets. Loading large rasters into SDE that took hours will only take seconds now and you will no longer have to allocated the storage space in your SDE database to accomodate these large rasters.

Re: replaceDataSource not working

ChristopherGraff — Thu, 21 Jul 2011 17:39:35 GMT

Do I need to submit this as a bug or have you?

Our organization only deals with new imagery every 3 years and we maintain tight controls ending up with standard projections, pixel sizes, etc, so many of the traditional uses of raster catalogs haven't really applied to us. Having a fairly robust enterprise desktop, server and mobile users we've enjoyed the benifet of having our aerials pre-processed into SDE. In 20years we've only now ever needed to move one.

Is it still the case that I/O performace is faster from an SDE raster dataset than a catalog (particularly if the imagery is accessed through files)?

Thanks so much for your help,
-chris

Re: replaceDataSource not working

JakeSkinner — Thu, 21 Jul 2011 18:38:15 GMT

I have not submitted this as a bug. Feel free to do so if you'd like, I'm not sure that I will have the time.

Mosaic Datasets are a hybrid of raster datasets and raster catalogs. They render extremely fast due to the mosaic dataset's overviews. Mosaic Datasets render just as fast as raster datasets, plus you receive the additional benefits that the mosaic dataset has to offer.

I would recommend taking a look at the following video to see how great and easy to use Mosaic Datasets are.

Re: replaceDataSource not working

DamianSchroder — Fri, 22 Jul 2011 23:41:47 GMT

I would like to add that I am experiencing this exact same problem and it is not just a raster dataset issue. It appears to fail with spatial views. It changes everything except the Feature Class when you view the source in ArcMap. This code is successful on normal feature classes and anno. (ArcGIS 10 sp2 build 3200)

import arcpy
import string
mxd = arcpy.mapping.MapDocument(r"E:\source_control\MXDs\Map_Services\General4.mxd")
sdePath = r"C:\Users\damian\AppData\Roaming\ESRI\Desktop10.0\ArcCatalog\atlas_spatial.sde"

for lyr in arcpy.mapping.ListLayers(mxd):
    if (lyr.supports("DATASOURCE") and lyr.supports("datasetName")):
        layerName = r"ATLAS_SPATIAL." + lyr.datasetName.split('.')[1]
        lyr.replaceDataSource(sdePath,"SDE_WORKSPACE",layerName,False)
mxd.saveACopy(r"E:\source_control\MXDs\Map_Services\General10.mxd")

del mxd

Re: replaceDataSource not working

MichaelVolz — Sat, 11 Dec 2021 09:16:11 GMT

Jake:

What would be the replaceDataSource syntax if one was converting from a raster in SDE to a mosaic dataset in a file geodatabase?

This is the syntax I am using, which follows the ArcGIS Help, but I am getting an error message:

elif layr.datasetName == "ORTHO2005.ORTHOS2005":
                print("The dataset name in Resource_mxd is " + layr.datasetName)
                layr.replaceDataSource(r"\\gis\GIS\Work\arcmap\mosaic_2010.gdb", "FILEGDB_WORKSPACE", "orthos_2010")

In this code, "orthos_2010" is the mosaic dataset that contains 2010 Orthophotos. I am not specifying where the data originates in SDE, other than making sure that the focus layer to be converted has a datasetName = ORTHO2005.ORTHOS2005, as I did not need to do this for vector datasets that I resourced from SDE to a file geodatabase.

Your assistance is greatly appreciated. Thanks.

Re: replaceDataSource not working

JakeSkinner — Mon, 13 Feb 2012 16:13:13 GMT

Hi Michael,

You won't be able to change data types (i.e. Raster Dataset to Mosaic Dataset). You can only change the data source if the data type matches (i.e. Raster Dataset to Raster Dataset or Mosaic Dataset to Mosaic Dataset).

Re: replaceDataSource not working

MichaelVolz — Mon, 13 Feb 2012 16:24:39 GMT

Jake:

In the original post, the posting person was trying to convert rasters from one SDE database to another. In your response, you said it would be better to use a mosaic dataset to store the raster data. As such, I would want to convert raster data from SDE to a mosaic dataset for layers that I have saved in many mxds. I thought you were indicating that this can be done, but your last post said this is not possible with replaceDataSource.

Is there any other python method that could be used to make this conversion? Your feedback is greatly appreciated. Thanks.

Re: replaceDataSource not working

JakeSkinner — Mon, 13 Feb 2012 16:52:51 GMT

In the previous post the user ran into an issue when attempting to update an SDE Raster Dataset in an MXD with another SDE Raster Dataset. I recommended to use Mosaic Datasets rather than SDE Raster Datasets for better data management and performance.

At the moment, updating the MXDs with Mosaic Datasets would be best accomplished manually. You could create a layer file of the mosaic dataset and then add the layer file using the 'addlayer' method, but you don't have many placement options when adding the layer file.

Re: replaceDataSource not working

MichaelVolz — Mon, 13 Feb 2012 17:11:37 GMT

Jake:

Do you know if it will be a capability of python scripting in ArcGIS v10.1 to convert SDE raster images to a mosaic dataset? I would need to perform this task on 10s of thousands of layers so a manual approach is extremely impractical. Thanks.

Re: replaceDataSource not working

JakeSkinner — Mon, 13 Feb 2012 17:26:54 GMT

I cannot say for sure if this will be available in 10.1. Your best bet would to be to remove the raster layers in the MXDs by iterating through each layer in each MXD, and then adding a layer file of the mosaic dataset to the MXD. I am assuming that the raster imagery is used as background data, so specifying "BOTTOM" for the position with the 'addlayer' method should work in most cases.

Re: replaceDataSource not working

MichaelVolz — Mon, 13 Feb 2012 17:42:10 GMT

Jake:

Besides a Mosaic Dataset, what do you think would be the best way to store raster images outside of SDE to get excellent performance in ArcMap, where I would be able to use the replaceDataSource in python to get an automated solution that can be performed during non-business hours?

In response to your last post, many of the the current SDE image layers can be found in group layers and not necessarily at the bottom of the TOC, so your approach would by no means be the universal solution for my organization. In the past I had resourced these same images from IMS to SDE using a programmatic approach in VBA, but group layers also presented a problem. I had to resource these layers manually after getting an inventory of how often this occurred on my network. I was hoping the replaceDataSource in python for ArcMap v10.0 would make this task easier, but it appears that this option was not accounted for at this point in python.

Re: replaceDataSource not working

JeffBarrette — Tue, 14 Feb 2012 13:13:43 GMT

To address the issue of replacing an SDE raster image with a raster mosaic, have you tried arcpy.mapping.UpdateLayer with symbology_only=False? Author a layer file pointing to your raster mosaic and then find and update your SDE raster images found within your MXDs. UpdateLayer has the ability to completely swap our entire layers (including data type).

Jeff

Re: replaceDataSource not working

MichaelVolz — Tue, 14 Feb 2012 14:40:52 GMT

Thanks Jeff.

The UpdateLayer method with the False parameter for symbology appears to do exactly what I need. Preliminary testing worked perfectly and now I need to test with more complex mxds where there are multiple dataframes and layers in a Group Layer that need to be replaced.

Re: replaceDataSource not working

MichaelVolz — Wed, 30 Jan 2013 19:33:50 GMT

This is still an issue.