GP Service Multiple Processes from MXD

989
12
03-09-2017 08:43 AM
LloydBronn
Occasional Contributor II

I have a GP service that generates a PDF report for a country based on a click event. This report includes charts generated on the fly with matplotlib, sourced from rasters. It also includes symbolized raster images exported from an MXD. While testing, we've found that if two users try to run the service at the same time, it errors out for both users. The browser console.log displays: "unable to complete operation."  The ArcGIS server logs displays: " MapDocObject: Unable to save. Check to make sure you have write access to the specified file and that there is enough space on the storage device to hold your document. Failed to execute." We don't get these errors when we run the process one at a time.I'm assuming that the MXD can only be accessed by one user at a time. Is there a way for the server to queue users if more than one are trying to access this process at the same time? Would saving multiple numbered copies of the MXD solve the problem?

0 Kudos
12 Replies
NeilAyres
MVP Frequent Contributor

Certainly the GP service can be used by multiple users. Each time the GP service executes a new job is created on the server. So no conflict there.

What do you mean by the same "mxd"? Are users actually opening the same file on a network somewhere?

0 Kudos
LloydBronn
Occasional Contributor II

The MXD and all shapefiles and rasters are housed locally on the server machine. The user clicks on a country on the web map. That country name is passed as an input parameter to the GP Python script tool. The Python script accesses the MXD, creates background outline mask layers for the country and exports symbolized raster images masked to the country. The background layers stay generic, but the script determines what the last country source was, updates the layers to the new country and saves the MXD. I think the problem is that the MXD can't be editied and saved with two or more countries at once. I'm thinking the solution is to create multiple copies of the MXD and have the script determine whether the first MXD is in use, if so, move to the next one. I just need to figure out the best method for that in Python. 

0 Kudos
KevinHibma
Esri Regular Contributor

Without seeing your code, I assume you're doing some sort of .save() on the MXD object. I could imagine that 2 people trying to work off and save the MXD would fail. Two people (2 instances) simply reading the same MXD should be fine.

If you're in fact doing a save, change it to a .saveAs() and write the output of this operation to your scratch.:

myNewMXD = os.path.join(arcpy.env.scratchFolder, "newMXD.mxd")
mxdObject.saveAs(myNewMXD)
#... carry on processing/exporting from this new MXD.‍‍‍‍‍‍‍‍‍‍‍‍
newMXDobj = arcpy.mapping.MapDocument(myNewMXD)
LloydBronn
Occasional Contributor II

Two people can't use the same MXD, because they will be creating different background layers based on the countries they've chosen. How about this?

try: 
 mxd = arcpy.mapping.MapDocument("C:\\Website_Test\\Report_Tests\\MXDs\\Analysis_For_Reports_1.mxd")
except RuntimeError:
 for num in range(2,6):
 mxd = "C:\\Website_Test\\Report_Tests\\MXDs\\Analysis_For_Reports_" + num + ".mxd"‍‍‍‍‍
0 Kudos
KevinHibma
Esri Regular Contributor

While that code is fine.... and nobody is stopping you from doing that....  I really dont think you need to go that route. (personally I wouldnt, but I understand at the end of the day we all just want something that works)

I can open 2 instances of ArcMap and reference the same MXD object in each.  I only receive a runtime error when I do a save (that matches your original error message). Having 1-n of instances of a GP Service executing against 1 MXD is the same principal. Each instance gets its own space (folder) to execute against. This folder is unique and other executing instances have zero knowledge of it -- thus no write problems. This is why I suggest doing a saveAs. All instances make use of 1 MXD (your source), do the modifications to it in memory (meaning just acting on the mxd object) and then do the final saveAs to persist this information and then export.

So like I said to begin with, if you really want to go down the path of multiple MXDs using a try/export....I wont stop you. It's just more heavy handed and will be more of a maintenance problem if you want to do updates later.

OHH EDIT -- ya, with the multiple MXD approach, if you're letting the publishing process copy the data, this will fail. The publishing process doesnt know to traverse code to find all MXDs and copy them. You'd need to create a variable that references the folder those MXDs exist in. That way the publishing process copies everything inside the folder (all mxds). Alternative if you set a data store folder to this location, nothing will need to be copied and it should work.

0 Kudos
LloydBronn
Occasional Contributor II

(I posted this reply before I saw your edit)

I see what you're saying, but I'm just not sure that it would actually work. The sources in the background layers keep changing based on the current chosen country. If two people pick Brazil and Iceland at the same time, I can't see how it wouldn't fail. Maybe I'm wrong. Here is the definition that updates the background layers. 

def update_background_layers(mxd,df,input_country):
    
    background_layers_list = ["cities", "country", "oceanout","whiteout", "six_largest_cities"]

    for background_layer in background_layers_list:
        
        for lyr in arcpy.mapping.ListLayers(mxd):
            if lyr.supports("DATASOURCE"):
                
                if background_layer == str(lyr.name):
                    old_source = os.path.dirname(lyr.dataSource)
                    new_source = "C:\\Website_Test\\Report_Tests\\Background_Layers\\" + input_country
             
                    lyr.findAndReplaceWorkspacePath(old_source, new_source, False)

    mxd.save()
    return()‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍
0 Kudos
KevinHibma
Esri Regular Contributor

I'm not setup to quickly test, but I'm 99% certain line 16 (save) is the problem. User A choosing ICELAND executes and has their own MXDObject. User B chooses Brazil and again they have their own MXDObject. These two objects exist in their own process (arcsoc.exe) independent from each other. It is only at that time of line 16: mxd.save() do worlds collide and one process tries to write over the other process.

So, if you were to slightly modify your code:

def update_background_layers(mxd,df,input_country):
    
    background_layers_list = ["cities", "country", "oceanout","whiteout", "six_largest_cities"]

    for background_layer in background_layers_list:
        
        for lyr in arcpy.mapping.ListLayers(mxd):
            if lyr.supports("DATASOURCE"):
                
                if background_layer == str(lyr.name):
                    old_source = os.path.dirname(lyr.dataSource)
                    new_source = "C:\\Website_Test\\Report_Tests\\Background_Layers\\" + input_country
             
                    lyr.findAndReplaceWorkspacePath(old_source, new_source, False)

    
    #mxd.save()
    tempMXD = os.path.join(arcpy.env.scratchFolder, "mxd2Export.mxd")
    mxd.saveACopy(tempMXD)
    # re-load that MXD up as a true MXD object, not just the path
    # because the next function wants the object
    newMXD = arcpy.mapping.MapDocument(tempMXD)
    return newMXD

# You'll need to change whatever called "update_background_layers" to accept the 
# new MXD bein returned. As you have it now, you're kind of relying on a "global" MXD
# so this function would spit out the new MXD and your code would have to be updated to
# handle that.
0 Kudos
LloydBronn
Occasional Contributor II

OK, so tempMXD would be the original Analysis_For_Reports.mxd,  and newMXD would be a copy of that?

0 Kudos
KevinHibma
Esri Regular Contributor

Not exactly. They're actually the same thing.

One is just a path on disk (tempMXD) and the other is an arcpy.mapping-MapDocument (newMXD).

You could pass the tempMXD out of the function and handle this line somewhere else if you want:

newMXD = arcpy.mapping.MapDocument(tempMXD)
0 Kudos