topic Re: gp.Merge_management using list in python in Geoprocessing Questions

gp.Merge_management using list in python

JamesHood — Sat, 11 Dec 2021 17:59:44 GMT

Hello again, colleagues,

I have a problem executing gp.merge_management or gp.merge tool within a script.

My script needs to merge a varying number of tables together. In this instance there are 7, though this number can and will change. In order to accomplish this I have used various different scripting techniques, all of which have failed.

I feel like I am so close and it should work, but there is something fishy stopping it, that I just can�??t see.

The error I continually get is: #ERROR 000582: Error occurred during execution.

Error code: 000582: Error occurred during execution.
Description: An error occurred during the tool's validation or execution. No specific messages were available.
The validation process checks that a tool's set of parameter values are of the expected number, data type, and value.
Solution: Check that a tool's required parameter values are specified and are the correct data type. If the parameters are entered correctly, it may be possible the execute routine for the tool is invalid. Have the developer of the custom tool check the Execute implementation.

See: http://webhelp.esri.com/arcgisdesktop/9.3/index.cfm?tocVisable=1&ID=-1&TopicName=tool%20errors%20and%20warnings:576-600&pid=1

Please see my attached code:


Stage2Tbl = r"E:\\FMP_Reports_Tool\\tempWS\\Stage2Tbl"

try:

    mergeString = r""
    for tbl in mlist:
        mergeString = mergeString + tempWS + tbl + ";"        

    insertStr = mergeString.rstrip(";")
    gp.Merge_management(str(insertStr), Stage2Tbl, "")    

except:
    # Get the geoprocessing error messages
    msgs = gp.GetMessage(0)
    msgs += gp.GetMessages(2)
    # Return gp error messages for use with a script tool
    gp.AddError(msgs)
    # Print gp error messages for use in Python/PythonWin
    print msgs

The variable mlist is a list that contains the output names of tables that were created into the temp workspace earlier in the script. This list is then used to select those same tables, and merge them back together.

I create the mergeString variable from the mlist , to be used in the actual gp.merge tool.

I was concerned that there may be a problem reading the workspace strings in the mergeString, so I added the �??r�?� to create a raw string. This had no effect.

I read previously in another thread that I might need to convert my string, to string within the actual tool. I added this but had again the same result.

I used my string variable insertStr in the merge tool of a separate script and it worked just fine! So it just will not work here.

Please, are there any suggestions on anything further I should try or other methods which might work in this circumstance?

I am using ArcInfo 9.3 and python 2.5 if that helps.

Thanks!

- James

Ps. This is worth at least 3 Stars on the old forums...

Re: gp.Merge_management using list in python

BradPosthumus — Wed, 18 Aug 2010 20:10:49 GMT

In this line:

mergeString = mergeString + tempWS + tbl + ";"

...I'm assuming "tbl" is a table in the workspace "tempWS", and should therefore read:

mergeString = mergeString + tempWS + "\\" + tbl + ";"

Edit: (unless you have the backslash at the end of "tempWS" ...what's the value of your variable "tempWS"?)

Re: gp.Merge_management using list in python

KarlOrmer — Sat, 11 Dec 2021 17:59:47 GMT

Are you sure that this evil blank except statement (normally one says that it is best practice to catch only specific Exceptions - never ALL possible exceptions as this may lead to errors which are hard to track down. But in the ArcGIS world this seems to be...well..different. At least for many 9.3 examples) does not catch some python error which you are not aware of?

And just a tip for file name/path operations:
You can use forward slashes instead of backwards slashes which may be less error-prone as you don't have to fight with escape sequences

path = 'c:/test'

instead of

path = 'c:\\test'

For path operations you might want to try os.path.join:

import os.path
base_dir = 'c:/test/shp'
shape_name = 'test.shp'
shape_path = os.path.join(base_dir, shape_name)

This is less error-prone as you don't need to take care of missing slashes at the end of base_dir etc.

I don't know if this solves your problem, but I will have to do the exact same task tomorrow so maybe I can then give you some more useful information.

Re: gp.Merge_management using list in python

KarlOrmer — Sat, 11 Dec 2021 17:59:50 GMT

It is hard to track your exact problem down as I don't know the content of your variables. But the following might help you as it is how I have just solved my problem.

import arcgisscripting, os.path

gp = arcgisscripting.create(9.3)
input_tables = [u'table_1', u'table_2', u'table_3']
in_dir = u'c:/path_to_my_tables'
out_dir = u'c:/whatever'

# let's use absolute file paths (c:/path_to_my_tables/table_1) and let's use an
# ESRI value table instead of building an insert string ('c:/.../table1;c:/..../table2')
# btw, why does the tool not accept a python list as input argument?
val_tab = gp.CreateObject("ValueTable")
for table in input_tables:
    val_tab.AddRow(os.path.join(in_dir, table))
    
merged_table = os.path.join(out_dir, 'merged_table')
gp.Merge_management(val_tab, merged_table, '')

Re: gp.Merge_management using list in python

RDHarles — Sat, 11 Dec 2021 17:59:52 GMT

Here's how I do it.
Note: I'm printing the 'path' variable to make sure everything looks right during testing.

# Start a blank list for appending
list = []        

for tbl in mlist:          
                
    # Create the full path
    path = tempWS+tbl
    print path
             
    # Append all the paths together into one big list       
    filepath = list.append(path)

## Create the proper format for the Merge tool
# Separate each item in the list by a ';' and put "" around the whole thing.
mergedlist = '"'+';'.join(list)+'"'

try:
    print "Merging to get Output.dbf..."    
    gp.merge_management(mergedlist, "Ouptput.dbf")
    print gp.GetMessages()
except:
    print "\n *** ERROR: Output.dbf failed to merge *** \n"
    print gp.GetMessages()

Re: gp.Merge_management using list in python

JamesHood — Thu, 19 Aug 2010 13:16:19 GMT

bposthumus:
My output path contained all the correct and required slashes, sorry that I did not post it:

tempWS = "E:\\FMP_Reports_Tool\\tempWS.mdb\\"

_garret_:

I originally had used the Value table method but it kept crashing so I switched to creating the string.

I followed a number of your ideas, changing all my paths from \\ to /, and using the os.path.join (tempws, tableName)... which didn't work either!

Just in a round about sort of trial and error, frustrating , smash my face on the keyboard, way, I started switching some of my inputs back to the way I had before I tried your methods...

Some where in the middle it worked!

So I have a bit of a spaghetti factory code, all mixed up using various different methods to create the out puts and it works and I don't want to touch a thing. So dirty it will stay.

I am still not sure why it wouldn't work though I suspect it possibly could have been the cursor used to create the list or value table, holding a lock on the variable? I dont know.

Any way thanks again everybody for the suggestions. Today I am a Happy Gisser!

Re: gp.Merge_management using list in python

ArcGISUser — Sat, 11 Dec 2021 17:59:55 GMT

Here's how I do it.
Note: I'm printing the 'path' variable to make sure everything looks right during testing.

# Start a blank list for appending
list = []        

for tbl in mlist:          
                
    # Create the full path
    path = tempWS+tbl
    print path
             
    # Append all the paths together into one big list       
    filepath = list.append(path)

## Create the proper format for the Merge tool
# Separate each item in the list by a ';' and put "" around the whole thing.
mergedlist = '"'+';'.join(list)+'"'

try:
    print "Merging to get Output.dbf..."    
    gp.merge_management(mergedlist, "Ouptput.dbf")
    print gp.GetMessages()
except:
    print "\n *** ERROR: Output.dbf failed to merge *** \n"
    print gp.GetMessages()

This snippet of code worked for my merge problem. Thank you very much rdharles. This is how I would do it too. 😄