Select to view content in your preferred language

How to setup BATCH process for my Script.

7354
29
Jump to solution
01-11-2013 10:07 AM
ChristopherClark1
Deactivated User
Ok, I have the following python script which migrates and maps fields and the values of those fields into a target Feature Class.
# Import arcpy module import arcpy  arcpy.env.workspace = "G:\ChrisGIS\WDFW_NWIFC_EVENT_17100101.gdb" arcpy.env.overwriteOutput = "True"                def GetFieldMappings(inputFC, targetFC, dico):     field_mappings = arcpy.FieldMappings()     field_mappings.addTable(inputFC)      for input, output in dico.iteritems():         field_map = arcpy.FieldMap()         field_map.addInputField(inputFC, input)                 field = field_map.outputField                 field.name = output         field_map.outputField = field         field_mappings.addFieldMap(field_map)         del field, field_map      return field_mappings  #Variables inputFC = "WDFW_CHFA_2HEM_evt_1" targetFC = "WDFW_CHFA_2HEM_evt_olay_1"  dico = {'Permanent_Identifier': 'Permanent_ID',                 'STRM_NAME':        'LLID_STRM_NAME',     'DATE_SRVY': 'SOURCE_DATE',     'SPPCODE': 'SPECRCODE',     'SRC_AGENCY': 'SOURCE_AGENCY',     'SRC_WHO': 'SOURCE_WHO'}      Mapper = GetFieldMappings(inputFC, targetFC, dico) # Process: Append arcpy.Append_management(arcpy.env.workspace+"\\"+inputFC, arcpy.env.workspace+"\\"+targetFC,"NO_TEST",Mapper)

I have several feature classes to run this script on. AN exmaple of the Feature Classes that I need to run my script on are :

WDFW_CCT_2HEM_evt_olay
WDFW_CHFA_2HEM_evt_olay
WDFW_CHMF_2HEM_evt_olay
WDFW_CHSP_2HEM_evt_olay
WDFW_CHSU_2HEM_evt_olay
WDFW_COHO_2HEM_evt_olay
WDFW_DBT_2HEM_evt_olay
WDFW_KOK_2HEM_evt_olay
WDFW_RBT_2HEM_evt_olay
WDFW_SOCK_2HEM_evt_olay

These feature classes are represented in several geodatabases (see image)
[ATTACH=CONFIG]20670[/ATTACH]
I could simply go to each individual geodatabase and run the script for each featureclass in that geodatabase. This however seems redundant and I would like to figure out how to batch my scrip or automate this process. All geodatabases are located in one working directory.

I appreciate any help or advice!
Tags (2)
0 Kudos
29 Replies
ChristopherClark1
Deactivated User
# Import arcpy module
import arcpy
arcpy.env.overwriteOutput = True
from os import path as p
        
def GetFieldMappings(inputFC, targetFC, dico):
    field_mappings = arcpy.FieldMappings()
    field_mappings.addTable(inputFC)

    for input, output in dico.iteritems():
        field_map = arcpy.FieldMap()
        field_map.addInputField(inputFC, input)        
        field = field_map.outputField        
        field.name = output
        field_map.outputField = field
        field_mappings.addFieldMap(field_map)
        del field, field_map

    return field_mappings

#Variables
dico = {'Permanent_Identifier': 'Permanent_ID',
                'STRM_NAME':        'LLID_STRM_NAME',
    'DATE_SRVY': 'SOURCE_DATE',
    'SPPCODE': 'SPECRCODE',
    'SRC_AGENCY': 'SOURCE_AGENCY',
    'SRC_WHO': 'SOURCE_WHO'}

arcpy.env.workspace = r"G:\ChrisGIS\COMBO_fishdist_events"
for ws in arcpy.ListWorkspaces("*", "FileGDB"):
 listFCS = arcpy.ListTables("WDFW*evt")
 arcpy.env.workspace = ws
 for inputFC in listFCS:
  targetFC = inputFC +"_olay"   
  Mapper = GetFieldMappings(inputFC, targetFC, dico)
  # Process: Append
  arcpy.Append_management(p.join(ws,inputFC), p.join(ws,targetFC),"NO_TEST",Mapper)
  print 'Appended "{0}" to "{1}" in {2}'.format(inputFC, targetFC, p.basename(ws))


After running this script I am again getting this error :

Runtime error <class 'arcgisscripting.ExecuteError'>: Failed to execute. Parameters are not valid. ERROR 000732: Target Dataset: Dataset G:\ChrisGIS\COMBO_fishdist_events\WDFW_NWIFC_EVENT_17100102.gdb\WDFW_CCT_2HEM_evt_olay does not exist or is not supported Failed to execute (Append).

I feel like I am close, but I do not understand why WDFW_NWIFC_EVENT_17100102.gdb\WDFW_CCT_2HEM_evt_olay cannot be found when it is supposed to be created. This is also the 2nd geodatabase in my list, so it seems like it goes through WDFW_NWIFC_EVENT_17100101 without generating any errors.....
0 Kudos
by Anonymous User
Not applicable
# Import arcpy module
import arcpy
arcpy.env.overwriteOutput = True
from os import path as p
        
def GetFieldMappings(inputFC, targetFC, dico):
    field_mappings = arcpy.FieldMappings()
    field_mappings.addTable(inputFC)

    for input, output in dico.iteritems():
        field_map = arcpy.FieldMap()
        field_map.addInputField(inputFC, input)        
        field = field_map.outputField        
        field.name = output
        field_map.outputField = field
        field_mappings.addFieldMap(field_map)
        del field, field_map

    return field_mappings

#Variables
dico = {'Permanent_Identifier': 'Permanent_ID',
                'STRM_NAME':        'LLID_STRM_NAME',
    'DATE_SRVY': 'SOURCE_DATE',
    'SPPCODE': 'SPECRCODE',
    'SRC_AGENCY': 'SOURCE_AGENCY',
    'SRC_WHO': 'SOURCE_WHO'}

arcpy.env.workspace = r"G:\ChrisGIS\COMBO_fishdist_events"
for ws in arcpy.ListWorkspaces("*", "FileGDB"):
 listFCS = arcpy.ListTables("WDFW*evt")
 arcpy.env.workspace = ws
 for inputFC in listFCS:
  targetFC = inputFC +"_olay"   
  Mapper = GetFieldMappings(inputFC, targetFC, dico)
  # Process: Append
  arcpy.Append_management(p.join(ws,inputFC), p.join(ws,targetFC),"NO_TEST",Mapper)
  print 'Appended "{0}" to "{1}" in {2}'.format(inputFC, targetFC, p.basename(ws))


After running this script I am again getting this error : 

Runtime error <class 'arcgisscripting.ExecuteError'>: Failed to execute. Parameters are not valid. ERROR 000732: Target Dataset: Dataset G:\ChrisGIS\COMBO_fishdist_events\WDFW_NWIFC_EVENT_17100102.gdb\WDFW_CCT_2HEM_evt_olay does not exist or is not supported Failed to execute (Append).

I feel like I am close, but I do not understand why WDFW_NWIFC_EVENT_17100102.gdb\WDFW_CCT_2HEM_evt_olay cannot be found when it is supposed to be created. This is also the 2nd geodatabase in my list, so it seems like it goes through WDFW_NWIFC_EVENT_17100101 without generating any errors.....


Look at my last post about creating the table before appending.
0 Kudos
ChristopherClark1
Deactivated User
Okay, I added the create table and here are some results

Appended "WDFW_CCT_2HEM_evt" to "WDFW_CCT_2HEM_evt_olay" in WDFW_NWIFC_EVENT_17100102.gdb
Appended "WDFW_CHFA_2HEM_evt" to "WDFW_CHFA_2HEM_evt_olay" in WDFW_NWIFC_EVENT_17100102.gdb
Appended "WDFW_CHMF_2HEM_evt" to "WDFW_CHMF_2HEM_evt_olay" in WDFW_NWIFC_EVENT_17100102.gdb
Appended "WDFW_CHSP_2HEM_evt" to "WDFW_CHSP_2HEM_evt_olay" in WDFW_NWIFC_EVENT_17100102.gdb
Runtime error <type 'exceptions.RuntimeError'>: FieldMappings: Error in adding table to field mappings

How can this error come to pass after a few tables have already been appended?
After I got these results I went into ArcCatalog and check the geodatabase it was working in and there were no _olay tables. I am assuming this is because the script hit the error before it finished?
0 Kudos
by Anonymous User
Not applicable
Okay, I added the create table and here are some results 

Appended "WDFW_CCT_2HEM_evt" to "WDFW_CCT_2HEM_evt_olay" in WDFW_NWIFC_EVENT_17100102.gdb
Appended "WDFW_CHFA_2HEM_evt" to "WDFW_CHFA_2HEM_evt_olay" in WDFW_NWIFC_EVENT_17100102.gdb
Appended "WDFW_CHMF_2HEM_evt" to "WDFW_CHMF_2HEM_evt_olay" in WDFW_NWIFC_EVENT_17100102.gdb
Appended "WDFW_CHSP_2HEM_evt" to "WDFW_CHSP_2HEM_evt_olay" in WDFW_NWIFC_EVENT_17100102.gdb

Runtime error <type 'exceptions.RuntimeError'>: FieldMappings: Error in adding table to field mappings

How can this error come to pass after a few tables have already been appended? 
After I got these results I went into ArcCatalog and check the geodatabase it was working in and there were no _olay tables. I am assuming this is because the script hit the error before it finished?


Hmmm...I am not sure, that is a good question! It looks like the code is correct now since it worked on several tables before crashing. Do the other olay tables look okay? Maybe on the table where it crashed on does not have all the same fields in your 'dico' variable? Do you know which table it crashed on? If so, I would look at that table and see if there are differences between that one and the others.

If this is not the case perhaps my only other idea is maybe a schema lock?
0 Kudos
ChristopherClark1
Deactivated User
The olay tables that get created only have Object ID field. Nothing else is copying over. Now I am really confused... seems as if the append is not really appending???
0 Kudos
by Anonymous User
Not applicable
Maybe it would be better to use Merge instead, it also supports field mappings.  I also removed the targetFC parameter from the GetFieldMappings() function.  Try this:

# Import arcpy module
import arcpy
arcpy.env.overwriteOutput = True
from os import path as p
        
def GetFieldMappings(inputFC, dico):
    field_mappings = arcpy.FieldMappings()
    field_mappings.addTable(inputFC)

    for input, output in dico.iteritems():
        field_map = arcpy.FieldMap()
        field_map.addInputField(inputFC, input)        
        field = field_map.outputField        
        field.name = output
        field_map.outputField = field
        field_mappings.addFieldMap(field_map)
        del field, field_map

    return field_mappings

#Variables
dico = {'Permanent_Identifier': 'Permanent_ID',
                'STRM_NAME':        'LLID_STRM_NAME',
    'DATE_SRVY': 'SOURCE_DATE',
    'SPPCODE': 'SPECRCODE',
    'SRC_AGENCY': 'SOURCE_AGENCY',
    'SRC_WHO': 'SOURCE_WHO'}

arcpy.env.workspace = r"G:\ChrisGIS\COMBO_fishdist_events"
for ws in arcpy.ListWorkspaces("*", "FileGDB"):
    arcpy.env.workspace = ws
    listFCS = arcpy.ListTables("WDFW*evt")
    for inputFC in listFCS:
        outTab = r'in_memory\%s_2' %inputFC
        targetFC = inputFC +"_olay"   
        # Process: Append
        Mapper = GetFieldMappings(inputFC, dico)
        arcpy.CreateTable_management(ws, outTab))
        arcpy.Merge_management([p.join(ws,inputFC),outTab],p.join(ws,targetFC),Mapper)
        print 'Appended "{0}" to "{1}" in {2}'.format(inputFC, targetFC, p.basename(ws))

0 Kudos
ChristopherClark1
Deactivated User
Runtime error 'outTab' not defined.

arcpy.Merge_management([p.join(ws,inputFC),outTab],p.join(ws,targetFC),Mapper)

I tried to change it to outTable, and got the same error.

arcpy.Merge_management([p.join(ws,inputFC),outTable],p.join(ws,targetFC),Mapper)

I also removed outTab all together but then I get this error

Runtime error <class 'arcgisscripting.ExecuteError'>: ERROR 000605: Output WDFW_CCT_2HEM_evt_olay already exists within G:\ChrisGIS\COMBO_fishdist_events\WDFW_NWIFC_EVENT_17100102.gdb. Failed to execute (Merge).
0 Kudos
by Anonymous User
Not applicable
Runtime error 'outTab' not defined. 

arcpy.Merge_management([p.join(ws,inputFC),  outTab],p.join(ws,targetFC),Mapper) 

I tried to change it to outTable, and got the same error.  

arcpy.Merge_management([p.join(ws,inputFC),  outTable],p.join(ws,targetFC),Mapper) 

I also removed outTab all together but then I get this error 

Runtime error <class 'arcgisscripting.ExecuteError'>: ERROR 000605: Output WDFW_CCT_2HEM_evt_olay already exists within G:\ChrisGIS\COMBO_fishdist_events\WDFW_NWIFC_EVENT_17100102.gdb. Failed to execute (Merge).


did you set up this variable in the 'in_memory' workspace?

outTab = r'in_memory\%s_2' %inputFC
0 Kudos
ChristopherClark1
Deactivated User
I had missed that variable. Once I add it I get another error :

Runtime error <class 'arcgisscripting.ExecuteError'>: Failed to execute. Parameters are not valid. ERROR 000732: Input Datasets: Dataset G:\ChrisGIS\COMBO_fishdist_events\WDFW_NWIFC_EVENT_17100102.gdb\WDFW_CCT_2HEM_evt;in_memory\WDFW_CCT_2HEM_evt_2 does not exist or is not supported Failed to execute (Merge).

I am starting to get pretty lost 🙂
0 Kudos
ChristopherClark1
Deactivated User
Wait, I figured out that error also....my bad. Starting to get sloppy and not have the 'sharp eyes'
0 Kudos