topic Re: Automating custom Arc Tool in Python Questions

Automating custom Arc Tool

Anonymous User — Sun, 12 Dec 2021 01:04:43 GMT

Hello.

I have created a custom ArcGIS tool, and need to automate this tool so that it can be scheduled to run weekly outside of ArcMap.

Right now, I have a version of the tool that runs inside of an ArcMap session, with the input table and target feature class as user defined parameters.

Here is the code that I have so far.

My Try/Except block is commented out for testing purposes.

The goal is that the script will join an SDE view table to a GDB feature class, select based on some criteria, calculate some fields, than remove the join. Works great as a tool, but getting it to work outside of that has proven impossible so far.

Thanks for any help!

import arcpy, sys, string, os, subprocess

gca = vca = "Conservation_Area_Name"
gcn = vcn = "Conservation_Area_Number"
gn = vn = "Network"

gfi = "Field_ID"
gai = "Asset_ID"
gt1n = "Tier_1_System_Name"
gt1t = "Tier_1_System_Type"
gt2n = "Tier_2_Major_Subsystem_Name"
gt2t = "Tier_2_Major_Subsystem_Type"
gt3n = "Tier_3_Minor_Subsystem_Name"
gt3t = "Tier_3_Minor_Subsystem_Type"
gt4n = "Asset_Name"

vfi = "FieldID"
vai = "AssetID"
vt1n = "Tier_1_Name"
vt1t = "Tier_1_Type"
vt2n = "Tier_2_Name"
vt2t = "Tier_2_Type"
vt3n = "Tier_3_Name"
vt3t = "Tier_3_Type"
vt4n = "Tier_4_Name"

gdfcin = r'C:\Projects\MDC\DataUpdate_20180315\MDC_PhysModel_FieldID_20180315.gdb\Exterior_Electrical\Phase_Converter_Exterior'
gdfc = gdfcin.split('\\')[-1]

vwfcin = r'C:\Users\rbasaria\AppData\Roaming\ESRI\Desktop10.4\ArcCatalog\MDCTest_1025.sde\MDCTest_1025.dbo.ViewETL_FacilityType_PhaseConverterExterior'
vwfc = vwfcin.split('\\')[-1]

slct = gdfc+"."+gai+" IS NULL OR ("+gdfc+"."+gai+" <> "+vwfc+"."+vai+")"

#try:
arcpy.AddJoin_management(gdfcin, gfi, vwfcin, vfi)
arcpy.SelectLayerByAttribute_management(gdfcin, "NEW_SELECTION", slct)
arcpy.CalculateField_management(gdfcin, gdfc+"."+gai, "["+vwfc+"."+vai+"]", "VB", "")
arcpy.CalculateField_management(gdfcin, gdfc+"."+gca, "["+vwfc+"."+vca+"]", "VB", "")
arcpy.CalculateField_management(gdfcin, gdfc+"."+gcn, "["+vwfc+"."+vcn+"]", "VB", "")
arcpy.CalculateField_management(gdfcin, gdfc+"."+gn, "["+vwfc+"."+vn+"]", "VB", "")
arcpy.CalculateField_management(gdfcin, gdfc+"."+gt1n, "["+vwfc+"."+vt1n+"]", "VB", "")
arcpy.CalculateField_management(gdfcin, gdfc+"."+gt1t, "["+vwfc+"."+vt1t+"]", "VB", "")
arcpy.CalculateField_management(gdfcin, gdfc+"."+gt2n, "["+vwfc+"."+vt2n+"]", "VB", "")
arcpy.CalculateField_management(gdfcin, gdfc+"."+gt2t, "["+vwfc+"."+vt2t+"]", "VB", "")
arcpy.CalculateField_management(gdfcin, gdfc+"."+gt3n, "["+vwfc+"."+vt3n+"]", "VB", "")
arcpy.CalculateField_management(gdfcin, gdfc+"."+gt3t, "["+vwfc+"."+vt3t+"]", "VB", "")
arcpy.CalculateField_management(gdfcin, gdfc+"."+gt4n, "["+vwfc+"."+vt4n+"]", "VB", "")
#except:
#    print arcpy.GetMessages()
    
arcpy.RemoveJoin_management(gdfcin)
arcpy.SelectLayerByAttribute_management(gdfcin, "CLEAR_SELECTION", "")
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Automating custom Arc Tool

DanPatterson_Retired — Mon, 16 Apr 2018 16:31:58 GMT

You want to run a 'scheduled task'

There are links on GeoNet if you search using that key phrase, plus options in the help topics

http://pro.arcgis.com/en/pro-app/arcpy/get-started/installing-python-for-arcgis-pro.htm

http://pro.arcgis.com/en/pro-app/help/analysis/geoprocessing/modelbuilder/scheduling-a-model-run.htm

Re: Automating custom Arc Tool

Anonymous User — Mon, 16 Apr 2018 16:39:30 GMT

I was planning on using windows task scheduler for that part. But this isn't really the issue...

The issue is getting the script to work outside of an ArcMap session. Right now, it doesn't seem to want to recognize the feature class and table that I am pointing to.

Re: Automating custom Arc Tool

DanPatterson_Retired — Mon, 16 Apr 2018 16:42:50 GMT

reference the featureclass and table on disk and not the layer or table view in the project

Re: Automating custom Arc Tool

Anonymous User — Mon, 16 Apr 2018 16:49:15 GMT

That's what my script is currently trying to do, and seems to be failing at. Right now, it crashes at the AddJoin function, telling me that the parameters are not valid and that the value cannot be a feature class.

Re: Automating custom Arc Tool

Anonymous User — Mon, 16 Apr 2018 16:50:18 GMT

Traceback (most recent call last):
File "C:\Projects\MDC\DataUpdate_20180315\MDC_ViewtoGIS_Automated.py", line 36, in <module>
arcpy.AddJoin_management(gdfcin, gfi, vwfcin, vfi)
File "C:\Program Files (x86)\ArcGIS\Desktop10.4\ArcPy\arcpy\management.py", line 6066, in AddJoin
raise e
ExecuteError: Failed to execute. Parameters are not valid.
The value cannot be a feature class
ERROR 000840: The value is not a Raster Layer.
ERROR 000840: The value is not a Raster Catalog Layer.
ERROR 000840: The value is not a Mosaic Layer.
WARNING 000970: The join field Field_ID in the join table Phase_Converter_Exterior is not indexed. To improve performance, we recommend that an index be created for the join field in the join table.
Failed to execute (AddJoin).

Re: Automating custom Arc Tool

JamesMacKay3 — Mon, 16 Apr 2018 16:59:45 GMT

While some geoprocessing tools accept both datasets as well as feature layers, others require a layer - and Add Join is one of them. First call the Make Feature Layer tool on the input, then use the output of that tool as input to Add Join.

Re: Automating custom Arc Tool

JamesCrandall — Mon, 16 Apr 2018 17:01:12 GMT

The vwfcin looks like it's being set to a named user directory? If this is not related to the current problem you are experiencing, my guess is that it will be a problem later on if setting this to run as a scheduled task. Also, is "Exterior_Electrical" a Feature Dataset in the gdb? You may have to go ahead and re-work those path references using os.path.join()

You may want to just eliminate anything like this for now and simply hardcode the value.

gdfc = gdfcin.split('\\')[-1]‍

Re: Automating custom Arc Tool

Anonymous User — Mon, 16 Apr 2018 17:12:17 GMT

I don't think the named user thing is causing the problem at this point, but I am not sure. The path for that is supposed to be pointing to a SQL database, and thats the only way I know how to connect to it...Yes, "Exterior_Electrical" is a feature dataset inside the GDB. I'm not sure what you mean by rework the path references...

Re: Automating custom Arc Tool

Anonymous User — Mon, 16 Apr 2018 17:12:34 GMT

Interesting! I did not know that. Will try and report back.

Re: Automating custom Arc Tool

DanPatterson_Retired — Mon, 16 Apr 2018 17:17:05 GMT

Join field http://pro.arcgis.com/en/pro-app/tool-reference/data-management/join-field.htm

if you don't want to use a layer

Re: Automating custom Arc Tool

Anonymous User — Sun, 12 Dec 2021 01:04:46 GMT

Updated script and error...seems MakeFeatureLayer does not like the SQL view table...

import arcpy, sys, string, os, subprocess

gca = vca = "Conservation_Area_Name"
gcn = vcn = "Conservation_Area_Number"
gn = vn = "Network"

gfi = "Field_ID"
gai = "Asset_ID"
gt1n = "Tier_1_System_Name"
gt1t = "Tier_1_System_Type"
gt2n = "Tier_2_Major_Subsystem_Name"
gt2t = "Tier_2_Major_Subsystem_Type"
gt3n = "Tier_3_Minor_Subsystem_Name"
gt3t = "Tier_3_Minor_Subsystem_Type"
gt4n = "Asset_Name"

vfi = "FieldID"
vai = "AssetID"
vt1n = "Tier_1_Name"
vt1t = "Tier_1_Type"
vt2n = "Tier_2_Name"
vt2t = "Tier_2_Type"
vt3n = "Tier_3_Name"
vt3t = "Tier_3_Type"
vt4n = "Tier_4_Name"

gdfc = "Phase_Converter_Exterior"
gdfcin = arcpy.MakeFeatureLayer_management(r'C:\Projects\MDC\DataUpdate_20180315\MDC_PhysModel_FieldID_20180315.gdb\Exterior_Electrical\Phase_Converter_Exterior', gdfc)

vwfc = "PhaseConverterExterior"
vwfcin = arcpy.MakeFeatureLayer_management(r'C:\Users\rbasaria\AppData\Roaming\ESRI\Desktop10.4\ArcCatalog\MDCTest_1025.sde\MDCTest_1025.dbo.ViewETL_FacilityType_PhaseConverterExterior', vwfc)

slct = gdfc+"."+gai+" IS NULL OR ("+gdfc+"."+gai+" <> "+vwfc+"."+vai+")"

#try:
arcpy.AddJoin_management(gdfc, gfi, vwfc, vfi)
arcpy.SelectLayerByAttribute_management(gdfc, "NEW_SELECTION", slct)
arcpy.CalculateField_management(gdfc, gdfc+"."+gai, "["+vwfc+"."+vai+"]", "VB", "")
arcpy.CalculateField_management(gdfc, gdfc+"."+gca, "["+vwfc+"."+vca+"]", "VB", "")
arcpy.CalculateField_management(gdfc, gdfc+"."+gcn, "["+vwfc+"."+vcn+"]", "VB", "")
arcpy.CalculateField_management(gdfc, gdfc+"."+gn, "["+vwfc+"."+vn+"]", "VB", "")
arcpy.CalculateField_management(gdfc, gdfc+"."+gt1n, "["+vwfc+"."+vt1n+"]", "VB", "")
arcpy.CalculateField_management(gdfc, gdfc+"."+gt1t, "["+vwfc+"."+vt1t+"]", "VB", "")
arcpy.CalculateField_management(gdfc, gdfc+"."+gt2n, "["+vwfc+"."+vt2n+"]", "VB", "")
arcpy.CalculateField_management(gdfc, gdfc+"."+gt2t, "["+vwfc+"."+vt2t+"]", "VB", "")
arcpy.CalculateField_management(gdfc, gdfc+"."+gt3n, "["+vwfc+"."+vt3n+"]", "VB", "")
arcpy.CalculateField_management(gdfc, gdfc+"."+gt3t, "["+vwfc+"."+vt3t+"]", "VB", "")
arcpy.CalculateField_management(gdfc, gdfc+"."+gt4n, "["+vwfc+"."+vt4n+"]", "VB", "")
#except:
#    print arcpy.GetMessages()
    
arcpy.RemoveJoin_management(gdfc)
arcpy.SelectLayerByAttribute_management(gdfc, "CLEAR_SELECTION", "")
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Traceback (most recent call last):
File "C:\Projects\MDC\DataUpdate_20180315\MDC_ViewtoGIS_Automated.py", line 31, in <module>
vwfcin = arcpy.MakeFeatureLayer_management(r'C:\Users\rbasaria\AppData\Roaming\ESRI\Desktop10.4\ArcCatalog\MDCTest_1025.sde\MDCTest_1025.dbo.ViewETL_FacilityType_PhaseConverterExterior', vwfc)
File "C:\Program Files (x86)\ArcGIS\Desktop10.4\ArcPy\arcpy\management.py", line 6477, in MakeFeatureLayer
raise e
ExecuteError: Failed to execute. Parameters are not valid.
ERROR 000732: Input Features: Dataset C:\Users\rbasaria\AppData\Roaming\ESRI\Desktop10.4\ArcCatalog\MDCTest_1025.sde\MDCTest_1025.dbo.ViewETL_FacilityType_PhaseConverterExterior does not exist or is not supported
Failed to execute (MakeFeatureLayer).

Re: Automating custom Arc Tool

JamesMacKay3 — Mon, 16 Apr 2018 17:24:36 GMT

Is it a non-spatial dataset? If so, try Make Table View instead.

(Edit: Add Join works with non-spatial table views as well as feature layers.)

Re: Automating custom Arc Tool

JamesCrandall — Mon, 16 Apr 2018 17:30:31 GMT

I don't think the named user thing is causing the problem at this point, but I am not sure

If your task is running with a different account than yours, then it will likely fail whenever you are referencing a path with your credentials in it like you've got now,

r'C:\Users\rbasaria\AppData\Roaming\ESRI

Re: Automating custom Arc Tool

Anonymous User — Mon, 16 Apr 2018 17:32:35 GMT

Oh yeah, I get that. These are my testing paths. These paths will change at some point, but for now...this is what I am working with.

Re: Automating custom Arc Tool

DanPatterson_Retired — Mon, 16 Apr 2018 17:32:43 GMT

SDE? can you try a simple local location to see whether the script works... just trying to rule out saving types and locations from the equation

Re: Automating custom Arc Tool

JamesCrandall — Sun, 12 Dec 2021 01:04:49 GMT

Yes, "Exterior_Electrical" is a feature dataset inside the GDB. I'm not sure what you mean by rework the path references...

Don't just set a variable reference to the full path of the feature class like you've done,

gdfcin = r'C:\Projects\MDC\DataUpdate_20180315\MDC_PhysModel_FieldID_20180315.gdb\Exterior_Electrical\Phase_Converter_Exterior'‍

Use os.path.join() to build up the reference

workspace = r'C:\Projects\MDC\DataUpdate_20180315\MDC_PhysModel_FieldID_20180315.gdb\Exterior_Electrical'

fcName = 'Phase_Converter_Exterior'

gdfcin = os.path.join(workspace, fcName)‍‍‍‍‍

Re: Automating custom Arc Tool

Anonymous User — Sun, 12 Dec 2021 01:04:52 GMT

Well, it looks like its working now!

Combination of the Make Feature Layer and Make Table View got this thing grabbing the right data, then I had an issue with my slct variable (which I know works)...which turned out to be the vwfc variable not having the "MDCTest_1025.dbo.ViewETL_FacilityType_" info before the table name.

Appreciate all the help, and its always a good day when I can learn something new!

import arcpy, sys, string, os, subprocess

gca = vca = "Conservation_Area_Name"
gcn = vcn = "Conservation_Area_Number"
gn = vn = "Network"

gfi = "Field_ID"
gai = "Asset_ID"
gt1n = "Tier_1_System_Name"
gt1t = "Tier_1_System_Type"
gt2n = "Tier_2_Major_Subsystem_Name"
gt2t = "Tier_2_Major_Subsystem_Type"
gt3n = "Tier_3_Minor_Subsystem_Name"
gt3t = "Tier_3_Minor_Subsystem_Type"
gt4n = "Asset_Name"

vfi = "FieldID"
vai = "AssetID"
vt1n = "Tier_1_Name"
vt1t = "Tier_1_Type"
vt2n = "Tier_2_Name"
vt2t = "Tier_2_Type"
vt3n = "Tier_3_Name"
vt3t = "Tier_3_Type"
vt4n = "Tier_4_Name"

gdfc = "Phase_Converter_Exterior"
gdfcin = arcpy.MakeFeatureLayer_management(r'C:\Projects\MDC\DataUpdate_20180315\MDC_PhysModel_FieldID_20180315.gdb\Exterior_Electrical\Phase_Converter_Exterior', gdfc)

vwfc = "MDCTest_1025.dbo.ViewETL_FacilityType_PhaseConverterExterior"
vwfcin = arcpy.MakeTableView_management(r'C:\Users\rbasaria\AppData\Roaming\ESRI\Desktop10.4\ArcCatalog\MDCTest_1025.sde\MDCTest_1025.dbo.ViewETL_FacilityType_PhaseConverterExterior', vwfc)

slct = gdfc+"."+gai+" IS NULL OR ("+gdfc+"."+gai+" <> "+vwfc+"."+vai+")"

#try:
arcpy.AddJoin_management(gdfc, gfi, vwfc, vfi)
arcpy.SelectLayerByAttribute_management(gdfc, "NEW_SELECTION", slct)
arcpy.CalculateField_management(gdfc, gdfc+"."+gai, "["+vwfc+"."+vai+"]", "VB", "")
arcpy.CalculateField_management(gdfc, gdfc+"."+gca, "["+vwfc+"."+vca+"]", "VB", "")
arcpy.CalculateField_management(gdfc, gdfc+"."+gcn, "["+vwfc+"."+vcn+"]", "VB", "")
arcpy.CalculateField_management(gdfc, gdfc+"."+gn, "["+vwfc+"."+vn+"]", "VB", "")
arcpy.CalculateField_management(gdfc, gdfc+"."+gt1n, "["+vwfc+"."+vt1n+"]", "VB", "")
arcpy.CalculateField_management(gdfc, gdfc+"."+gt1t, "["+vwfc+"."+vt1t+"]", "VB", "")
arcpy.CalculateField_management(gdfc, gdfc+"."+gt2n, "["+vwfc+"."+vt2n+"]", "VB", "")
arcpy.CalculateField_management(gdfc, gdfc+"."+gt2t, "["+vwfc+"."+vt2t+"]", "VB", "")
arcpy.CalculateField_management(gdfc, gdfc+"."+gt3n, "["+vwfc+"."+vt3n+"]", "VB", "")
arcpy.CalculateField_management(gdfc, gdfc+"."+gt3t, "["+vwfc+"."+vt3t+"]", "VB", "")
arcpy.CalculateField_management(gdfc, gdfc+"."+gt4n, "["+vwfc+"."+vt4n+"]", "VB", "")
#except:
#    print arcpy.GetMessages()
    
arcpy.RemoveJoin_management(gdfc)
arcpy.SelectLayerByAttribute_management(gdfc, "CLEAR_SELECTION", "")
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍