I'm trying to resolve some weird (to me) problems with arcpy.Exists() I'm running AGP 3.0. I created a script that I'm running from a toolbox. I've searched for bugs and past posts. I've seen several other posts about problems with Exists() but nothing seems relevant to my situation.
The script takes 3 params. 2 of which may not exist. I'm using Exists to check.
If it's not there, I copy them from a template using FeatureClassToFeatureClass (later switched to CopyFeatures).
I don't want to overwrite param2 as that contains appended data collected through successive runs, but since it's not being detected, the script overwrites it.
Exists doesn't detect the FC, even when I can see it in the Catalog stored in the GDB.
My script (at the end of this post) prints out the workspace to verify it, twice.
If I copy my code and params into the immediate window, it works.
What am I not getting?
When I run the script multiple times, a yellow "!" appears letting me know that the FC exists.
I'm using full paths in the dialog box, but I tested the script first with basenames, later with full paths and there's no difference.
1) I print the workspace variable when the script first runs and see that it is the GDB that the files reside.
2) I run Exists next (it's false)
3) I run ListFeatureClasses, and the files I want aren't listed (even though they're in the catalog view)
4) I copy a template and store it in the names of param1 and param2
5) Exists finally sees it now, but not on the next run.
Other things I've done:
I'm running this in succession, so the files were just created by the previous run below.
p0= a point file (not being used. I commented out the execution of the payload)
p1 = Output_July22
p2 = AppendTo_2022
Files are there!
Files aren't there?!
I'm lead to believe that the workspace value isn't properly set?
Exists honors the workspace environment (thanks Tip!)
ListFeatureClasses uses the workspace setting.
The workspace is set in Analysis > Environments
It's set explicitly in code.
But when I switch to full paths there's no difference. Walk is using the workspace variable.
The files still aren't being shown. What's happening?
The code with all my redundant checks.
aprx = arcpy.mp.ArcGISProject("CURRENT")
aprxMap = aprx.listMaps("Map")[0]
# Parameters
param0 = arcpy.GetParameterAsText(0) # Geocoded Points
param1 = arcpy.GetParameterAsText(1) # File to output to
param2 = arcpy.GetParameterAsText(2) # File to append
arcpy.env.workspace = aprx.defaultGeodatabase
arcpy.AddMessage(f"Workspace is: {arcpy.env.workspace}")
arcpy.env.overwriteOutput = True
p1 = param1 #tried using basename(param1). No difference!
arcpy.AddMessage(f"p1 is {p1}")
p2 = param2
arcpy.AddMessage(f"p2 is {p2}")
if not arcpy.Exists(p1):
arcpy.AddMessage(f"Exists says {p1} doesn't exist")
else:
arcpy.AddMessage(f"Exists says {p1} exists")
if not arcpy.Exists(p2):
arcpy.AddMessage(f"Exists says {p2} doesn't exist")
else:
arcpy.AddMessage(f"Exists says {p2} exists")
paramExists = arcpy.ListFeatureClasses("*",feature_type="line")
arcpy.AddMessage(f"List Feature Classes: {paramExists}")
walk = arcpy.da.Walk(arcpy.env.workspace, datatype="FeatureClass")
for dirpath, dirnames, filenames in walk:
arcpy.AddMessage(f"From Walk: {filenames}")
if arcpy.Exists(p1):
arcpy.AddMessage(f"Exists: {p1}")
else:
arcpy.AddMessage(f"{p1} doesn't exist. Making from template")
arcpy.CopyFeatures_management("WF_Dist_To_Homes_template",p1)
if arcpy.Exists(p2):
arcpy.AddMessage(f"Exists: {p2}")
else:
arcpy.AddMessage(f"{p2} doesn't exist. Making from template")
arcpy.CopyFeatures_management("WF_Dist_To_Homes_template", p2)
#Checking that features were added
if not arcpy.Exists(p1):
arcpy.AddMessage(f"Exists says {p1} doesn't exist")
else:
arcpy.AddMessage(f"Exists says {p1} exists")
if not arcpy.Exists(p2):
arcpy.AddMessage(f"Exists says {p2} doesn't exist")
else:
arcpy.AddMessage(f"Exists says {p2} exists")
#verifying Workspace again
arcpy.AddMessage(f" Workspace: {arcpy.env.workspace}")
#Listing Feature Classes
arcpy.AddMessage(f"List FC: {arcpy.ListFeatureClasses('*', feature_type='line')}")
#Walk says:
walk = arcpy.da.Walk(arcpy.env.workspace, datatype="FeatureClass")
for dirpath, dirnames, filenames in walk:
arcpy.AddMessage(f"From Walk: {filenames}")
Could it be some kind of refreshing problem and that you can't refresh the map/ catalog view programatically in pro? Don't know, maybe something to read:
Interesting idea, but I'm getting updates in the catalog and contents pane.
My tool script does update two FCs but it's commented out. I know that part works. My prob is trying to determine if the FC is there or not.
Nice work troubleshooting so far. I can't think of an explanation Esri could offer that would not make this a defect given that you have tried several approaches and the expected results are not working for many or all. I would contact Esri Support to log a defect, although you will still need to find a workaround in the meantime.
Thanks. I'll submit a support request today.
I guess I could output to a shp and use os.path.exists() to see if the file, now residing outside of a GDB is there.
I would need to separate the date and time parts in 3 fields though.
Or I could just remove the test for the file completely and live on the edge, but that's a path I don't want to go down. It's a slippery slope. It starts with not checking if the file is there. The next step is to get a toothpick then a black leather jacket, then a motorcycle and finally a neck tattoo with "GIS 4 lyfe" or "ESRI" across the knuckles. You've given me a lot to think about. 😉
I submitted a bug request and had a call with the support person on Tues of last week.
I referenced this post, and thought it was pretty clear why I used ListFeatureClasses and Walk in addition to Exists, but they didn't seem to understand why I was doing this and told me to pare down my code to remove these other function calls before they would go further in the testing process. Hey, I would have loved to not have to call ListFeatureClasses and Walk either! If Exists worked, I would have been happy.
If anyone has a development environment with virtual machines with different versions of AGP, I'd like to see if it will run OK on a previous version because I'd like to roll back to a prev version but would prefer to know which one to go with first.
Apparently this is expected behavior!
Here's what support said:
"If a script tool has an output data parameter, and the data supplied to that parameter exists when the script tool is executed, the existing output will essentially be overwritten. The tool will delete data at the output location to make space for the new file when the tool executes. While the data does exist initially, when arcpy.Exists is called, it is already deleted.
This is an expected outcome when using this function in a script tool. It is moreso related to how parameters are treated in the software rather than the Exists() function not working as expected.
One suggestion to get around this would be to set Overwrite Outputs to "False" in the environment of the tool so that existing files can't be overwritten. "
In addition to what was said by support, I found that setting all params to input also alleviated my problems. Thanks @JoshuaBixby for your response.