arcpy.Exists doesn't show files that do exist

1224
6
08-09-2022 04:27 PM
Labels (1)
GabrielMarcus_C
New Contributor III

I'm trying to resolve some weird (to me) problems with arcpy.Exists() I'm running AGP 3.0. I created a script that I'm running from a toolbox. I've searched for bugs and past posts. I've seen several other posts about problems with Exists() but nothing seems relevant to my situation. 

The script takes 3 params. 2 of which may not exist. I'm using Exists to check.
If it's not there, I copy them from a template using FeatureClassToFeatureClass (later switched to CopyFeatures). 
I don't want to overwrite param2 as that contains appended data collected through successive runs, but since it's not being detected, the script overwrites it. 

Exists doesn't detect the FC, even when I can see it in the Catalog stored in the GDB. 
My script (at the end of this post) prints out the workspace to verify it, twice. 
If I copy my code and params into the immediate window, it works. 

 

What am I not getting?

When I run the script multiple times, a yellow "!" appears letting me know that the FC exists.
I'm using full paths in the dialog box, but I tested the script first with basenames, later with full paths and there's no difference. 
1) I print the workspace variable when the script first runs and see that it is the GDB that the files reside. 
2) I run Exists next (it's false)
3) I run ListFeatureClasses, and the files I want aren't listed (even though they're in the catalog view)
4) I copy a template and store it in the names of param1 and param2
5) Exists finally sees it now, but not on the next run. 


Other things I've done:

  • Switched back to the default python environment. I'm running the script from either the toolbox and the immediate window. 
  • Switched the "FeatureClasstoFeatureClass" line with CopyFeatures. I believe that anything output by these is written to disk? There wasn't a noticeable affect between these two. 
  • I am defining an aprx variable with aprx = arcpy.mp.ArcGISProject("CURRENT") then defining the workspace as aprx.defaultGeodatabase  I replaced it with a hardcoded value. no difference. 
  • I tried using full paths or basenames. No difference. 

 

I'm running this in succession, so the files were just created by the previous run below. 

p0= a point file (not being used. I commented out the execution of the payload)

p1 = Output_July22

p2 = AppendTo_2022

CatalogView.png

Files are there!

GP Window.png

Files aren't there?!

Output.png

I'm lead to believe that the workspace value isn't properly set?
Exists honors the workspace environment (thanks Tip!)
ListFeatureClasses uses the workspace setting.

The workspace is set in Analysis > Environments
It's set explicitly in code. 

But when I switch to full paths there's no difference. Walk is using the workspace variable. 
The files still aren't being shown. What's happening?

FullPathNames.png

The code with all my redundant checks. 

 

 

aprx = arcpy.mp.ArcGISProject("CURRENT")
aprxMap = aprx.listMaps("Map")[0]

# Parameters
param0 = arcpy.GetParameterAsText(0) # Geocoded Points
param1 = arcpy.GetParameterAsText(1) # File to output to
param2 = arcpy.GetParameterAsText(2) # File to append

arcpy.env.workspace = aprx.defaultGeodatabase
arcpy.AddMessage(f"Workspace is: {arcpy.env.workspace}")
arcpy.env.overwriteOutput = True
p1 = param1              #tried using basename(param1).  No difference!
arcpy.AddMessage(f"p1 is {p1}")
p2 = param2
arcpy.AddMessage(f"p2 is {p2}")
if not arcpy.Exists(p1):
arcpy.AddMessage(f"Exists says {p1} doesn't exist")
else:
arcpy.AddMessage(f"Exists says {p1} exists")

if not arcpy.Exists(p2):
arcpy.AddMessage(f"Exists says {p2} doesn't exist")
else:
arcpy.AddMessage(f"Exists says {p2} exists")

paramExists = arcpy.ListFeatureClasses("*",feature_type="line")
arcpy.AddMessage(f"List Feature Classes: {paramExists}")

walk = arcpy.da.Walk(arcpy.env.workspace, datatype="FeatureClass")
for dirpath, dirnames, filenames in walk:
arcpy.AddMessage(f"From Walk: {filenames}")

if arcpy.Exists(p1):
arcpy.AddMessage(f"Exists: {p1}")
else:
arcpy.AddMessage(f"{p1} doesn't exist. Making from template")
arcpy.CopyFeatures_management("WF_Dist_To_Homes_template",p1)

if arcpy.Exists(p2):
arcpy.AddMessage(f"Exists: {p2}")
else:
arcpy.AddMessage(f"{p2} doesn't exist. Making from template")
arcpy.CopyFeatures_management("WF_Dist_To_Homes_template", p2)

#Checking that features were added

if not arcpy.Exists(p1):
arcpy.AddMessage(f"Exists says {p1} doesn't exist")
else:
arcpy.AddMessage(f"Exists says {p1} exists")

if not arcpy.Exists(p2):
arcpy.AddMessage(f"Exists says {p2} doesn't exist")
else:
arcpy.AddMessage(f"Exists says {p2} exists")


#verifying Workspace again
arcpy.AddMessage(f" Workspace: {arcpy.env.workspace}")
#Listing Feature Classes
arcpy.AddMessage(f"List FC: {arcpy.ListFeatureClasses('*', feature_type='line')}")

#Walk says:
walk = arcpy.da.Walk(arcpy.env.workspace, datatype="FeatureClass")
for dirpath, dirnames, filenames in walk:
arcpy.AddMessage(f"From Walk: {filenames}")

 

 

 

Tags (2)
0 Kudos
6 Replies
JohannesBierer
Occasional Contributor III

Could it be some kind of refreshing problem and that you can't refresh the map/ catalog view programatically in pro? Don't know, maybe something to read:

https://community.esri.com/t5/arcgis-pro-questions/arcgis-pro-arcpy-how-to-refresh-the-map/td-p/4063...

0 Kudos
GabrielMarcus_C
New Contributor III

Interesting idea, but I'm getting updates in the catalog and contents pane.

My tool script does update two FCs but it's commented out. I know that part works. My prob is trying to determine if the FC is there or not. 

0 Kudos
JoshuaBixby
MVP Esteemed Contributor

Nice work troubleshooting so far.  I can't think of an explanation Esri could offer that would not make this a defect given that you have tried several approaches and the expected results are not working for many or all.  I would contact Esri Support to log a defect, although you will still need to find a workaround in the meantime.

0 Kudos
GabrielMarcus_C
New Contributor III

Thanks. I'll submit a support request today.

I guess I could output to a shp and use os.path.exists() to see if the file, now residing outside of a GDB is there.
I would need to separate the date and time parts in 3 fields though. 

Or I could just remove the test for the file completely and live on the edge, but that's a path I don't want to go down. It's a slippery slope. It starts with not checking if the file is there. The next step is to get a toothpick then a black leather jacket, then a motorcycle and finally a neck tattoo with "GIS 4 lyfe" or "ESRI" across the knuckles.  You've given me a lot to think about. 😉 

0 Kudos
GabrielMarcus_C
New Contributor III

I submitted a bug request and had a call with the support person on Tues of last week. 
I referenced this post, and thought it was pretty clear why I used ListFeatureClasses and Walk in addition to Exists, but they didn't seem to understand why I was doing this and told me to pare down my code to remove these other function calls before they would go further in the testing process. Hey, I would have loved to not have to call ListFeatureClasses and Walk either! If Exists worked, I would have been happy. 

If anyone has a development environment with virtual machines with different versions of AGP, I'd like to see if it will run OK on a previous version because I'd like to roll back to a prev version but would prefer to know which one to go with first. 

0 Kudos
GabrielMarcus_C
New Contributor III

Apparently this is expected behavior! 

Here's what support said:

"If a script tool has an output data parameter, and the data supplied to that parameter exists when the script tool is executed, the existing output will essentially be overwritten. The tool will delete data at the output location to make space for the new file when the tool executes. While the data does exist initially, when arcpy.Exists is called, it is already deleted. 

This is an expected outcome when using this function in a script tool. It is moreso related to how parameters are treated in the software rather than the Exists() function not working as expected. 

One suggestion to get around this would be to set Overwrite Outputs to "False" in the environment of the tool so that existing files can't be overwritten. "

In addition to what was said by support, I found that setting all params to input also alleviated my problems. Thanks @JoshuaBixby for your response. 

 

0 Kudos