topic Re: python script tool - files in given directory not found in Python Questions

python script tool - files in given directory not found

Nicole_Ueberschär — Fri, 28 Apr 2017 12:05:39 GMT

Hello friendly people,

I have written a python script to split up text files. It has an input folder (and an output folder) that I refer to in the beginning:

hardcoded version: _location_for_tables = r"C:\data\2017\LakeKivu\Data\Python_testing\python\Pilot\source"
script tool version: _location_for_tables = arcpy.GetParameterAsText(0)

When I execute the script in Pyscripter and in the command window, also in the python window in ArcGIS, everything runs as expected. But when I run it as a script tool it doesn't find my files. I tried it with the hardcoded path in the script and with an input field where I selected the (same) folder - no difference. I tried to put the path variables in my main() function and in the beginning of my script - no difference.

The only error message I receive is my defined message when the list of files is empty.

Has anyone an idea where the problem is? Please let me know if you need more details. Thanks in advance!

Re: python script tool - files in given directory not found

NeilAyres — Fri, 28 Apr 2017 12:36:58 GMT

And, in the script tool properties / parameters, what have you set the data type to? As these are text files, this should by type Folder.

Do you have try / except blocks in your code? Get rid of them to reveal any actual error.

Version?

A look at the code would also be useful to see.

Re: python script tool - files in given directory not found

Nicole_Ueberschär — Sat, 11 Dec 2021 17:14:07 GMT

Hi Neil, thanks for your reply.

In the script tool properties the data type for both folders is "folder", both are required, input and no multivalue.

I don't have try / except blocks.

I'm using Python 2.7 in ArcGIS 10.4.1, PyScripter 2.6, all running on Windows 10.

Thank you!

Here is my main() function:

def main():
    #input file
    ##_location_for_tables = r"C:\data\2017\LakeKivu\Data\Python_testing\python\Pilot\source"
    _location_for_tables = arcpy.GetParameterAsText(0)

    #output folder
    ##DirOutTables = r"C:\data\2017\LakeKivu\Data\Python_testing\python\Pilot\attachments"
    DirOutTables= arcpy.GetParameterAsText(1)


    # Step 1: find the source files to process
    list_of_files = FindFiles(_location_for_tables)

    if list_of_files==[]:
        arcpy.AddError("list empty")
    else:
        arcpy.AddMessage(list_of_files)
    # Step2: prozess each file to create a point in points
    for each_file in list_of_files:
        # create and save sub tables
        store_table(each_file)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: python script tool - files in given directory not found

NeilAyres — Sat, 11 Dec 2021 17:14:10 GMT

Have you put a print / AddMessage statement to see what is in the variable "list_of_files"

And this if below....

if list_of_files==[]:
        arcpy.AddError("list empty")

If you want to test if list_of_files has something in it, then use...

if len(list_of_files) == 0:

Is there any reason that the variable _location_for_tables has to start with an "_". Not sure but that leading underscore might mean something

Re: python script tool - files in given directory not found

DanPatterson_Retired — Fri, 28 Apr 2017 14:19:22 GMT

FindFiles ... does that appear elsewhere? are you sure it returns a python list object? why not add a print/addMessage line to confirm?

Online 15, you just add an error, but the script will proceed on at line 19 as if nothing happened, you should bail out of the script or do something else if the list_of_files is empty since there is no point continuing at that point

ie sys.exit()

Re: python script tool - files in given directory not found

Nicole_Ueberschär — Sat, 11 Dec 2021 17:14:13 GMT

Hi Dan & Neil, ok, here comes the whole script. It might look in some parts familiar, still dealing with my profiles. But at least it's in general working fine....:

That the _location variable had the underscore is only due to the programming habits of my colleague...(doesn't make a difference)

import arcpy
import os
import numpy as np
import unicodedata
import codecs
import csv

#Workspace settings
arcpy.env.overwriteOutput= True

#input file
##location_for_tables = r"C:\data\2017\LakeKivu\Data\Python_testing\python\Pilot\source"
location_for_tables = arcpy.GetParameterAsText(0)

#output folder
##DirOutTables = r"C:\data\2017\LakeKivu\Data\Python_testing\python\Pilot\attachments"
DirOutTables= arcpy.GetParameterAsText(1)

def FindFiles(location):
    arcpy.env.workspace = location
    ListFiles=arcpy.ListFiles("*.txt")
    return ListFiles

def unique_values(table, field):
    with arcpy.da.SearchCursor(table, [field]) as cursor:
        return sorted({row[0] for row in cursor})

def CreateTableView(location, filename):
    arcpy.env.overwriteOutput = True
    complete_filename = os.path.join(location, filename)
    arcpy.MakeTableView_management(complete_filename, "kibu_tview")
    return "kibu_tview"

def store_table(table):
    arcpy.AddMessage("open "+table)

    profile_nrs=unique_values(table, "Station")
    arcpy.AddMessage(profile_nrs)

    #get field names
    desc = arcpy.Describe(table)
    fieldnames=[field.name.encode("utf-8") for field in desc.fields]


    #safe header line
    tablefile=location_for_tables+"\\"+table #original table


    # Select rows by station nr and save to txt file
    for profile in profile_nrs:
         if profile >0:
            expression="Station="+str(profile)
            print(expression)

            with arcpy.da.SearchCursor(table,"*",expression) as sCursor:
                for sRow in sCursor:
                    table_name=str(sRow[0])+"_"+str(sRow[1])+".txt"
                    filepath=DirOutTables +"\\"+ table_name   #seperated tables
                    UIDvalue=str(sRow[0])+"_"+str(sRow[1])
                    break
                if arcpy.Exists(filepath):
                    arcpy.AddMessage("Table "+table_name+" exists")
                else:
                    with open(tablefile, 'r') as t:
                        with open(filepath, 'a') as f:
                            w = csv.writer(f, delimiter="\t", lineterminator='\n')
                            r = csv.reader(t, delimiter="\t")
                            all=[]
                            header=r.next()
                            header.append('UPI')
                            all.append(header)
                            ##print(header)
                            for sRow in sCursor:
                                sRow=sRow + (UIDvalue,)
                                all.append(sRow)
                            w.writerows(all)

                    del sRow
                    arcpy.AddMessage("Table "+table_name+" created")
         else:
            print("empty line")


def main():
    #input file
    ##location_for_tables = r"C:\data\2017\LakeKivu\Data\Python_testing\python\Pilot\source"
    location_for_tables = arcpy.GetParameterAsText(0)

    #output folder
    ##DirOutTables = r"C:\data\2017\LakeKivu\Data\Python_testing\python\Pilot\attachments"
    DirOutTables= arcpy.GetParameterAsText(1)


    # Step 1: find the source files to process
    list_of_files = FindFiles(location_for_tables)

    if len(list_of_files)==0:
        arcpy.AddError("list empty")
        sys.exit()
    else:
        arcpy.AddMessage(list_of_files)
    # Step2: prozess each file to create a point in points
    for each_file in list_of_files:
        # create and save sub tables
        store_table(each_file)

if __name__ == '__main__':
    main()‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

I don't see why the script should work in PyScripter, in the command window, in the python window in ArcGIS but not as a tool even if don't use any parameters.

The difference I find between executing the tool and the same script in cmd is that in ArcGIS it does not put the paths in "path comes here" but in cmd they are required.

Re: python script tool - files in given directory not found

DanPatterson_Retired — Fri, 28 Apr 2017 16:23:19 GMT

I am not sure why you would want to implement a tool without parameters...I have never had a use case for one, so I can't help

Re: python script tool - files in given directory not found

Nicole_Ueberschär — Tue, 02 May 2017 06:58:16 GMT

Of course it is not meant to be without parameters but I was trying if it works with or without receiving the parameters through the tool. I was assuming that if the parameters are hard coded the script should just run as it is running in pyscripter, in cmd and in the python window in ArcGIS 10.4 - but it is not. That is why I was asking for help.

My script as I posted it is taking the paths as parameters. But no matter if I take them as parameters or hard coded the script won't find any files in my directory when executed as Script Tool.

Re: python script tool - files in given directory not found

DanPatterson_Retired — Tue, 02 May 2017 08:14:50 GMT

last try... move 'main()' from line 108 to 106 and dedent... since you report no errors, is 'main' actually being run would be my last question when the script is associated with a tool and not a standalone script (where it would)

Re: python script tool - files in given directory not found

Nicole_Ueberschär — Tue, 02 May 2017 08:57:20 GMT

Hi Dan, thanks for your last try.

Main is obviously executed because I get the error message as defined in line 98.

Re: python script tool - files in given directory not found

DanPatterson_Retired — Tue, 02 May 2017 09:15:35 GMT

and that was with path lines uncommented and the getparameterastext lines commented out?

Re: python script tool - files in given directory not found

Nicole_Ueberschär — Tue, 02 May 2017 09:40:35 GMT

I tried it both, with folder path names in the script and as parameters.

Re: python script tool - files in given directory not found

DanPatterson_Retired — Sun, 12 Dec 2021 16:23:33 GMT

ListFiles=arcpy.ListFiles("*.txt")

maybe rename your ListFiles variable incase it is conflicting with arcpy's ListFiles (and that's all I can think of)

Re: python script tool - files in given directory not found

Nicole_Ueberschär — Tue, 02 May 2017 12:54:57 GMT

Thank you both for looking into this. I found the problem: I unchecked the "Run in process" Checkbox when creating the tool. So in case someone else is facing similar problems: Make sure to leave the "Run in process" checkbox checked!

(I had a colleague also looking at this and he didn't have a clue why it would not run without this checkbox though...)