topic Re: How do you loop through a list of files, getting the filenames, and then adding the names to a standalone table? in Python Questions

How do you loop through a list of files, getting the filenames, and then adding the names to a standalone table?

RPGIS — Sat, 11 Dec 2021 22:03:25 GMT

Hi,

i am relatively new to python but I am trying to work on a script that works in conjunction with modelbuilder and so far things were working up to a point. The issue that I am having is I am trying to loop through a list of files, get the filenames, and update a table with the names from the files. Here is the script that I have currently:

import arcpy
import os

#Set Parameters
inFiles = arcpy.GetParameterAsText(0)
inTable = arcpy.GetParameterAsText(1)
inField = arcpy.GetParameterAsText(2)
outTable = arcpy.SetParameterAsText(3)

#Iterate through files in folder, insert rows based on file names, and make table view layer
with arcpy.da.InsertCursor(inTable, inField) as cursor:
    #List Files
    flist = arcpy.List(inFiles)

    for f in flist:
        
        #Make table view
        arcpy.MakeTableView_management(inTable)
        
        #Get filename(s)
        f = os.path.basename(flist).rstrip(os.path.splitext(flist)[1])
        
        #Insert values and rows in table
        cursor.insertRow([f])

del cursor

#Add Message
arcpy.AddMessage("This row {0} was added.".format(f))‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Any help on this would be greatly appreciated.

Re: How do you loop through a list of files, getting the filenames, and then adding the names to a standalone table?

JohannesBierer — Mon, 16 Sep 2019 13:10:34 GMT

Could you use code formatting?

/blogs/dan_patterson/2016/08/14/script-formatting

Re: How do you loop through a list of files, getting the filenames, and then adding the names to a standalone table?

RPGIS — Mon, 16 Sep 2019 13:54:17 GMT

I never used the code formatting before so it's new to me.

Re: How do you loop through a list of files, getting the filenames, and then adding the names to a standalone table?

JohannesBierer — Mon, 16 Sep 2019 13:56:59 GMT

Does arcpy.List () exists? I would start with this:

Erstellen von Datenlisten—ArcPy Get Started | ArcGIS Desktop

Re: How do you loop through a list of files, getting the filenames, and then adding the names to a standalone table?

RPGIS — Mon, 16 Sep 2019 14:21:37 GMT

Thanks Johannes. I did manage to get the script working, however, I get an empty output when I run the tool and so I am not sure what I am doing wrong exactly.

Re: How do you loop through a list of files, getting the filenames, and then adding the names to a standalone table?

JohannesBierer — Sat, 11 Dec 2021 22:03:28 GMT

What if you put in a

arcpy.AddMessage(f)
after line 20, does it throws something out?

I think your code needs to be "rebuilt", it's easier to do this with hard coded "parameters" rather thanarcpy.GetParameterAsText(0)

Re: How do you loop through a list of files, getting the filenames, and then adding the names to a standalone table?

JoshuaBixby — Mon, 16 Sep 2019 14:47:54 GMT

A few questions:

What does arcpy.GetParameterAsText(0) look like?
1. A comma- or semicolon-delimited string?
2. Is the list using full paths?
You have an outTable parameter but aren't using it anywhere? Why are you including it?
As Johannes Bierer‌ mentioned already, what is arcpy.List?

Re: How do you loop through a list of files, getting the filenames, and then adding the names to a standalone table?

JamesCrandall — Sat, 11 Dec 2021 22:03:30 GMT

Looks like you just need to update the list of files to remove the path, leaving a file name then just append to a new list to use in the InsertCursor. Also, MakeTableView_management is expecting another parameter (you only provided a single "inTable" and it needs the output name of the table view you want to create.

inFiles = ['H:\\file1.txt','H:\\file2.txt','H:\\file3.txt']
inTable = r'\\computer1\home$\user\Docs\ArcGIS\Default.gdb\tabSourceIn'
inField = ['filenames']

rowvalues = []

for file in inFiles:
    rowvalues.append(os.path.splitext(os.path.basename(file))[0])

cursor = arcpy.da.InsertCursor(inTable, ['filenames'])
for row in rowvalues:
    cursor.insertRow([row])

del cursor

arcpy.MakeTableView_management(inTable, 'outputTableView')‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: How do you loop through a list of files, getting the filenames, and then adding the names to a standalone table?

RPGIS — Mon, 16 Sep 2019 15:31:14 GMT

Thanks James,

I was trying to make sense of the parse path script and loop. I am vaguely familiar with loops but the parse path script form the esri documentation wasn't making much sense to me. The reason I was trying to utilize the Make Table View script is because when I ran the script in process within a model, I would receive an error stating the item cannot be opened or it does not exist. I think this link that I was utilizing might have been adding my confusion but I could be wrong.

Parse Path—Help | ArcGIS for Desktop

I am trying to get a better understanding on how python works in terms of context and flow. I have several scripts that I have created that work the way I intended, but when I start to get into developing more complicated script, I start to get a bit confused.

Huge thanks for your help James

-Robert.

Re: How do you loop through a list of files, getting the filenames, and then adding the names to a standalone table?

RPGIS — Mon, 16 Sep 2019 17:24:17 GMT

Hey James,

So I tested your script and it works but it populates the table with 20+ records, and so I am wondering if there is a way for me to have the table records update based on the number of files added. So if I add 3 files, then the tool adds 3 records to the table with the file names. I am really close to getting where I want to be and so I am working on this little tiny detail.

Thanks,

Robert

Re: How do you loop through a list of files, getting the filenames, and then adding the names to a standalone table?

JamesCrandall — Mon, 16 Sep 2019 18:05:26 GMT

That's what it should be doing as it's simply using whatever you add to the "inFiles" list. In my example it would only add the 3 items. If you had populated the list with 20 file references then it would have added 20 rows to the table. Not sure if that answers your question.

Re: How do you loop through a list of files, getting the filenames, and then adding the names to a standalone table?

LaurenceLanglois1 — Mon, 16 Sep 2019 19:20:35 GMT

This might be an obvious trick, it is a manual step though.

In Windows Explorer.

Select all files in you directory.

Hold Shift and 'Right Click'

'Copy as Path'

Paste into excel or notepad.

Save record

Point to your new file...

Re: How do you loop through a list of files, getting the filenames, and then adding the names to a standalone table?

RPGIS — Mon, 16 Sep 2019 19:52:31 GMT

Ok. Because for some reason I am able to parse the path using modelbuilder and it returns the correct record. The only issue I have with using the script in modelbuilder is that I receive an error message stating the table either does't exists or it simply doesn't open. I have a slightly more dumbed down version of the model that works but I my end goal is to use the script in process with the model which would make this process more seamless. Otherwise the model and script combination that is working currently, works as two separate model tools. So I am simply trying to figure out the best approach to make two processes work as one.

Re: How do you loop through a list of files, getting the filenames, and then adding the names to a standalone table?

PavanYadav — Mon, 16 Sep 2019 20:50:34 GMT

Not sure if arcpy.da.Walk will fix your issue but you might want to use if you're interested in listing datasets in subdirectories as well.

Re: How do you loop through a list of files, getting the filenames, and then adding the names to a standalone table?

RPGIS — Sat, 11 Dec 2021 22:03:33 GMT

Thanks Pavan,

I checked with one of my coworkers on this and he walked me through a process that you mentioned. Here is the script that he gave me as well as my own additions(mostly his).

import os

path = r"C:\Folder\FolderName"

files = []

for r, d, f in os.walk(path):
    for file in f:
        if '.mdb' in file:
            files.append(file)

for f in files:
    ##print(f)
    strip = os.path.basename(file).rstrip(os.path.splitext(file)[1])
    print strip‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

This solution was what I was actually trying to get to but, with my knowledge of python being so limited, I don't have a firm grasp on it. I meet with a friend every now and then to study python, but there is more to python than I can feasibly grasp.

-Robert

Re: How do you loop through a list of files, getting the filenames, and then adding the names to a standalone table?

GailMorrison — Tue, 17 Sep 2019 16:06:57 GMT

This might be easier to do in Excel: I can't give you all the steps as they have govt. pathnames within the screen scratches. When you click on From Folder, a gui opens to allow you to Browse to a folder path, then click OK. Once a directory list is created, you can move the columns around. Using the Extension down'arrow, you can sort for just *gdb, *shp, etc. Last, Close & Load To ... (an excel worksheet). It's very fast & easy to use.

www.youtube.com/watch?v=GO-Q8j2lW2s

Get a List of File Names from Folders & Sub-folders in Excel (using Power Query) - YouTube

Last step:

Re: How do you loop through a list of files, getting the filenames, and then adding the names to a standalone table?

PavanYadav — Tue, 17 Sep 2019 16:43:53 GMT

Hi Robert , I notice you are using os.walk not arcpy.da.Walk. That's okay if it serves your purpose but os.walk is file based and does not recognize database contents such as geodatabase feature classes, tables, or rasters. I thought I should point it out because in the future you might find it useful.

Re: How do you loop through a list of files, getting the filenames, and then adding the names to a standalone table?

RPGIS — Tue, 17 Sep 2019 16:58:48 GMT

Thanks Pavan. I didn't write that portion of the script, somebody else did and so I used it to test the script and it worked. I know that with the cursors you have to use .da. I didn't know that os.walk also needed the .da as well. I will be sure to remember that.

Re: How do you loop through a list of files, getting the filenames, and then adding the names to a standalone table?

PavanYadav — Tue, 17 Sep 2019 17:09:05 GMT

Robert, they are two different but similar functions. See the Discussion section at this help page: Walk—Data Access module | ArcGIS Desktop
in most cases, os.walk works. So, please don't worry about changing it.

Re: How do you loop through a list of files, getting the filenames, and then adding the names to a standalone table?

RPGIS — Wed, 18 Sep 2019 00:34:56 GMT

Thanks Pavan,

I won't worry about making changes. I'm still trying to get a better understanding of how the scripting in python affects the behavior of the overall script, and how this can change the overall outcome of the desired result. The other thing that I'm still trying to grasp, that I simply don't fully understand at this point, is the direction. I know what I would like for my end result to be at times, but getting there via scripting is more challenging. I'm really familiar with modelbuilder and I can create most tools from simple models. I realized that the end result that I want to get to is sometimes missing with the given esri tools. I realized that I could fill in for those missing pieces by creating script tools to fill in those gaps, or simplify long drawn out models by simplifying them to scripts. I arrived to the conclusion that I might need to practice more and get familiar with the basics. It's just that there's more to python than I originally thought, and there's other components that I was unaware of. So this is going to take time, but I would like to eventually get to the point that I can understand these things well enough that I won't get lost in the process. Any advice on this would be heavily appreciated.

- Robert