loop through a folder with text files

[MarkBalman]

Hi all

I am stuck with how to loop through a folder containing numerous text files containing lat and long values. The idea is to open a text file, create an XY layer and then append the result into a layer within a geodatabase. I can do the following which takes a single text file, creates the xy layer and then appends this xy layer to the geodatabase layer:

# XYTableToPoint.py
# Description: Creates a point feature class from input table

# import system modules
import arcpy

# Set environment settings
arcpy.env.workspace = r"C:\Projects\points\test.gdb"

# Set the local variables
in_table = r"D:\points\split_data\file_1.txt"
out_feature_class = "file1"
x_coords = "field7"
y_coords = "field6"
z_coords = ""

# Make the XY event layer...
arcpy.management.XYTableToPoint(in_table, out_feature_class,x_coords, y_coords, z_coords,
arcpy.SpatialReference(4326))

# Print the total rows
print(arcpy.GetCount_management(out_feature_class))

arcpy.management.Append("file1", r"C:\Projects\points\test.gdb\all_Points", "NO_TEST", 'id "id" true true false 25 Text 0 0,First,#,file1,Field1,0,8000;individualCount "individualCount" true true false 15 Text 0 0,First,#,file1,Field2,0,8000;year "year" true true false 4 Long 0 0,First,#,file1,Field3,-1,-1;month "month" true true false 2 Short 0 0,First,#,file1,Field4,-1,-1;country "country" true true false 255 Text 0 0,First,#,file1,Field5,0,8000;decimalLatitude "decimalLatitude" true true false 8 Double 0 0,First,#,file1,Field6,-1,-1;decimalLongitude "decimalLongitude" true true false 8 Double 0 0,First,#,file1,Field7,-1,-1;Name "Name" true true false 100 Text 0 0,First,#,file1,Field8,0,8000', '', '')

This works fine but this would be tedious to do edit and rerun manually for each file (potentially over a 1000 files).

Any advice most welcome.

Mark

[George_Thompson]

Not Python so to speak but another option would be to use the LocateXT functionality within ArcGIS Pro; What is LocateXT—ArcGIS Help | ArcGIS Desktop 

That may be easier to adjust. There is also more functionality coming out with the next release at ArcGIS Pro 2.5.

[MarkBalman]

Thank you George,

Not tried Locate yet but will have a look when I have some spare time.

Best regards

Mark

[MarkBalman]

Hi George, Locate looks interesting but sadly we don't have a license for it.

Best wishes,

Mark

[JoshuaBixby]

Python has several ways to find TXT files on a file system:  Find all files in a directory with extension .txt in Python - Stack Overflow 

[MarkBalman]

Hi Joshua

Thanks for the suggestions, will try these once I have more time to spend on the task

Best regards

Mark

[LorneDmitruk2]

Use the arcpy.da.Walk function to loop through the files.

[MarkBalman]

Hi Lorne

Thank you for the suggestion. Am quite new to python so have yet to try this method

Best regards

Mark

[DavidPike · Accepted solution]

import arcpy, os

# Set environment settings
arcpy.env.workspace = r"C:\Projects\points\test.gdb"

# Set the local variables
in_folder = whatever your folder is
merged_fc = name of final merged fc to be created in gdb

x_coords = "field7"
y_coords = "field6"
z_coords = ""


#grab all the text files and put them in  a list
text_file_list = [ ]

for file in os.listdir(in_folder):
    if ".txt" in file:
        text_file_list.append(file)

# Make the point feature classes
path_list = [ ]
for text_file in text_file_list:
    in_table = os.path.join(in_folder,text_file)
    name = text_file.strip(".txt")
    out_fc = os.path.join("in_memory",name)
    path_list.append(out_fc)
    arcpy.management.XYTableToPoint(in_table, out_fc,x_coords, y_coords, z_coords, arcpy.SpatialReference(4326))

arcpy.Merge(path_list,merged_fc)

arcpy.Delete_management("in_memory")

‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[MarkBalman]

Thank you David,

I tried this but got error message which I am trying to work out (will be trying again next week)

Best regards,

Mark