|
POST
|
Thanks Darren....quick and painless...I was trying this to the variable with no success...perfect....Cheers
... View more
10-06-2016
10:52 AM
|
0
|
0
|
2206
|
|
POST
|
I have a string that is being created from a listdir. It is Comma delimited... I want to write this to the txt file as separate lines. NOTING that each time I run this there will a different number in the list. So I cannot count on any indexing etc. What can I do to write this to separate lines one for each of the files located from the listdir? THANKS Code:
#...snip
dir_src = "C:\\users\\tjv36463\\Desktop\\BearCollar\\NewFiles\\"
from os import listdir
from os.path import isfile, join
onlyfiles = [f for f in listdir(dir_src) if isfile(join(dir_src, f))]
onlyfiles2 = str(onlyfiles)
file = open("log.txt", 'w')
file.write(onlyfiles2)
file.close()
#...snip
Result:
['rangedate_D032495_20160822124736.txt', 'rangedate_D032498_20160822125645.txt', 'rangedate_D032499_20160822125129.txt', 'rangedate_D032500_20160822125032.txt', 'rangedate_D032501_20160822125559.txt', 'rangedate_D032502_20160822125356.txt', 'rangedate_D033108_20160822125458.txt']
Im looking for:
['rangedate_D032495_20160822124736.txt',
'rangedate_D032498_20160822125645.txt',
'rangedate_D032499_20160822125129.txt',
'rangedate_D032500_20160822125032.txt',
'rangedate_D032501_20160822125559.txt',
'rangedate_D032502_20160822125356.txt',
'rangedate_D033108_20160822125458.txt']
... View more
10-06-2016
10:17 AM
|
0
|
5
|
2921
|
|
POST
|
OK this is the FINAL SCRIPT... I cant thank you all enough for you help... Reads through a Folder looking for CSV or txt files Reads through each CSV file Splits the comma delinted CSV file and write all the vales to variables A couple functions are called to process a couple of the variables to create new values for calculated fields that exist in the Feature Class but NOT the CSV file An if than statement populates a new field BearID based on a value from another field and adds this to the insert. Each row is inserted into the Feature Class Once completed the txt/csv files are moved to another folder for backup import arcpy
import datetime
import os, fnmatch
import shutil
# Function to modify two fields into a Time variable
def getdate(Hour, Minute):
return datetime.datetime.strptime(str(Hour).zfill(2) + ' ' + str(Minute).zfill(2), '%H %M')
#Function to loop through a directory and work through one file at a time
def findFiles (path, filter):
for root, dirs, files in os.walk(path):
for file in fnmatch.filter(files, filter):
yield os.path.join(root, file)
fc = r'C:\Users\tjv36463\Desktop\BearCollar\BearLocationImports.gdb\BearCollar'
#Insert cursor for the Bikes feature class
cursor = arcpy.da.InsertCursor (fc,["CollarSeri", "Year", "Julianday", "Hour", "Minute",
"Activity", "Temperatur", "Latitude", "Longitude", "HDOP", "NumSats", "FixTime",
"Date", "_2D_3D", "BearID", "CalcDate", "CalcDate3", "FileName", "SHAPE@XY"])
#Read through each line of the csv, but skip the first row because it is the header
#The count variable skips of the first row
for textFile in findFiles(r'C:/Users/tjv36463/Desktop/BearCollar/NewFiles/', '*.txt'):
#Read through each line of the csv, but skip the first row because it is the header
#The count variable skips of the first row
count = 0
for ln in open (textFile, 'r').readlines():
if count > 0:
#Each line in the csv is a string, so turn it into a list so you can reference
#each column
lineSplit = ln.split(",")
#Use index positions to get the right column from the csv
#Make sure the data is the correct type (i.e. X and Y need to be numbers,
#not strings)
CollarSeri = long(lineSplit[0])
Year = str(lineSplit[1])
Julianday = str(lineSplit[2])
Hour = int(lineSplit[3])
Minute = int(lineSplit[4])
Activity = int(lineSplit[5])
Temperature = int(lineSplit[6])
Latitude = float(lineSplit[7])
Longitude = float(lineSplit[8])
HDOP = str(lineSplit[9])
NumSats = int(lineSplit[10])
FixTime = int(lineSplit[11])
_2D_3D = int(lineSplit[12])
Date = str(lineSplit[13])
# convert the two time fields into one field to get 4:02:00PM format
# calls Function getDate at the top
getTime = getdate(Hour, Minute)
getTime2 = str(getTime)
getTime3 = getTime2[-8:]
# concatonate Date and Time to get 6/15/2016 4:02:00PM
getDateTime = (Date + getTime3)
# set variable for the File Name in which the record was take from
FileName = str(textFile)
#print (FileName)
if CollarSeri == 32488:
BearID = "1"
else:
if CollarSeri == 32491:
BearID = "2"
else:
if CollarSeri == 32495:
BearID = "3"
else:
if CollarSeri == 32498:
BearID = "4"
else:
if CollarSeri == 32500:
BearID = "5"
else:
if CollarSeri == 32502:
BearID = "6"
else:
if CollarSeri == 32501:
BearID = "7"
else:
if CollarSeri == 33108:
BearID = "8"
else:
if CollarSeri == 32499:
BearID = "9"
else:
if CollarSeri == 32488:
BearID = "10"
else:
if CollarSeri == 32496:
BearID = "11"
else:
BearID = "0"
shapeVal = (Longitude, Latitude)
#Insert the values from csv into feature class
#The order of the fields here, matches the order of the fields in line 4
cursor.insertRow ([CollarSeri, Year, Julianday, Hour, Minute, Activity,
Temperature, Latitude, Longitude, HDOP, NumSats, FixTime, Date,
_2D_3D, BearID, getTime3, getDateTime, FileName, shapeVal])
count += 1
del cursor
## make sure that these directories exist
# move the files to new location to clear the folder for the next batch of files
dir_src = "C:\\users\\tjv36463\\Desktop\\BearCollar\\NewFiles\\"
dir_dst = "C:\\users\\tjv36463\\Desktop\\BearCollar\\MovedFiles\\"
for file in os.listdir(dir_src):
#print file # testing
src_file = os.path.join(dir_src, file)
dst_file = os.path.join(dir_dst, file)
shutil.move(src_file, dst_file)
print ("Processing Finished")
... View more
10-06-2016
07:01 AM
|
2
|
0
|
7742
|
|
POST
|
Think I might have it...I move the Count=0 inside the For ... it was outside the first For for textFile in findFiles(r'C:/Users/tjv36463/Desktop/BearCollar/NewFiles/', '*.txt'): #print(textFile) count = 0 for ln in open (textFile, 'r').readlines(): if count > 0: #Each line in the csv is a string, so turn it into a list so you can reference each column lineSplit = ln.split(",")
... View more
10-05-2016
01:54 PM
|
0
|
1
|
3488
|
|
POST
|
I swear I had it there testing a few times...I dont like this indenting thing in python...lol...would rather close with an End If... THANK YOU SO very much...your help was fantastic...
... View more
10-05-2016
01:45 PM
|
0
|
0
|
3488
|
|
POST
|
Indented to where: the "for" or the "if"....I tried that and got errors that the field types are wrong...seems like its still reading the first line of the csv and seeing text...hmmm for ln in open (textFile, 'r').readlines(): if count > 0:
... View more
10-05-2016
12:44 PM
|
0
|
4
|
3488
|
|
POST
|
Actually didnt work again.... Can you look at my indents and see if you see something out of the ordinary? import arcpy import datetime import os, fnmatch import shutil def getdate(Hour, Minute): return datetime.datetime.strptime(str(Hour).zfill(2) + ' ' + str(Minute).zfill(2), '%H %M') def findFiles (path, filter): for root, dirs, files in os.walk(path): for file in fnmatch.filter(files, filter): yield os.path.join(root, file) fc = r'C:\Users\tjv36463\Desktop\BearCollar\BearLocationImports.gdb\BearCollar' #Insert cursor for the Bikes feature class cursor = arcpy.da.InsertCursor (fc,["CollarSeri", "Year", "Julianday", "Hour", "Minute", "Activity", "Temperatur", "Latitude", "Longitude", "HDOP", "NumSats", "FixTime", "Date", "_2D_3D", "BearID", "CalcDate", "CalcDate3", "FileName", "SHAPE@XY"]) #Read through each line of the csv, but skip the first row because it is the header #The count variable skips of the first row count = 0 for textFile in findFiles(r'C:/Users/tjv36463/Desktop/BearCollar/NewFiles/', '*.txt'): #print(textFile) for ln in open (textFile, 'r').readlines(): if count > 0: #Each line in the csv is a string, so turn it into a list so you can reference each column lineSplit = ln.split(",") #Use index positions to get the right column from the csv #Make sure the data is the correct type (i.e. X and Y need to be numbers, not strings) CollarSeri = long(lineSplit[0]) Year = str(lineSplit[1]) Julianday = str(lineSplit[2]) Hour = int(lineSplit[3]) Minute = int(lineSplit[4]) Activity = int(lineSplit[5]) Temperature = int(lineSplit[6]) Latitude = float(lineSplit[7]) Longitude = float(lineSplit[8]) HDOP = str(lineSplit[9]) NumSats = int(lineSplit[10]) FixTime = int(lineSplit[11]) _2D_3D = int(lineSplit[12]) Date = str(lineSplit[13]) getTime = getdate(Hour, Minute) getTime2 = str(getTime) getTime3 = getTime2[-8:] getDateTime = (Date + getTime3) FileName = str(textFile) #print (FileName) shapeVal = (Longitude, Latitude) #Insert the values from csv into feature class #The order of the fields here, matches the order of the fields in line 4 cursor.insertRow ([CollarSeri, Year, Julianday, Hour, Minute, Activity, Temperature, Latitude, Longitude, HDOP, NumSats, FixTime, Date, _2D_3D, BearID, getTime3, getDateTime, FileName, shapeVal]) count += 1 del cursor print ("Processing Finished")
... View more
10-05-2016
09:58 AM
|
0
|
6
|
2644
|
|
POST
|
Dude cant thank you enough for all your help…it is so gratefully appreciated. Thanks Jay Jay Kapalczynski GIS Coordinator - Virginia Department of Game & Inland Fisheries (new address) 7870 Villa Park Drive Suite 400, Henrico VA 23228 Phone: 804.367.6796 | Fax: 804.367.2628 ü Please consider the environment before printing this email.
... View more
10-04-2016
06:40 PM
|
0
|
0
|
2365
|
|
POST
|
Mitch Holley, Darren Wiens THANK YOU FOR YOUR HELP AND SUPPORT....Things are running great...although not the most efficient way but I got it...with YOUR help of course ANY QUESTIONS: please feel free to reply and I will get back with you.. THANKS Again Mitch and Darren....SOOOO very much appreciated. 1. Read CSV to file to get all records 2. I calculated the Time field (CalcTime) from an Hour and Minute Field.. 12:02:00 AM 3. Once I had that I calculated the Full CalcDateTime field from CalcTime above and the Date field from the CSV 6/29/2016 12:02:00 AM 4. Inserted ROWS into the Feature Class using SHAPE@XY Token to create the geometry Hour Minute from CSV File 0 2 getTime3 Variable - CalcTime Field Result in New Feature Class 12:02:00 AM getDateTime Variable - CalcDateTime field result in the New Feature Class concatenated with the Date field from the CSV File 6/29/2016 12:02:00 AM FULL CODE BELOW import arcpy import datetime def getdate(Hour, Minute): return datetime.datetime.strptime(str(Hour).zfill(2) + ' ' + str(Minute).zfill(2), '%H %M') fc = r'C:\Users\\Collar\BearLocationImports.gdb\BearCollar' cursor = arcpy.da.InsertCursor (fc,["CollarSeri", "Year", "Julianday", "Hour", "Minute", "Activity", "Temperatur", "Latitude", "Longitude", "HDOP", "NumSats", "FixTime", "Date", "_2D_3D", "BearID", "CalcDate", "CalcDate3", "SHAPE@XY"]) for ln in open (r"C:\Users\tjv36463\Desktop\Bear Collar\rangedate_D032495_20160822124736_NH.txt", 'r').readlines(): #Each line in the csv is a string, so turn it into a list so you can reference each column lineSplit = ln.split(",") #Use index positions to get the right column from the csv #Make sure the data is the correct type (i.e. X and Y need to be numbers, not strings) CollarSeri = long(lineSplit[0]) Year = str(lineSplit[1]) Julianday = str(lineSplit[2]) Hour = int(lineSplit[3]) Minute = int(lineSplit[4]) Activity = int(lineSplit[5]) Temperature = int(lineSplit[6]) Latitude = float(lineSplit[7]) Longitude = float(lineSplit[8]) HDOP = str(lineSplit[9]) NumSats = int(lineSplit[10]) FixTime = int(lineSplit[11]) _2D_3D = int(lineSplit[12]) BearID = 0 Date = str(lineSplit[13]) # working with Date Time http://stackoverflow.com/questions/1521906/how-to-specify-date-and-time-in-python getTime = getdate(Hour, Minute) getTime2 = str(getTime) getTime3 = getTime2[-8:] getDateTime = (Date + getTime3) print getDateTime shapeVal = (Longitude, Latitude) #Insert the values from csv into feature class cursor.insertRow ([CollarSeri, Year, Julianday, Hour, Minute, Activity, Temperature, Latitude, Longitude, HDOP, NumSats, FixTime, Date, _2D_3D, BearID, getTime3, getDateTime, shapeVal]) count += 1 del cursor print ("done")
... View more
10-04-2016
06:29 PM
|
0
|
0
|
7857
|
|
POST
|
I did it like this but dont think this is the best way.... getDateTime = getdate(Hour, Minute) getDateTime2 = str(getDateTime) getDateTime3 = getDateTime2[-8:] # send getDateTime3 to the insert....came out alright...but lots of processing....
... View more
10-04-2016
06:03 PM
|
0
|
0
|
2365
|
|
POST
|
If I could get that one last step to only time I would be done...I cant thank you enough for this help...newbee is very grateful.
... View more
10-04-2016
05:55 PM
|
0
|
0
|
2365
|
|
POST
|
That did it....when I wrote to my GDB field it was in normal time and had the AM PM on it....with my lack of knowledge here I think I might just strip the characters(Date) from the front of the variable and try that....I hate dates.... Unless you have an example of the .strftime
... View more
10-04-2016
05:46 PM
|
0
|
3
|
5492
|
|
POST
|
When I use my example above why do I still get military time and no AM PM...I can deal with the date at the front...just want the time to at least come out right. Im gonna write this to a GDB field and see if it differs.... THANKS for you help....very much appreciated.
... View more
10-04-2016
05:40 PM
|
0
|
4
|
5492
|
|
POST
|
Think I got it....one second... 1. I still get military time and missing the AM PM 2. Only looking for the time not the date on the front... Not really....did this and got this..but getting closer... If I use this I get error: getDateTime = getdate(!Hour!, !Minute!) RESULT: 1900-01-01 20:01:00 1900-01-01 00:01:00 1900-01-01 04:01:00 1900-01-01 12:02:00 1900-01-01 16:01:00 1900-01-01 20:01:00 CODE: import arcpy import datetime def getdate(hour, minute): return datetime.datetime.strptime(str(Hour).zfill(2) + ' ' + str(Minute).zfill(2), '%H %M') fc = r'C:\Users\tjv36463\Desktop\Bear Collar\BearLocationImports.gdb\BearCollar' cursor = arcpy.da.InsertCursor (fc,["CollarSeri", "Year", "Julianday", "Hour", "Minute", "Activity", "Temperatur", "Latitude", "Longitude", "HDOP", "NumSats", "FixTime", "Date", "_2D_3D", "BearID", "SHAPE@XY"]) for ln in open (r"C:\Users\tjv36463\Desktop\Bear Collar\rangedate_D032495_20160822124736_NH.txt", 'r').readlines(): lineSplit = ln.split(",") Hour = int(lineSplit[3]) Minute = int(lineSplit[4]) getDateTime = getdate(Hour, Minute) print getDateTime
... View more
10-04-2016
05:01 PM
|
1
|
9
|
5492
|
|
POST
|
This is how I am going through the CSV and assigning the variables for each record....I then write to the Feature Class. for ln in open (r"C:\Users\tjv36463\Desktop\Bear Collar\rangedate_D032495_20160822124736_NH.txt", 'r').readlines(): lineSplit = ln.split(",") Hour = int(lineSplit[3]) Minute = int(lineSplit[4]) CalcDate1 = Right('00' & Hour,2) & ':' & Right('00' & Minute,2) In ArcGIS using calculate on a date field with this: Right('00' & [Hour],2) & ':' & Right('00' & [Minute],2) I get this: 4:00:00 PM it the Values were 16 for hour and 2 for Minute Just need to translate this to Python Edit: Looks like ArcGIS is converting the Military time to normal time....think I need to add this into Python or is there a date handler of some sort to do that conversion?
... View more
10-04-2016
12:41 PM
|
0
|
0
|
5492
|
| Title | Kudos | Posted |
|---|---|---|
| 1 | 09-20-2018 11:09 AM | |
| 1 | 09-10-2018 06:26 AM | |
| 1 | 09-15-2022 11:02 AM | |
| 1 | 05-21-2021 07:35 AM | |
| 1 | 08-09-2022 12:39 PM |
| Online Status |
Offline
|
| Date Last Visited |
09-19-2022
09:23 PM
|