Convert multiple csv (specific date range) to shp using ArcPy

[ShouvikJha]

I have multiples CSV files. Every CSV file contains 365 days of different weather parameters for different location. I want to take specific month from every csv column and export it to point shp file using ArcPy. Can you help me to do it using ArcPy. In csv date format is (MM DD YY)

Below i have attached my python code. This code giving me error while i running date specific date range. Below i have attached my csv file also.

import arcpy,os

... shpworkspace = r"D:\TRY-Python"

... arcpy.env.workspace = shpworkspace

... arcpy.env.overwriteOutput = True

...

... csvlist = arcpy.ListFiles("*.csv")

...

... for csvfile in csvlist:

...     print csvfile

...     outlayer = "CSVEventLayer"

...     spatialreference = "GEOGCS['GCS_WGS_1984',DATUM['D_WGS_1984',SPHEROID['WGS_1984',6378137.0,298.257223563]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]];-400 -400 1000000000;-100000 10000;-100000 10000;8.98315284119522E-09;0.001;0.001;IsHighPrecision"

...     arcpy.MakeXYEventLayer_management(csvfile,"Longitude","Latitude",outlayer,spatialreference,"#")

...     arcpy.SelectLayerByAttribute_management(outlayer, Date 1/2/2013-1/31/2013, "#")

...     shpfile = os.path.splitext(csvfile)[0].replace('-','_')       

...     arcpy.CopyFeatures_management(outlayer,shpfile)

...

...     del outlayer

[JakeSkinner · Accepted solution]

Hi Shouvik,

You will need an OBJECTID field in order to perform the Select Layer By Attribute.  I updated the code to import the CSV file into a scratch geodatabase since the CSV does not have a unique ID field.  Doing so will add the OBJECTID field.  Also, I updated the syntax to query the date.  See below:

import arcpy,os

shpworkspace = r"D:\TRY-Python"
arcpy.env.workspace = shpworkspace
arcpy.env.overwriteOutput = True
scratchGDB = arcpy.env.scratchGDB

csvlist = arcpy.ListFiles("*.csv")


for csvfile in csvlist:
    print csvfile
    arcpy.TableToTable_conversion(csvfile, scratchGDB, "CSV")
    outLayer = "CSVEventLayer"
    spatialreference = arcpy.SpatialReference(4326)
    arcpy.MakeXYEventLayer_management(os.path.join(scratchGDB, "CSV"),"Longitude","Latitude",outLayer,spatialreference,"#")
    arcpy.SelectLayerByAttribute_management(outLayer, "NEW_SELECTION", "Date > date '2013-01-02' AND Date < date '2013-01-31'")
    shpfile = os.path.splitext(csvfile)[0].replace('-','_')
    arcpy.CopyFeatures_management(outLayer,shpfile)

del outLayer

[ShouvikJha]

Thank you for your suggestion. I have run the code. There is still error

Runtime error

Traceback (most recent call last):

  File "<string>", line 11, in <module>

TypeError: 'NoneType' object is not iterable

[JakeSkinner]

This error is usually an indication that there are no CSV files in the following directory: "D:\TRY-Python".  I would double-check to make sure there is a CSV file located there.
 

[ShouvikJha]

@Jake Skinner Thank you very much sir. Your code is working fine. Once again thank you very much.

[ShouvikJha]

jskinner-esristaff Can we modify above code for extracting different months separately (e.g Above code generating single feature class based on input date range but instead of generate single feature class i want generate month wise different feature class and save it into different month folder created by python script using same above input data). I have to analyze monthly profile of weather data therefor i want to create separate feature class for each month.  

[JakeSkinner]

Try the following:

import arcpy,os

shpworkspace = r"D:\temp\python\userdata"
arcpy.env.workspace = shpworkspace
arcpy.env.overwriteOutput = True
scratchGDB = arcpy.env.scratchGDB

csvlist = arcpy.ListFiles("*.csv")

# Create dictionary of months
dict = {"JANUARY": "01", "FEBRUARY": "02", "MARCH": "03", "APRIL": "04", "MAY": "05", "JUNE": "06", "JULY": "07", "AUGUST": "08", "SEPTEMBER": "09", "OCTOBER": "10", "NOVEMBER": "11", "DECEMBER": "12"}

for csvfile in csvlist:
    print csvfile
    arcpy.TableToTable_conversion(csvfile, scratchGDB, "CSV")
    outLayer = "CSVEventLayer"
    spatialreference = arcpy.SpatialReference(4326)
    # Create XY Event Layer
    arcpy.MakeXYEventLayer_management(os.path.join(scratchGDB, "CSV"),"Longitude","Latitude",outLayer,spatialreference,"#")
    # Iterate through dictionary
    for key in dict:
        # Create workspace for each month
        workspace = os.path.join(shpworkspace, key)
        if not os.path.exists(workspace):
            os.makedirs(workspace)

        month = str(dict[key])

        # Create empty tuple
        OIDs = ()

        # Obtain each OID of all features for each month
        with arcpy.da.SearchCursor(os.path.join(scratchGDB, "CSV"), ["OID@", "Date"]) as cursor:
            for row in cursor:
                stringDate = str(row[1])
                month = stringDate.split("-")[1]
                if str(dict[key]) in month:
                    # Add each OID to tuple
                    OIDs = OIDs + (row[0],)

        # Select features using tuple
        arcpy.SelectLayerByAttribute_management(outLayer, "NEW_SELECTION", "OBJECTID IN " + str(OIDs))

        # Copy to shapefile
        csvFileName =  os.path.splitext(csvfile)[0].replace('-','_')
        shpfile = os.path.join(workspace, "Weather_" + csvFileName + ".shp")
        arcpy.CopyFeatures_management(outLayer,shpfile)

del outLayer‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[ShouvikJha]

Thanks Jake Skinner‌. Code is working perfectly. Once again thank you very much for the cooperation.  

[ShouvikJha]

Jake Skinner I  have run the code, there is nothing error with this code while i run it on single CSV file but when i run it on 514 CSVs file then Arc GIS suddenly gets shut down (''Arcgis desktop has encountered a serious application error and is unable to continue'') while the time of program execution into 45 number CSVs file.

I have total 514 CSVs file. Earlier which CSV file I had sent for reference purpose that is one of my total CSV file.  So upto 44 CSVs program run correctly but when program finished 44 number CSVs conversion than automatically Arc GIS gets shut down. I separately run 45 number CSVs file, but I did not find any error with this file. But when I execute 514 CSVs file at same time then after converting 44 CSVs Arc GIS automatically shut down.

I am using Arc GIS 10.1, on windows 7 32 bit platform. Below I have attached a video file where i pointed the CSV file of error occurrence and  Arc GIS error massage and Kindly find attached CSV file also. 

I have checked this code on another computers also but there also same issue arising.  

[JakeSkinner]

Try running the script outside of ArcMap using an IDE such as PyScripter.  I ran the script using this with ArcGIS Desktop 10.4.1 successfully.