Convert multiple csv (specific date range) to shp using ArcPy

[ShouvikJha]

I have multiples CSV files. Every CSV file contains 365 days of different weather parameters for different location. I want to take specific month from every csv column and export it to point shp file using ArcPy. Can you help me to do it using ArcPy. In csv date format is (MM DD YY)

Below i have attached my python code. This code giving me error while i running date specific date range. Below i have attached my csv file also.

import arcpy,os

... shpworkspace = r"D:\TRY-Python"

... arcpy.env.workspace = shpworkspace

... arcpy.env.overwriteOutput = True

...

... csvlist = arcpy.ListFiles("*.csv")

...

... for csvfile in csvlist:

...     print csvfile

...     outlayer = "CSVEventLayer"

...     spatialreference = "GEOGCS['GCS_WGS_1984',DATUM['D_WGS_1984',SPHEROID['WGS_1984',6378137.0,298.257223563]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]];-400 -400 1000000000;-100000 10000;-100000 10000;8.98315284119522E-09;0.001;0.001;IsHighPrecision"

...     arcpy.MakeXYEventLayer_management(csvfile,"Longitude","Latitude",outlayer,spatialreference,"#")

...     arcpy.SelectLayerByAttribute_management(outlayer, Date 1/2/2013-1/31/2013, "#")

...     shpfile = os.path.splitext(csvfile)[0].replace('-','_')       

...     arcpy.CopyFeatures_management(outlayer,shpfile)

...

...     del outlayer

[JakeSkinner]

One error I see is how you are referencing your mask shapefile:

mask="D:\Gujarta Shape file\GUJARATSTATE.shp"‍

should be:

mask=r"D:\Gujarta Shape file\GUJARATSTATE.shp"‍

[ShouvikJha]

Hi Jake, Thank you. i have replaced the line but still same issue. there is no output.

Actually earlier whatever CSV we converted to point shape for month wise, then we merged  it and save it in different month's folder. Now using above programme i just want to make a loop function that program can take automatically point shape file from different month folder to perform IDW, and  save it as same month folder along with month number and Z filed name  (e.g. For JANUARY: 001_Max_Temper, 001_Min_Temper, 001_Wind, 001_Precipitat, 002_Relative_H, 001_Solar, For FEBRUARY:  002_Max_Temper, 002_Min_Temper, 002_Wind, 002_Precipitat, 002_Relative_H, 002_Solar)

Below i have  attached a reference vector. Kindly find it. 

[JakeSkinner]

Below is what I was able to get working.  A couple notes:

1.  Once you set your arcpy.env.workspace, you won't have to specify the full path to feature classes/shapefiles/rasters if they exist in that directory.  You will just need to reference the feature's name.

2.  It's best not to begin a feature class/shapefile/raster with a number.

import arcpy, os, calendar
from arcpy import env
from arcpy.sa import *
arcpy.CheckOutExtension("Spatial")
topWorkspace = r'D:\Temp\Python\UserData\Reference-Vector'
env.workspace = topWorkspace
env.overwriteOutput = True

#Mask region of interest
mask=r"D:\Temp\Python\UserData\Shape_File\Boundary.shp"

 # Get dict of months and month-number (i.e. January = 001, March = 003 etc.)
months = {calendar.month_name[i].upper(): str(i).zfill(3) for i in range(1, 13)} # Get dict of months and month number (i.e. January = 001, March = 003 etc.)

 # Step through list of all folders
for folderPath in arcpy.ListWorkspaces():
    baseName = os.path.basename(folderPath).upper()
    if baseName in months: # Test that subfolder is a month name
        monthNumber = months[baseName] # Get month-number for use in output filename
    arcpy.env.workspace = folderPath

    #Loop every month folder ta take shape file from each month (e.g. Merged_001 (JANUARY), Merged_002 (FEBRUARY)....Merged_012 (DECEMBER))
    shapeFile = (arcpy.ListFeatureClasses('Merged_*.shp')[0])

    zFields = ["Max_Temper", "Min_Temper", "Precipitat", "Wind", "Relative_H", "Solar"]
    # IDW variable
    cellSize = 0.05
    power = 2
    searchRadius = RadiusVariable(10, 150000)

    # Execute IDW
    for zField in zFields:
        outIDW = Idw(shapeFile, zField, cellSize, power, searchRadius)

        # Execute Mask
        IDWMASk = ExtractByMask(outIDW, mask)

        if not zField == "Solar":
            # Save output, except Solar Radiation raster (e.g for JANUARY:001_Max_Temper, FEBRUARY:002_Max_Temper)
            out_name_IDW = zField + "_" + str(monthNumber) + ".tif"
            IDWMASk.save(out_name_IDW)
        else:
            #Only for Solar Radiation raster
            PAR = (IDWMASk * 30) * 0.5
            #Save output, Only for Solar Radiation raster
            out_name_IDW = zField + "_" + str(monthNumber) + ".tif"
            PAR.save(out_name_IDW)

        #print done
        print 'Successfully created ' + str(out_name_IDW)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[ShouvikJha]

Jake Skinner‌. Thank you. I ran the code except SEPTEMBER month all months are calculating correctly. While it execute on month of SEPTEMBER, its showing error massage. 

Traceback (most recent call last):
  File "<module1>", line 23, in <module>
IndexError: list index out of range‍‍‍

[ShouvikJha]

jskinner-esristaff‌. I found the reason of error. Actually in between OCTOBER and SEPTEMBER, there were one another file. I deleted it and rerun the program. Now program runs perfectly. Thank you very much for your kind cooperation.  

[ShouvikJha]

Jake Skinner I am trying to calculate  some parameters (parameters mentioned in my code ) using same time span from diferent band  using ArcPy. 

I have list of raster, raster name like

 MOD09A1.MRTWEB.A2012001.005.sur_refl_b01.tif      

 MOD09A1.MRTWEB.A2012001.005.sur_refl_b02.tif      

 MOD09A1.MRTWEB.A2012001.005.sur_refl_b03.tif      

 MOD09A1.MRTWEB.A2012001.005.sur_refl_b04.tif      

 MOD09A1.MRTWEB.A2012001.005.sur_refl_b06.tif

 MOD09A1.MRTWEB.A2012009.005.sur_refl_b01.tif      

 MOD09A1.MRTWEB.A2012009.005.sur_refl_b02.tif      

 MOD09A1.MRTWEB.A2012009.005.sur_refl_b03.tif      

 MOD09A1.MRTWEB.A2012009.005.sur_refl_b04.tif      

 MOD09A1.MRTWEB.A2012009.005.sur_refl_b06.tif      

 MOD09A1.MRTWEB.A2012017.005.sur_refl_b01.tif      

 MOD09A1.MRTWEB.A2012017.005.sur_refl_b02.tif      

 MOD09A1.MRTWEB.A2012017.005.sur_refl_b03.tif      

 MOD09A1.MRTWEB.A2012017.005.sur_refl_b04.tif      

 MOD09A1.MRTWEB.A2012017.005.sur_refl_b06.tif 

In raster name different bands and  8 days interval time spam is there (e.g. A2012001, A2012009, A2012017……………A2012361) I trying to calculate some parameters for each time spam using different band and will loop same function on all time span (time spam 8 days interval: A2012001, A2012009, A2012017……………A2012361).

For instance, whatever raster  bands in 2012001 time span , will calculate different parameters , and save it 001_EVI, 001_LSWI, 001_PSCALAR, after calculation program will take  2012009 time span all bands and do same calculation and save it 002_EVI, 002_LSWI, 002_PSCALAR.

I have written some code. I think its not enough to do it. And I am not understanding how to loop 8 days interval time span to all raster band.

I have attached below, reference raster and raster name list.

def main():
    import arcpy
    from arcpy import env
    from arcpy.sa import *
    import os
    arcpy.env.overwriteOutput = True

    # Checkout extension
    arcpy.CheckOutExtension("Spatial")
    arcpy.env.overwriteOutput = True
    ws_in_mean = r'D:\NPP_GUJARAT\MODIS_DATA\MODIS_NDVI_2012\MASKED_2012C'
    ws_out_EVI= r'D:\MODIS_SURFACE_RF2012\EVI'
    ws_out_LSWI= r'D:\MODIS_SURFACE_RF2012\LSWI'
    ws_out_pscalar = r'D:\MODIS_SURFACE_RF2012\PSCALAR'

    # list "mean" rasters
    arcpy.env.workspace = ws_in_mean
    lst_ras_mean = arcpy.ListRasters()
    print "lst_ras_mean", lst_ras_mean

    for ras_name in lst_ras_mean:
        ras_num = ras_name[:3]

        ras_mean = arcpy.Raster(os.path.join(ws_in_mean, ras_name))
        print "ras_mean", ras_name
        #Setp:1
        # calculate EVI 2.5 * (band 01 - band 02) / ((band 02 + (6 * band 01) - (7.5 * band 03) + 1) (Min Raster Value )
        ras_EVI = 2.5 * (b01 - b02) / ((b02 + (6 * b01) - (7.5 * b03) + 1)

        # save the output raster for EVI (for 01, 001_EVI, 009_EVI so on.......for 361, 361 _EVI)
        out_name_EVI = os.path.join(ws_out_EVI, 'r{0}_EVI.TIF'.format(ras_num))
        ras_EVI.save(out_name_EVI)
        #Setp:2
        # Next calculate LSWI from BAND 2 and band 6
        ras_LSWI = (b02 - b01 ) / (b02 + b01)

        # save the output raster for LSWI (for 001, 001_LSWI, 009_LSWI so on.......for 361, 361 _LSWI)
        out_name_LSWI = os.path.join(ws_out_LSWI, 'r{0}_LSWI.TIF'.format(ras_num))
        ras_LSWI.save(out_name_LSWI)
        #Setp:3
        # Whatever raster we get from step 2, we just take each raster from step 2 and execute below euation
        ras_pscalar = 1 + ras_LSWI / 2

        # save the output raster for pscalar (for 001, 001_PSCALAR, for 009, 009_PSCALAR so on.......for 361, 361 _LSWI)
        out_name_pscalar = os.path.join(ws_out_pscalar, 'r{0}_PSCALAR.TIF'.format(ras_num))
        ras_pscalar.save(out_name_pscalar)

if __name__ == '__main__':
    main()‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[ShouvikJha]

Jake Skinner‌. I am still stuck to write above code. I am  not understanding how to execute 8 days time span loop  . Any  guidance would be greatly appreciated.