topic Re: ArcPy Error: Date' is not in list, While extracting monthly data from timeseries NetCDF in Python Questions

ArcPy Error: Date' is not in list, While extracting monthly data from timeseries NetCDF

ShouvikJha — Sun, 12 Dec 2021 02:07:02 GMT

Hi All,

I have one NetCDF file, that contain monthly value for the different year (1910 - 2014). I want to extract few years data (e.g 2000 - 2014), date range reading from a csv file and save it as a raster map as the name format of year-month-date. I am working on below script but I am getting error

Below i have attached the csv file which i am using for date range purpose.

Any help would be greatly appreciated! Thanks

The image showing the format of Year-month-date

Error massage

Traceback (most recent call last):
  File "<module4>", line 18, in <module>
ValueError: 'Date' is not in list‍‍‍‍‍‍

Here what i have so far

import os, sys
import arcpy

# Check out any necessary licenses
arcpy.CheckOutExtension("spatial")
arcpy.env.overwriteOutput = True

# Script arguments
netCDF = "C:\\Users\\Downloads\\SPIE"
rainfall = "C:\\Users\\Downloads\\SPIE\\New folder"

arcpy.env.workspace = netCDF

# Read Date from csv file
eveDate = open ("C:\\Users\\Downloads\\SPIE\\Name1.csv")
headerLine = eveDate.readline()
valueList = headerLine.split(",")
dateValueIndex = valueList.index("Date")
eventList = []
for line in eveDate.readlines():
    segmenLine = line.split(",")
    variable = "spei"
    x_dimension = "lon"
    y_dimension = "lat"
    band_dimension = ""

for year in range(2010):
    nc_name = 'spei.%d.nc' % (year,)
    valueSelectionMethod = "BY_VALUE"
    outFile = "Pre"
    # extract dimensionValues from csv file
    arcpy.MakeNetCDFRasterLayer_md("spei.nc", variable, x_dimension, y_dimension, outFile, band_dimension, segmenLine[dateValueIndex], valueSelectionMethod)
    print "layer done"
    #copy and save as raster tif file
    arcpy.CopyRaster_management(outFile, rainfall + segmenLine[dateValueIndex] + ".tif" , "", "", "", "NONE", "NONE", "")
    print "raster done"‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: ArcPy Error: Date' is not in list, While extracting monthly data from timeseries NetCDF

JoshuaBixby — Tue, 07 Mar 2017 14:39:02 GMT

The CSV file you attached has one column, Date. When you do readline, it is including the newline character. So, what you have is ['Date\n'] and not ['Date'], hence the value error.

Re: ArcPy Error: Date' is not in list, While extracting monthly data from timeseries NetCDF

ShouvikJha — Sun, 12 Dec 2021 02:07:05 GMT

@joshua Bixby, Thank you. I have corrected my csv file, as replaced in line series all the date value, after the correction of my csv file , script is running without any error but i am getting only single output as newfolder name. Now what should i do to loop through all the date range which i have in csv file

I have mentioned date range in csv file, like.

Date     2010-01-16     2010-02-16     2010-03-16     2010-04-16     2010-05-16     2010-06-16     2010-07-16     2010-08-16     2010-09-16     2010-10-16     2010-11-16     2010-12-16
‍

Re: ArcPy Error: Date' is not in list, While extracting monthly data from timeseries NetCDF

JoshuaBixby — Tue, 07 Mar 2017 15:20:22 GMT

Not sure this will solve your issue, but there are a couple other items that caught my eye. One, the loop you start on Line 20. You are looping through a file to read values, but you are not storing all the values, just the last one read. The way the code is written, segmenLine will only be the last Date value in the CSV file.

Second, I am not sure what you are attempting to do with the loop that starts on Line 27. Range(2010) will return a list containing 2010 values starting at 0 and ending at 2009, is that really what you want?

Re: ArcPy Error: Date' is not in list, While extracting monthly data from timeseries NetCDF

ShouvikJha — Tue, 07 Mar 2017 15:53:19 GMT

Actually i am trying to extract one year data from time series netCDF file. My NetCDF file contain monthly data from 1901-2014. So i need to extract the monthly data for different year and make it as raster map.

And i removed the second loop, line no 27 and 28, but problem is same. Now also i am getting the single output as name of output "newfolde". Why such type of output and output's name occurring?. Please guide us to solve this issue.

Why this script not looping all the date value and not make any raster which i have in csv file?

Re: ArcPy Error: Date' is not in list, While extracting monthly data from timeseries NetCDF

JoshuaBixby — Tue, 07 Mar 2017 17:16:08 GMT

Please repost the code as it appears now, since the original line numbers won't line up since you have removed code.

Re: ArcPy Error: Date' is not in list, While extracting monthly data from timeseries NetCDF

ShouvikJha — Sun, 12 Dec 2021 02:07:07 GMT

Hi @joshua Bixby, I am reposting my code along with my NetCDF data file which link is given below. Because of overlimit i could not attach NetCDF file directly,

NetCDF file link : Dropbox - spei01.nc

When i running below script, there is only single raster generating as name newfolder

import os, sys
import arcpy

# Check out any necessary licenses
arcpy.CheckOutExtension("spatial")
arcpy.env.overwriteOutput = True

# Script arguments
netCDF = "C:\\Users\\Downloads\\SPIE"
rainfall = "C:\\Users\\Downloads\\SPIE\\New folder"

arcpy.env.workspace = netCDF

# Read Date from csv file
eveDate = open ("C:\\Users\\Downloads\\SPIE\\name1.csv", "r")
headerLine = eveDate.readline()
valueList = headerLine.split(",")
dateValueIndex = valueList.index("Date")
eventList = []
for line in eveDate.readlines():
    segmenLine = line.split(",")
    variable = "spei"
    x_dimension = "lon"
    y_dimension = "lat"
    band_dimension = ""

    valueSelectionMethod = "BY_VALUE"
    outFile = "Pre"
    # extract dimensionValues from csv file
    arcpy.MakeNetCDFRasterLayer_md("spei01.nc", variable, x_dimension, y_dimension, outFile, band_dimension, segmenLine[dateValueIndex], valueSelectionMethod)
    print "layer done"
    #copy and save as raster tif file
    arcpy.CopyRaster_management(outFile, rainfall + segmenLine[dateValueIndex] + ".tif" , "", "", "", "NONE", "NONE", "")
    print "raster done"‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: ArcPy Error: Date' is not in list, While extracting monthly data from timeseries NetCDF

ShouvikJha — Wed, 08 Mar 2017 05:58:12 GMT

Hi All, Please provide your suggestion to solve this issue. i still stuck

Re: ArcPy Error: Date' is not in list, While extracting monthly data from timeseries NetCDF

DanPatterson_Retired — Wed, 08 Mar 2017 09:35:59 GMT

throw a print statement in and show what

rainfall + segmenLine[dateValueIndex] + ".tif"

this reveals. Also, why do you have a folder called "New Folder" ? spaces aren't great for most things just to be on the safe side

Re: ArcPy Error: Date' is not in list, While extracting monthly data from timeseries NetCDF

ShouvikJha — Sun, 12 Dec 2021 02:07:10 GMT

Dan Patterson‌, @Joshus, I have made some changes in the script, rather than using CSV for extracting particular time period, i am using below script to extract desired time period raster from NetCDF, here line no 14 controlling the time period,

if int(nowFile[-2]) > 6

In above line > 6 is extracting the all year data after 1988, when i increase the value by 1 then extracting year also sifting by1 year (e.g. If i put 7 then it would be extract all the year of 1989).

But i want to make a condition for extracting the specific year, e.g1985 to 1990, i don't want pre and post data of that year.

Any assist would be greatly appreciated. Thanks

nc_FP = arcpy.NetCDFFileProperties(inNetCDF)
print nc_FP
nc_Dim = nc_FP.getDimensions()

for dimension in nc_Dim:
        if dimension == "time":
            top = nc_FP.getDimensionSize(dimension)
            for i in range(0, top):

                dimension_values = nc_FP.getDimensionValue(dimension, i)
                nowFile = str(dimension_values)
                nowFile = nowFile.translate(None, '/')
                # I needed only the years 1990, post 1990 data i don't want 
                if int(nowFile[-2]) > 8:

                    dv1 = ["time", dimension_values]
                    dimension_values = [dv1]

                    arcpy.MakeNetCDFRasterLayer_md(inNetCDF, variable, x_dimension, y_dimension, nowFile, band_dimension, dimension_values, valueSelectionMethod)
                    print "success"
                    outname = outLoc + nowFile
                    arcpy.CopyRaster_management(nowFile, outname + ".tif", "", "", "", "NONE", "NONE", "")


                else: print "Enter a Valid year"‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: ArcPy Error: Date' is not in list, While extracting monthly data from timeseries NetCDF

DanPatterson_Retired — Sun, 12 Dec 2021 02:07:13 GMT

you should be using colons when you are parsing, inyour case, you want the last two characters and you will be doing an equality check

>>> nowfile = '1989'
>>> whatisit = int(nowfile[-2])
>>> whatisit
8
>>> howabout = int(nowfile[:-2])
>>> howabout == int(nowfile[:-2])
True
>>>‍‍‍‍‍‍‍‍

Re: ArcPy Error: Date' is not in list, While extracting monthly data from timeseries NetCDF

ShouvikJha — Sun, 12 Dec 2021 02:07:16 GMT

Dan Patterson‌ sorry for late response. I was out of station.

Here i tried to follow your suggestion. after assign the year (2002), its create only one raster not extracting all month data in 2002.

dimension_values = nc_FP.getDimensionValue(dimension, i)
                nowFile = str(dimension_values)
                nowFile = nowFile.translate(None, '/')
                nowFile = ('2002')
                # I needed only the years 2002
                if int(nowFile[:-2]):‍‍‍‍‍‍

Re: ArcPy Error: Date' is not in list, While extracting monthly data from timeseries NetCDF

ShouvikJha — Thu, 23 Mar 2017 18:48:04 GMT

Dan Patterson‌, Thank you. I just solved the issue, using below command

if int(nowFile[-4:]) == 2002:‍‍

Thank you all for your kind cooperation.

Re: ArcPy Error: Date' is not in list, While extracting monthly data from timeseries NetCDF

DanPatterson_Retired — Thu, 23 Mar 2017 22:05:49 GMT

good, now you know how slicing works