topic Re: Extracting rasters from NetCDF files based on time dimension in Python Questions

Extracting rasters from NetCDF files based on time dimension

ShouvikJha — Sat, 11 Dec 2021 14:25:45 GMT

I have single netCDF file which contain monthly data for different year. (Format of date e.g 2001-1-1, 2001-2-1, 2001-3-1…….2002-12-1...............2015-0-1......2015-12-1).

Here-with i have attached a snapshot of data format from Arc GIS multidimension tool

I trying to extract each month raster for selected year (e.g only 2002). Save it as 01-2002, 02-2002, 03-2002…..12-2002.

I appreciate any help you can provide on the crucial bottom code.

Here is what I have so far:

# Import system modules
import arcpy
from arcpy import env
from arcpy.sa import *

#Workspace
outLoc = r"D:\MODIS_NPP_02_15"
inNetCDF = r"D:\MODIS_NPP_02_15\NPP_2001_15.nc"

#Veriable
variable = "npp"
x_dimension = "lon"
y_dimension = "lat"
band_dimension = ""
dimension = "time"
valueSelectionMethod = "BY_VALUE"

nc_FP = arcpy.NetCDFFileProperties(inNetCDF)
nc_Dim = nc_FP.getDimensions()

for dimension in nc_Dim:
        if dimension == "time":
            top = nc_FP.getDimensionSize(dimension)
            for i in range(0, top):

                dimension_values = nc_FP.getDimensionValue(dimension, i)
                nowFile = str(dimension_values)
                nowFile = nowFile.translate(None, '/')
                # I needed only the years 2002
                if int(nowFile[-12]) > 6:

                    dv1 = ["time", dimension_values]
                    dimension_values = [dv1]

                    arcpy.MakeNetCDFRasterLayer_md(inNetCDF, variable, x_dimension, y_dimension, nowFile, band_dimension, dimension_values, valueSelectionMethod)
                    print "success"
                    outname = outLoc + nowFile

                   arcpy.CopyRaster_management(nowFile, outname, +".tif","", "", "", "NONE", "NONE", "")


                else: print "DATA OUT OF RANGE"‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Extracting rasters from NetCDF files based on time dimension

NeilAyres — Mon, 12 Sep 2016 06:43:18 GMT

So, what happens at the moment?

What errors are you getting or are you asking to validate this code prior to actually trying it?

I can see one problem with this :

arcpy.CopyRaster_management(nowFile, outname, +".tif","", "", "", "NONE", "NONE", "")

Shouldn't you add the "*.tif" to outname in the line above. Here the "*.tif" is the 3rd parameter. Which I am sure will not work.

Re: Extracting rasters from NetCDF files based on time dimension

ShouvikJha — Sat, 11 Dec 2021 14:25:47 GMT

Thank you. Output raster i am trying to save in tif format. I am getting following error massage.

Message File Name Line Position 
Traceback 
 <module> D:\Arc-GIS-Python-Script\NetCDF_RASTER_TIME_SERIES.py 30 
IndexError: string index out of range ‍‍‍‍

Re: Extracting rasters from NetCDF files based on time dimension

DanPatterson_Retired — Mon, 12 Sep 2016 07:42:39 GMT

you had better throw a print statement in there to ensure that the negative slice doe indeed return the proper formatted year prior to testing. you might want to split on "/" and return the last slice instead to get the year itself, rather than relying on a position count.

Re: Extracting rasters from NetCDF files based on time dimension

NeilAyres — Sat, 11 Dec 2021 14:25:50 GMT

Never dealt with this sort of data before, but that error is obviously your indexing of the value in Nowfile.

>>> n = "dngthrysdfg"
>>> n[-12]
Traceback (most recent call last):
 File "<pyshell#5>", line 1, in <module>
 n[-12]
IndexError: string index out of range
>>>‍‍‍‍‍‍‍

So, as Dan says, perhaps have a look at that variable before this step.

Re: Extracting rasters from NetCDF files based on time dimension

NeilAyres — Sat, 11 Dec 2021 14:25:53 GMT

And, not sure what that [-12] is doing.

>>> n = "fkghthsyfgtrekh"
>>> n[-12]
'h'
>>> n[:-12]
'fkg'‍‍‍‍‍

Re: Extracting rasters from NetCDF files based on time dimension

DanPatterson_Retired — Sat, 11 Dec 2021 14:25:55 GMT

>>> a ='abcdefghijklmnopqrstuvwxyz'
>>> a[:-12]        # doubt it
'abcdefghijklmn'
>>> a[-12:]        # this one perhaps???????????
'opqrstuvwxyz'
>>> a[-12]         # nope
'o'
‍‍‍‍‍‍‍

Re: Extracting rasters from NetCDF files based on time dimension

DanPatterson_Retired — Tue, 13 Sep 2016 21:26:38 GMT

Any more info? or is this closed?

Re: Extracting rasters from NetCDF files based on time dimension

ShouvikJha — Wed, 14 Sep 2016 02:45:59 GMT

Dan , Neil, i extremely sorry for the late reply, here holiday was going on . Problem still persist. by this command i am trying to extract selected time spam images.

n[:-12]

Re: Extracting rasters from NetCDF files based on time dimension

DanPatterson_Retired — Wed, 14 Sep 2016 02:56:38 GMT

so you did see my example and you definitely want the first 12 characters of the string (line 4 in my example) and not the last 12 (ie line 4 in my example). You need to print the string prior to extracting it so we know what is it is

Re: Extracting rasters from NetCDF files based on time dimension

ShouvikJha — Sat, 11 Dec 2021 14:25:58 GMT

Dan Patterson‌, I corrected the line by replace line no 4 suggested by you. Now i am getting following error massage. Error massage displaying output raster name can't be more than 13 character. So whats is the possible way to short the output raster name

Message     File Name     Line     Position     
Traceback                    
    <module>     D:\Arc-GIS-Python-Script\NetCDF_RASTER_TIME_SERIES.py     39          
    CopyRaster     C:\Program Files\ArcGIS\Desktop10.1\arcpy\arcpy\management.py     11034          
ExecuteError: Failed to execute. Parameters are not valid.
ERROR 000472: Name of single band grid cannot have more than 13 characters
Failed to execute (CopyRaster).
                    
‍‍‍‍‍‍‍‍

Re: Extracting rasters from NetCDF files based on time dimension

DanPatterson_Retired — Sat, 11 Dec 2021 14:26:01 GMT

Yes... esri grids can't have filenames that long... the solution is to shorten what your have or make up a new name

>>> outLoc = r"D:\MODIS_NPP_02_15"
>>> outFile = "abcdefghijklmnopqrstuvwxyz"
>>> outfile = "abcdefghijklmnNPP_2001_15"
>>> result = outLoc + "\\N" + outfile[-7:]
>>> result
'D:\\MODIS_NPP_02_15\\N2001_15'
>>>
>>> # or for slice/dice and replace all at once...
>>>
>>> result2 = outLoc + "\\N" + outfile[-7:].replace("200","0")
>>> result2
'D:\\MODIS_NPP_02_15\\N01_15'
>>>
 ‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Extracting rasters from NetCDF files based on time dimension

ShouvikJha — Sat, 11 Dec 2021 14:26:03 GMT

i renamed the input name as well as folder name to make it short, but error still persist . PyScripter getting crash

Message     File Name     Line     Position     
Traceback                    
    __call__     E:\Geo_Softwares\PyScripter\PyScripter\Lib\rpyc.zip\rpyc\core\netref.py     196          
    syncreq     E:\Geo_Softwares\PyScripter\PyScripter\Lib\rpyc.zip\rpyc\core\netref.py     71          
    sync_request     E:\Geo_Softwares\PyScripter\PyScripter\Lib\rpyc.zip\rpyc\core\protocol.py     431          
    serve     E:\Geo_Softwares\PyScripter\PyScripter\Lib\rpyc.zip\rpyc\core\protocol.py     379          
    _recv     E:\Geo_Softwares\PyScripter\PyScripter\Lib\rpyc.zip\rpyc\core\protocol.py     337          
    recv     E:\Geo_Softwares\PyScripter\PyScripter\Lib\rpyc.zip\rpyc\core\channel.py     50          
    read     E:\Geo_Softwares\PyScripter\PyScripter\Lib\rpyc.zip\rpyc\core\stream.py     166          
exceptions.EOFError: [Errno 10054] An existing connection was forcibly closed by the remote host                    
‍‍‍‍‍‍‍‍‍‍

Re: Extracting rasters from NetCDF files based on time dimension

DanPatterson_Retired — Wed, 14 Sep 2016 09:15:32 GMT

It is the folder name and the file name that is more important. And rpyc.zip... how does that come into play?