I am dabbling with the resolution of the error says RuntimeError: ERROR 010240: Could not save raster dataset to as attached as a screen shot. I am working with a DEM file and trying to generate raster based on the conditional values from an excel file. As the excel file has about 15000 row, I need to generate about 15000 raster. I am very much disappointed when the script takes long time and at last it fails every time at different point and this has been happening from the yesterday. I am just reading date and associated 3 temperature values from the excel file and applying con operation on DEM arcgis grid raster to generate another raster. This process is repeated for all the dates i.e. rows in the excel file as attached. My script is a below #!/usr/bin/python # -*- coding: utf-8 -*- import arcpy import os import sys import shutil from arcpy . sa import Con from arcpy . sa import Raster from openpyxl import load_workbook arcpy . env . overwriteOutput = True arcpy . CheckOutExtension ( 'spatial' ) INPUT_TEMP_EXCEL_PATH = \ r "C:\Users\Winrock\Desktop\Ryan\Sept02ModularApproach\Temperature Model Data.xlsx" # arcpy.GetParameterAsText(0) INPUT_DEM_RASTER_PATH = \ r "C:\Users\Winrock\Desktop\Ryan\Sept02ModularApproach\DEM\dem_clip_11" # arcpy.GetParameterAsText(1) Second_Discrete_variable = 10 # arcpy.GetParameterAsText(2) OUTPUT_TEMP_RASTER_FOLDER = \ r "C:\Users\Winrock\Desktop\Ryan\Sept02ModularApproach\OutputRaster" # arcpy.GetParameterAsText(3) TEMP_FOLDER_PATH = \ r "C:\Users\Winrock\Desktop\Ryan\Sept02ModularApproach\Temp" # arcpy.GetParameterAsText(4) Second_Discrete_variable = float ( Second_Discrete_variable ) Temp_Data = [ ] # Loading temperature data temp_wb = load_workbook ( filename = INPUT_TEMP_EXCEL_PATH , read_only = True ) temp_ws = temp_wb [ temp_wb . sheetnames [ 0 ] ] for row in temp_ws . rows : rw = [ cell . value for cell in row ] Temp_Data . append ( rw ) Temp_Data = Temp_Data [ 1 : ] # Folder content deleter def folder_content_deleter ( folder_path ) : for the_file in os . listdir ( folder_path ) : file_path = os . path . join ( folder_path , the_file ) try : if os . path . isfile ( file_path ) : os . unlink ( file_path ) elif os . path . isdir ( file_path ) : shutil . rmtree ( file_path ) except Exception , e : pass # set temporary places, grid format needs a gdb for placing intermediate data folder_content_deleter ( TEMP_FOLDER_PATH ) arcpy . CreateFileGDB_management ( out_folder_path = TEMP_FOLDER_PATH , out_name = 'ScratchData_solRaster' , out_version = 'CURRENT' ) arcpy . env . workspace = arcpy . env . scratchWorkspace = \ os . path . join ( TEMP_FOLDER_PATH , 'ScratchData_solRaster.gdb' ) # processing Second part for tdata in Temp_Data : T1 = tdata [ 1 ] T2 = tdata [ 2 ] T3 = tdata [ 3 ] output_second = Con ( Raster ( INPUT_DEM_RASTER_PATH ) 2573 , T1 , Con ( Raster ( INPUT_DEM_RASTER_PATH ) 2754 , T2 , T3 ) ) final_temp_raster = output_second + Second_Discrete_variable # save out_path = os . path . join ( OUTPUT_TEMP_RASTER_FOLDER , str ( tdata [ 0 ] ) ) final_temp_raster . save ( out_path ) # Cleaning if arcpy . Exists ( 'in_memory' ) : arcpy . Delete_management ( 'in_memory' ) folder_content_deleter ( TEMP_FOLDER_PATH ) My error in gist is- RuntimeError: ERROR 010240: Could not save raster dataset to C:\Users\Winrock\Desktop\Ryan\Sept02ModularApproach\Temp\ScratchData_solRaster.gdb\ifthe_ras with output format FGDBR The excel file I am using is https://www.dropbox.com/s/qacfhipo4ry7o2b/Temperature%20Model%20Data.xlsx?dl=0 My error- N.B. I tried several thread some of them are- * What causes RuntimeError: ERROR 010240 saving after CellStatistics? * Why does CON statement give ERROR 010240: Could not save raster dataset to (value) with output format GRID? * error 010240 with output format grid * arcgis 10.0 - python.multiprocessing and "FATAL ERROR (INFADI) MISSING DIRECTORY" - Geographic Information Systems Stack… System specification: --------------------------------------------------------------------------------------------------------------------------------------------------------------- Update: *I tried with different scratch and current workspace *I tried with arcpy.gp.Times_sa(it stops after processing almost 3000 rasters) and arcpy.sa.Times(it stops after processing almost 1070 rasters) *I tried with arcpy.TestSchemaLock *I tried with setting output to tif format too. I see that it stops and raises error when it processed and output exact number of 1070 grid files. It is giving me pain for several days- please help!

Thanks all Dan and Xander, Al last I solved the problem! My understanding that solved the problem: * Use separate scratch and output work-space * Using multiprocessing is imperative if raster to process is so many(I am obliged to Dan for this concept)- it is really helpful though slow in comparison to the other multiprocessing tasks- esri engineers can tell more about it. * Be careful about locks. * Always clean the garbage in each work-space. * If given multiple raster to process then arcpy.gp.Times_sa can handle more than the arcpy.sa.Times can handle.Really it is absurd that deprecated tool is more powerful that the new one (i call it a nascent babe or at least an apple of Sodom) My understanding about arcpy in raster processing at least in their grid format: * Arcpy can not handle task that involves more than 3000 raster at a stretch to process. Below is my script so far: #!/usr/bin/python # -*- coding: utf-8 -*- #Author:Shariful Islam #Contact: msi_g@yahoo.com import arcpy , os , shutil , multiprocessing , re from arcpy . sa import Con from arcpy . sa import Raster arcpy . env . overwriteOutput = True arcpy . CheckOutExtension ( "spatial" ) try : from openpyxl import load_workbook except : raise Exception ( "install the openpyxl module in your python system" ) #================================================================================================================# #change below as it suits your system. try to make short all path below e.g. C:\OutputRsater D:\InputTempFile.xlsx #just change the paths i.e. right side of the equations below nothing else- be careful. INPUT_TEMP_EXCEL_PATH = r "C:\test\Temprt1.xlsx" INPUT_DEM_RASTER_PATH = r "C:\test\dem_clip_11" OUTPUT_TEMP_RASTER_FOLDER = r "C:\test\myout1" TEMP_FOLDER_PATH = r "C:\test\mytemp" #do not change below from here #================================================================================================================# Temp_Data = [ ] #Loading temperature data temp_wb = load_workbook ( filename = INPUT_TEMP_EXCEL_PATH , read_only = True ) temp_ws = temp_wb [ temp_wb . sheetnames [ 0 ] ] for row in temp_ws . rows : d = [ ] if len ( row ) > 3 : for cell in row : if cell . value == None : pass elif cell . value == 0 : d . append ( 0.000000 ) elif isinstance ( cell . value , float ) : d . append ( round ( cell . value , 6 ) ) else : d . append ( cell . value ) Temp_Data . append ( d ) #process collected excel data Temp_Data = Temp_Data [ 1 : ] seen = set ( ) Temp_Data = [ x for x in Temp_Data if x [ 0 ] not in seen and not seen . add ( x [ 0 ] ) ] # removing duplicate date #Folder content deleter def folder_content_deleter ( folder_path ) : for the_file in os . listdir ( folder_path ) : file_path = os . path . join ( folder_path , the_file ) try : if os . path . isfile ( file_path ) : os . unlink ( file_path ) elif os . path . isdir ( file_path ) : shutil . rmtree ( file_path ) except Exception as e : pass #folder deleter def purge ( dirpth , pattern ) : for f in os . listdir ( dirpth ) : if re . search ( pattern , f ) : pth = os . path . join ( dirpth , f ) shutil . rmtree ( pth , ignore_errors = True ) #gdb content deleter def gdb_content_deleter ( wrkspc ) : for r , d , fls in arcpy . da . Walk ( wrkspc , datatype = "FeatureClass" ) : for f in fls : print f try : arcpy . Delete_management ( os . path . join ( r , f ) ) except : pass #copy and group by year def grouperByYear ( input_folder_path , output_folder_path ) : for dirpath , dirnames , filenames in arcpy . da . Walk ( input_folder_path , topdown = True , datatype = "RasterDataset" , type = "GRID" ) : for filename in filenames : out_folder_name = re . findall ( r '(? , filename ) [ 0 ] out_folder_path = os . path . join ( output_folder_path , out_folder_name ) if not os . path . exists ( out_folder_path ) : print "Creating and populating folder for year %s ......" % out_folder_name os . mkdir ( out_folder_path ) in_data = os . path . join ( dirpath , filename ) ou_feature_name = 'g' + re . findall ( r '(? , filename ) [ 0 ] out_data = os . path . join ( out_folder_path , ou_feature_name ) arcpy . Copy_management ( in_data , out_data ) #processing Second part def times_worker ( times_range_list ) : #set temporary places, grid format needs a gdb for placing intermediate data #folder_content_deleter(TEMP_FOLDER_PATH) scratch_db_name = "Scratch_" + str ( times_range_list [ 0 ] [ 0 ] ) arcpy . CreateFileGDB_management ( out_folder_path = TEMP_FOLDER_PATH , out_name = scratch_db_name , out_version = "CURRENT" ) scr_db = os . path . join ( TEMP_FOLDER_PATH , scratch_db_name + ".gdb" ) arcpy . env . scratchWorkspace = scr_db #set output db out_db_name = "RData_" + str ( times_range_list [ 0 ] [ 0 ] ) #arcpy.CreateFileGDB_management(out_folder_path=OUTPUT_TEMP_RASTER_FOLDER, out_name=out_db_name, out_version="CURRENT") #out_db = os.path.join(OUTPUT_TEMP_RASTER_FOLDER,out_db_name+".gdb") out_db = os . path . join ( OUTPUT_TEMP_RASTER_FOLDER , out_db_name ) if not os . path . exists ( out_db ) : os . mkdir ( out_db ) arcpy . env . workspace = out_db for tdata in times_range_list : T1 = float ( '%.6f' % tdata [ 1 ] ) T2 = float ( '%.6f' % tdata [ 2 ] ) T3 = float ( '%.6f' % tdata [ 3 ] ) out_path = os . path . join ( out_db , 'g' + str ( tdata [ 0 ] ) ) outRast_name = "in_memory\\%s" % out_db_name arcpy . MakeRasterLayer_management ( INPUT_DEM_RASTER_PATH , outRast_name ) output_second = Con ( Raster ( outRast_name ) 2573 , T1 , Con ( Raster ( outRast_name ) 2754 , T2 , T3 ) ) final_temp_raster = output_second #save final_temp_raster . save ( out_path ) #cleaning gdb_content_deleter ( scr_db ) def main ( cu , worker , d_range ) : pool = multiprocessing . Pool ( cu ) pool . map ( worker , d_range , 1 ) pool . close ( ) pool . join ( ) if __name__ == '__main__' : core_usage = 5 chunk_size = 1000 needed_cpu = int ( round ( ( len ( Temp_Data ) / chunk_size ) , 0 ) + 1 ) offsetter = list ( divmod ( needed_cpu , core_usage ) ) cpu_distribution = [ core_usage ] * offsetter [ 0 ] + [ offsetter [ 1 ] ] cpu_distribution = [ cp for cp in cpu_distribution if cp != 0 ] #just remove zero temp_data_range = [ Temp_Data [ i : i + chunk_size ] for i in range ( 0 , len ( Temp_Data ) , chunk_size ) ] print r"Doing raster math . It may take upto 3 - 7 hours even and may use your cpu at the heighest . \ So stop using your cpu fo this time . Go and enjoy elsewhere , let me do the job for you! . . . . . . . . . . " loopcnt = 0 for cpu in cpu_distribution : temp_data_range_splitted = temp_data_range [ loopcnt : loopcnt + cpu ] if len ( temp_data_range_splitted ) > 0 : main ( cpu , times_worker , temp_data_range_splitted ) loopcnt += cpu #Cleaning if arcpy . Exists ( "in_memory" ) : arcpy . Delete_management ( "in_memory" ) folder_content_deleter ( TEMP_FOLDER_PATH ) #group by year print "\nGrouping raster math output by year for you. It may take 1-2 hours at best.So stay tuned!........\n" grouperByYear ( OUTPUT_TEMP_RASTER_FOLDER , OUTPUT_TEMP_RASTER_FOLDER ) #delete unnecessary folders print "\nCleaning all unnecessary files........\n" purge ( OUTPUT_TEMP_RASTER_FOLDER , r 'RData_[0-9]{8}' ) print r "All job finished! Now you are ready for the processing:)"

nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

RuntimeError: ERROR 010240 in Con operation in arcpy

SharifulIslam

I am dabbling with the resolution of the error says RuntimeError: ERROR 010240: Could not save raster dataset to as attached as a screen shot. I am working with a DEM file and trying to generate raster based on the conditional values from an excel file. As the excel file has about 15000 row, I need to generate about 15000 raster. I am very much disappointed when the script takes long time and at last it fails every time at different point and this has been happening from the yesterday.

I am just reading date and associated 3 temperature values from the excel file and applying con operation on DEM arcgis grid raster to generate another raster. This process is repeated for all the dates i.e. rows in the excel file as attached.

My script is a below

<SPAN class="comment token">#!/usr/bin/python</SPAN>
<SPAN class="comment token"># -*- coding: utf-8 -*-</SPAN>
<SPAN class="keyword token">import</SPAN> arcpy
<SPAN class="keyword token">import</SPAN> os
<SPAN class="keyword token">import</SPAN> sys
<SPAN class="keyword token">import</SPAN> shutil
<SPAN class="keyword token">from</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>sa <SPAN class="keyword token">import</SPAN> Con
<SPAN class="keyword token">from</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>sa <SPAN class="keyword token">import</SPAN> Raster
<SPAN class="keyword token">from</SPAN> openpyxl <SPAN class="keyword token">import</SPAN> load_workbook
arcpy<SPAN class="punctuation token">.</SPAN>env<SPAN class="punctuation token">.</SPAN>overwriteOutput <SPAN class="operator token">=</SPAN> <SPAN class="token boolean">True</SPAN>
arcpy<SPAN class="punctuation token">.</SPAN>CheckOutExtension<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'spatial'</SPAN><SPAN class="punctuation token">)</SPAN>

INPUT_TEMP_EXCEL_PATH <SPAN class="operator token">=</SPAN> \
&nbsp;&nbsp;&nbsp; r<SPAN class="string token">"C:\Users\Winrock\Desktop\Ryan\Sept02ModularApproach\Temperature Model Data.xlsx"</SPAN>&nbsp; <SPAN class="comment token"># arcpy.GetParameterAsText(0)</SPAN>

INPUT_DEM_RASTER_PATH <SPAN class="operator token">=</SPAN> \
&nbsp;&nbsp;&nbsp; r<SPAN class="string token">"C:\Users\Winrock\Desktop\Ryan\Sept02ModularApproach\DEM\dem_clip_11"</SPAN>&nbsp; <SPAN class="comment token"># arcpy.GetParameterAsText(1)</SPAN>
Second_Discrete_variable <SPAN class="operator token">=</SPAN> <SPAN class="number token">10</SPAN>&nbsp; <SPAN class="comment token"># arcpy.GetParameterAsText(2)</SPAN>

OUTPUT_TEMP_RASTER_FOLDER <SPAN class="operator token">=</SPAN> \
&nbsp;&nbsp;&nbsp; r<SPAN class="string token">"C:\Users\Winrock\Desktop\Ryan\Sept02ModularApproach\OutputRaster"</SPAN>&nbsp; <SPAN class="comment token"># arcpy.GetParameterAsText(3)</SPAN>

TEMP_FOLDER_PATH <SPAN class="operator token">=</SPAN> \
&nbsp;&nbsp;&nbsp; r<SPAN class="string token">"C:\Users\Winrock\Desktop\Ryan\Sept02ModularApproach\Temp"</SPAN>&nbsp; <SPAN class="comment token"># arcpy.GetParameterAsText(4)</SPAN>

Second_Discrete_variable <SPAN class="operator token">=</SPAN> float<SPAN class="punctuation token">(</SPAN>Second_Discrete_variable<SPAN class="punctuation token">)</SPAN>
Temp_Data <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">]</SPAN>

<SPAN class="comment token"># Loading temperature data</SPAN>

temp_wb <SPAN class="operator token">=</SPAN> load_workbook<SPAN class="punctuation token">(</SPAN>filename<SPAN class="operator token">=</SPAN>INPUT_TEMP_EXCEL_PATH<SPAN class="punctuation token">,</SPAN> read_only<SPAN class="operator token">=</SPAN><SPAN class="token boolean">True</SPAN><SPAN class="punctuation token">)</SPAN>
temp_ws <SPAN class="operator token">=</SPAN> temp_wb<SPAN class="punctuation token">[</SPAN>temp_wb<SPAN class="punctuation token">.</SPAN>sheetnames<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">]</SPAN>
<SPAN class="keyword token">for</SPAN> row <SPAN class="keyword token">in</SPAN> temp_ws<SPAN class="punctuation token">.</SPAN>rows<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; rw <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN>cell<SPAN class="punctuation token">.</SPAN>value <SPAN class="keyword token">for</SPAN> cell <SPAN class="keyword token">in</SPAN> row<SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp; Temp_Data<SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN>rw<SPAN class="punctuation token">)</SPAN>

Temp_Data <SPAN class="operator token">=</SPAN> Temp_Data<SPAN class="punctuation token">[</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">:</SPAN><SPAN class="punctuation token">]</SPAN>


<SPAN class="comment token"># Folder content deleter</SPAN>

<SPAN class="keyword token">def</SPAN> <SPAN class="token function">folder_content_deleter</SPAN><SPAN class="punctuation token">(</SPAN>folder_path<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> the_file <SPAN class="keyword token">in</SPAN> os<SPAN class="punctuation token">.</SPAN>listdir<SPAN class="punctuation token">(</SPAN>folder_path<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; file_path <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>folder_path<SPAN class="punctuation token">,</SPAN> the_file<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">try</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>isfile<SPAN class="punctuation token">(</SPAN>file_path<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; os<SPAN class="punctuation token">.</SPAN>unlink<SPAN class="punctuation token">(</SPAN>file_path<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">elif</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>isdir<SPAN class="punctuation token">(</SPAN>file_path<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; shutil<SPAN class="punctuation token">.</SPAN>rmtree<SPAN class="punctuation token">(</SPAN>file_path<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">except</SPAN> Exception<SPAN class="punctuation token">,</SPAN> e<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">pass</SPAN>


<SPAN class="comment token"># set temporary places, grid format needs a gdb for placing intermediate data</SPAN>

folder_content_deleter<SPAN class="punctuation token">(</SPAN>TEMP_FOLDER_PATH<SPAN class="punctuation token">)</SPAN>
arcpy<SPAN class="punctuation token">.</SPAN>CreateFileGDB_management<SPAN class="punctuation token">(</SPAN>out_folder_path<SPAN class="operator token">=</SPAN>TEMP_FOLDER_PATH<SPAN class="punctuation token">,</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; out_name<SPAN class="operator token">=</SPAN><SPAN class="string token">'ScratchData_solRaster'</SPAN><SPAN class="punctuation token">,</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; out_version<SPAN class="operator token">=</SPAN><SPAN class="string token">'CURRENT'</SPAN><SPAN class="punctuation token">)</SPAN>
arcpy<SPAN class="punctuation token">.</SPAN>env<SPAN class="punctuation token">.</SPAN>workspace <SPAN class="operator token">=</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>env<SPAN class="punctuation token">.</SPAN>scratchWorkspace <SPAN class="operator token">=</SPAN> \
&nbsp;&nbsp;&nbsp; os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>TEMP_FOLDER_PATH<SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'ScratchData_solRaster.gdb'</SPAN><SPAN class="punctuation token">)</SPAN>

<SPAN class="comment token"># processing Second part</SPAN>

<SPAN class="keyword token">for</SPAN> tdata <SPAN class="keyword token">in</SPAN> Temp_Data<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; T1 <SPAN class="operator token">=</SPAN> tdata<SPAN class="punctuation token">[</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp; T2 <SPAN class="operator token">=</SPAN> tdata<SPAN class="punctuation token">[</SPAN><SPAN class="number token">2</SPAN><SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp; T3 <SPAN class="operator token">=</SPAN> tdata<SPAN class="punctuation token">[</SPAN><SPAN class="number token">3</SPAN><SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp; output_second <SPAN class="operator token">=</SPAN> Con<SPAN class="punctuation token">(</SPAN>Raster<SPAN class="punctuation token">(</SPAN>INPUT_DEM_RASTER_PATH<SPAN class="punctuation token">)</SPAN> <SPAN class="operator token"><</SPAN> <SPAN class="number token">2573</SPAN><SPAN class="punctuation token">,</SPAN> T1<SPAN class="punctuation token">,</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Con<SPAN class="punctuation token">(</SPAN>Raster<SPAN class="punctuation token">(</SPAN>INPUT_DEM_RASTER_PATH<SPAN class="punctuation token">)</SPAN> <SPAN class="operator token"><=</SPAN> <SPAN class="number token">2754</SPAN><SPAN class="punctuation token">,</SPAN> T2<SPAN class="punctuation token">,</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; T3<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; final_temp_raster <SPAN class="operator token">=</SPAN> output_second <SPAN class="operator token">+</SPAN> Second_Discrete_variable

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="comment token"># save</SPAN>

&nbsp;&nbsp;&nbsp; out_path <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>OUTPUT_TEMP_RASTER_FOLDER<SPAN class="punctuation token">,</SPAN> str<SPAN class="punctuation token">(</SPAN>tdata<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; final_temp_raster<SPAN class="punctuation token">.</SPAN>save<SPAN class="punctuation token">(</SPAN>out_path<SPAN class="punctuation token">)</SPAN>

<SPAN class="comment token"># Cleaning</SPAN>

<SPAN class="keyword token">if</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>Exists<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'in_memory'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>Delete_management<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'in_memory'</SPAN><SPAN class="punctuation token">)</SPAN>
folder_content_deleter<SPAN class="punctuation token">(</SPAN>TEMP_FOLDER_PATH<SPAN class="punctuation token">)</SPAN>
<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

My error in gist is-

RuntimeError: ERROR 010240: Could not save raster dataset to C:\Users\Winrock\Desktop\Ryan\Sept02ModularApproach\Temp\ScratchData_solRaster.gdb\ifthe_ras with output format FGDBR

The excel file I am using is https://www.dropbox.com/s/qacfhipo4ry7o2b/Temperature%20Model%20Data.xlsx?dl=0

My error-

Error

N.B. I tried several thread some of them are-

System specification:

System

---------------------------------------------------------------------------------------------------------------------------------------------------------------

Update:

*I tried with different scratch and current workspace

*I tried with arcpy.gp.Times_sa(it stops after processing almost 3000 rasters) and arcpy.sa.Times(it stops after processing almost 1070 rasters)

*I tried with arcpy.TestSchemaLock

*I tried with setting output to tif format too.

I see that it stops and raises error when it processed and output exact number of 1070 grid files.

It is giving me pain for several days- please help!

Encuentra más publicaciones etiquetadas con

Accepted answers

SharifulIslam

Thanks all Dan and Xander,

Al last I solved the problem!

My understanding that solved the problem:

Use separate scratch and output work-space
Using multiprocessing is imperative if raster to process is so many(I am obliged to Dan for this concept)- it is really helpful though slow in comparison to the other multiprocessing tasks- esri engineers can tell more about it.
Be careful about locks.
Always clean the garbage in each work-space.
If given multiple raster to process then arcpy.gp.Times_sa can handle more than the arcpy.sa.Times can handle.Really it is absurd that deprecated tool is more powerful that the new one (i call it a nascent babe or at least an apple of Sodom)

My understanding about arcpy in raster processing at least in their grid format:

Arcpy can not handle task that involves more than 3000 raster at a stretch to process.

Below is my script so far:

<SPAN class="comment token">#!/usr/bin/python</SPAN>
<SPAN class="comment token"># -*- coding: utf-8 -*-</SPAN>
<SPAN class="comment token">#Author:Shariful Islam</SPAN>
<SPAN class="comment token"><SPAN>#Contact: </SPAN><A class="jive-link-email-small" href="mailto:msi_g@yahoo.com" rel="nofollow noopener noreferrer" target="_blank">msi_g@yahoo.com</A></SPAN>

<SPAN class="keyword token">import</SPAN> arcpy<SPAN class="punctuation token">,</SPAN>os<SPAN class="punctuation token">,</SPAN>shutil<SPAN class="punctuation token">,</SPAN>multiprocessing<SPAN class="punctuation token">,</SPAN>re
<SPAN class="keyword token">from</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>sa <SPAN class="keyword token">import</SPAN> Con
<SPAN class="keyword token">from</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>sa <SPAN class="keyword token">import</SPAN> Raster
arcpy<SPAN class="punctuation token">.</SPAN>env<SPAN class="punctuation token">.</SPAN>overwriteOutput <SPAN class="operator token">=</SPAN> <SPAN class="token boolean">True</SPAN>
arcpy<SPAN class="punctuation token">.</SPAN>CheckOutExtension<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"spatial"</SPAN><SPAN class="punctuation token">)</SPAN>
<SPAN class="keyword token">try</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">from</SPAN> openpyxl <SPAN class="keyword token">import</SPAN> load_workbook
<SPAN class="keyword token">except</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">raise</SPAN> Exception<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"install the openpyxl module in your python system"</SPAN><SPAN class="punctuation token">)</SPAN>


<SPAN class="comment token">#================================================================================================================#</SPAN>
<SPAN class="comment token">#change below as it suits your system. try to make short all path below e.g. C:\OutputRsater&nbsp; D:\InputTempFile.xlsx</SPAN>
<SPAN class="comment token">#just change the paths i.e. right side of the equations below nothing else- be careful.</SPAN>



INPUT_TEMP_EXCEL_PATH <SPAN class="operator token">=</SPAN>&nbsp; r<SPAN class="string token">"C:\test\Temprt1.xlsx"</SPAN>
INPUT_DEM_RASTER_PATH <SPAN class="operator token">=</SPAN>&nbsp; r<SPAN class="string token">"C:\test\dem_clip_11"</SPAN>
OUTPUT_TEMP_RASTER_FOLDER <SPAN class="operator token">=</SPAN> r<SPAN class="string token">"C:\test\myout1"</SPAN>
TEMP_FOLDER_PATH <SPAN class="operator token">=</SPAN> r<SPAN class="string token">"C:\test\mytemp"</SPAN>






<SPAN class="comment token">#do not change below from here</SPAN>
<SPAN class="comment token">#================================================================================================================#</SPAN>

Temp_Data <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">]</SPAN>
<SPAN class="comment token">#Loading temperature data</SPAN>
temp_wb <SPAN class="operator token">=</SPAN> load_workbook<SPAN class="punctuation token">(</SPAN>filename<SPAN class="operator token">=</SPAN>INPUT_TEMP_EXCEL_PATH<SPAN class="punctuation token">,</SPAN> read_only<SPAN class="operator token">=</SPAN><SPAN class="token boolean">True</SPAN><SPAN class="punctuation token">)</SPAN>
temp_ws <SPAN class="operator token">=</SPAN> temp_wb<SPAN class="punctuation token">[</SPAN>temp_wb<SPAN class="punctuation token">.</SPAN>sheetnames<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">]</SPAN>
<SPAN class="keyword token">for</SPAN> row <SPAN class="keyword token">in</SPAN> temp_ws<SPAN class="punctuation token">.</SPAN>rows<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; d <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> len<SPAN class="punctuation token">(</SPAN>row<SPAN class="punctuation token">)</SPAN><SPAN class="operator token">></SPAN><SPAN class="number token">3</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> cell <SPAN class="keyword token">in</SPAN> row<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> cell<SPAN class="punctuation token">.</SPAN>value <SPAN class="operator token">==</SPAN> None<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">pass</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">elif</SPAN> cell<SPAN class="punctuation token">.</SPAN>value <SPAN class="operator token">==</SPAN> <SPAN class="number token">0</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; d<SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN><SPAN class="number token">0.000000</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">elif</SPAN> isinstance<SPAN class="punctuation token">(</SPAN>cell<SPAN class="punctuation token">.</SPAN>value<SPAN class="punctuation token">,</SPAN> float<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; d<SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN>round<SPAN class="punctuation token">(</SPAN>cell<SPAN class="punctuation token">.</SPAN>value<SPAN class="punctuation token">,</SPAN><SPAN class="number token">6</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">else</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; d<SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN>cell<SPAN class="punctuation token">.</SPAN>value<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Temp_Data<SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN>d<SPAN class="punctuation token">)</SPAN>

<SPAN class="comment token">#process collected excel data</SPAN>
Temp_Data<SPAN class="operator token">=</SPAN> Temp_Data<SPAN class="punctuation token">[</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">:</SPAN><SPAN class="punctuation token">]</SPAN>
seen <SPAN class="operator token">=</SPAN> set<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>
Temp_Data <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN>x <SPAN class="keyword token">for</SPAN> x <SPAN class="keyword token">in</SPAN> Temp_Data <SPAN class="keyword token">if</SPAN> x<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="operator token">not</SPAN> <SPAN class="keyword token">in</SPAN> seen <SPAN class="operator token">and</SPAN> <SPAN class="operator token">not</SPAN> seen<SPAN class="punctuation token">.</SPAN>add<SPAN class="punctuation token">(</SPAN>x<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="comment token"># removing duplicate date</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 
<SPAN class="comment token">#Folder content deleter</SPAN>
<SPAN class="keyword token">def</SPAN> <SPAN class="token function">folder_content_deleter</SPAN><SPAN class="punctuation token">(</SPAN>folder_path<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> the_file <SPAN class="keyword token">in</SPAN> os<SPAN class="punctuation token">.</SPAN>listdir<SPAN class="punctuation token">(</SPAN>folder_path<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; file_path <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>folder_path<SPAN class="punctuation token">,</SPAN> the_file<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">try</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>isfile<SPAN class="punctuation token">(</SPAN>file_path<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; os<SPAN class="punctuation token">.</SPAN>unlink<SPAN class="punctuation token">(</SPAN>file_path<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">elif</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>isdir<SPAN class="punctuation token">(</SPAN>file_path<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN> shutil<SPAN class="punctuation token">.</SPAN>rmtree<SPAN class="punctuation token">(</SPAN>file_path<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">except</SPAN> Exception <SPAN class="keyword token">as</SPAN> e<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">pass</SPAN>

<SPAN class="comment token">#folder deleter</SPAN>
<SPAN class="keyword token">def</SPAN> <SPAN class="token function">purge</SPAN><SPAN class="punctuation token">(</SPAN>dirpth<SPAN class="punctuation token">,</SPAN> pattern<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> f <SPAN class="keyword token">in</SPAN> os<SPAN class="punctuation token">.</SPAN>listdir<SPAN class="punctuation token">(</SPAN>dirpth<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> re<SPAN class="punctuation token">.</SPAN>search<SPAN class="punctuation token">(</SPAN>pattern<SPAN class="punctuation token">,</SPAN> f<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; pth <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>dirpth<SPAN class="punctuation token">,</SPAN> f<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; shutil<SPAN class="punctuation token">.</SPAN>rmtree<SPAN class="punctuation token">(</SPAN>pth<SPAN class="punctuation token">,</SPAN> ignore_errors<SPAN class="operator token">=</SPAN><SPAN class="token boolean">True</SPAN><SPAN class="punctuation token">)</SPAN>
 
<SPAN class="comment token">#gdb content deleter</SPAN>
<SPAN class="keyword token">def</SPAN> <SPAN class="token function">gdb_content_deleter</SPAN><SPAN class="punctuation token">(</SPAN>wrkspc<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> r<SPAN class="punctuation token">,</SPAN>d<SPAN class="punctuation token">,</SPAN>fls <SPAN class="keyword token">in</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>da<SPAN class="punctuation token">.</SPAN>Walk<SPAN class="punctuation token">(</SPAN>wrkspc<SPAN class="punctuation token">,</SPAN> datatype<SPAN class="operator token">=</SPAN><SPAN class="string token">"FeatureClass"</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> f <SPAN class="keyword token">in</SPAN> fls<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">print</SPAN> f
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">try</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>Delete_management<SPAN class="punctuation token">(</SPAN>os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>r<SPAN class="punctuation token">,</SPAN>f<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">except</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">pass</SPAN> 

<SPAN class="comment token">#copy and group by year</SPAN>
<SPAN class="keyword token">def</SPAN> <SPAN class="token function">grouperByYear</SPAN><SPAN class="punctuation token">(</SPAN>input_folder_path<SPAN class="punctuation token">,</SPAN> output_folder_path<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> dirpath<SPAN class="punctuation token">,</SPAN> dirnames<SPAN class="punctuation token">,</SPAN> filenames <SPAN class="keyword token">in</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>da<SPAN class="punctuation token">.</SPAN>Walk<SPAN class="punctuation token">(</SPAN>input_folder_path<SPAN class="punctuation token">,</SPAN> topdown<SPAN class="operator token">=</SPAN><SPAN class="token boolean">True</SPAN><SPAN class="punctuation token">,</SPAN> datatype<SPAN class="operator token">=</SPAN><SPAN class="string token">"RasterDataset"</SPAN><SPAN class="punctuation token">,</SPAN> type<SPAN class="operator token">=</SPAN><SPAN class="string token">"GRID"</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> filename <SPAN class="keyword token">in</SPAN> filenames<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; out_folder_name <SPAN class="operator token">=</SPAN> re<SPAN class="punctuation token">.</SPAN>findall<SPAN class="punctuation token">(</SPAN>r<SPAN class="string token">'(?<=\g)\d{4}'</SPAN><SPAN class="punctuation token">,</SPAN> filename<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; out_folder_path <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>output_folder_path<SPAN class="punctuation token">,</SPAN>out_folder_name<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> <SPAN class="operator token">not</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>exists<SPAN class="punctuation token">(</SPAN>out_folder_path<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">print</SPAN> <SPAN class="string token">"Creating and populating folder for year %s ......"</SPAN><SPAN class="operator token">%</SPAN>out_folder_name
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; os<SPAN class="punctuation token">.</SPAN>mkdir<SPAN class="punctuation token">(</SPAN>out_folder_path<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; in_data <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>dirpath<SPAN class="punctuation token">,</SPAN>filename<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; ou_feature_name <SPAN class="operator token">=</SPAN> <SPAN class="string token">'g'</SPAN><SPAN class="operator token">+</SPAN>re<SPAN class="punctuation token">.</SPAN>findall<SPAN class="punctuation token">(</SPAN>r<SPAN class="string token">'(?<=\g\d{4})\d{4}$'</SPAN><SPAN class="punctuation token">,</SPAN>filename<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; out_data <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>out_folder_path<SPAN class="punctuation token">,</SPAN>ou_feature_name<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>Copy_management<SPAN class="punctuation token">(</SPAN>in_data<SPAN class="punctuation token">,</SPAN> out_data<SPAN class="punctuation token">)</SPAN>


<SPAN class="comment token">#processing Second part</SPAN>
<SPAN class="keyword token">def</SPAN> <SPAN class="token function">times_worker</SPAN><SPAN class="punctuation token">(</SPAN>times_range_list<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#set temporary places, grid format needs a gdb for placing intermediate data</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#folder_content_deleter(TEMP_FOLDER_PATH)</SPAN>
&nbsp;&nbsp;&nbsp; scratch_db_name <SPAN class="operator token">=</SPAN> <SPAN class="string token">"Scratch_"</SPAN><SPAN class="operator token">+</SPAN>str<SPAN class="punctuation token">(</SPAN>times_range_list<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>CreateFileGDB_management<SPAN class="punctuation token">(</SPAN>out_folder_path<SPAN class="operator token">=</SPAN>TEMP_FOLDER_PATH<SPAN class="punctuation token">,</SPAN> out_name<SPAN class="operator token">=</SPAN>scratch_db_name<SPAN class="punctuation token">,</SPAN> out_version<SPAN class="operator token">=</SPAN><SPAN class="string token">"CURRENT"</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; scr_db <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>TEMP_FOLDER_PATH<SPAN class="punctuation token">,</SPAN>scratch_db_name<SPAN class="operator token">+</SPAN><SPAN class="string token">".gdb"</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>env<SPAN class="punctuation token">.</SPAN>scratchWorkspace <SPAN class="operator token">=</SPAN> scr_db
&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#set output db</SPAN>
&nbsp;&nbsp;&nbsp; out_db_name <SPAN class="operator token">=</SPAN> <SPAN class="string token">"RData_"</SPAN><SPAN class="operator token">+</SPAN>str<SPAN class="punctuation token">(</SPAN>times_range_list<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#arcpy.CreateFileGDB_management(out_folder_path=OUTPUT_TEMP_RASTER_FOLDER, out_name=out_db_name, out_version="CURRENT")</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#out_db = os.path.join(OUTPUT_TEMP_RASTER_FOLDER,out_db_name+".gdb")</SPAN>
&nbsp;&nbsp;&nbsp; out_db <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>OUTPUT_TEMP_RASTER_FOLDER<SPAN class="punctuation token">,</SPAN>out_db_name<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> <SPAN class="operator token">not</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>exists<SPAN class="punctuation token">(</SPAN>out_db<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>os<SPAN class="punctuation token">.</SPAN>mkdir<SPAN class="punctuation token">(</SPAN>out_db<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>env<SPAN class="punctuation token">.</SPAN>workspace&nbsp; <SPAN class="operator token">=</SPAN> out_db
&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> tdata <SPAN class="keyword token">in</SPAN> times_range_list<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; T1 <SPAN class="operator token">=</SPAN> float<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'%.6f'</SPAN><SPAN class="operator token">%</SPAN>tdata<SPAN class="punctuation token">[</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; T2 <SPAN class="operator token">=</SPAN> float<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'%.6f'</SPAN><SPAN class="operator token">%</SPAN>tdata<SPAN class="punctuation token">[</SPAN><SPAN class="number token">2</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; T3 <SPAN class="operator token">=</SPAN> float<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'%.6f'</SPAN><SPAN class="operator token">%</SPAN>tdata<SPAN class="punctuation token">[</SPAN><SPAN class="number token">3</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; out_path <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>out_db<SPAN class="punctuation token">,</SPAN><SPAN class="string token">'g'</SPAN><SPAN class="operator token">+</SPAN>str<SPAN class="punctuation token">(</SPAN>tdata<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; outRast_name <SPAN class="operator token">=</SPAN> <SPAN class="string token">"in_memory\\%s"</SPAN><SPAN class="operator token">%</SPAN>out_db_name
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>MakeRasterLayer_management<SPAN class="punctuation token">(</SPAN>INPUT_DEM_RASTER_PATH<SPAN class="punctuation token">,</SPAN>outRast_name<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; output_second <SPAN class="operator token">=</SPAN> Con<SPAN class="punctuation token">(</SPAN>Raster<SPAN class="punctuation token">(</SPAN>outRast_name<SPAN class="punctuation token">)</SPAN><SPAN class="operator token"><</SPAN><SPAN class="number token">2573</SPAN><SPAN class="punctuation token">,</SPAN>T1<SPAN class="punctuation token">,</SPAN>Con<SPAN class="punctuation token">(</SPAN>Raster<SPAN class="punctuation token">(</SPAN>outRast_name<SPAN class="punctuation token">)</SPAN><SPAN class="operator token"><=</SPAN><SPAN class="number token">2754</SPAN><SPAN class="punctuation token">,</SPAN>T2<SPAN class="punctuation token">,</SPAN>T3<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; final_temp_raster <SPAN class="operator token">=</SPAN> output_second
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#save</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; final_temp_raster<SPAN class="punctuation token">.</SPAN>save<SPAN class="punctuation token">(</SPAN>out_path<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#cleaning</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; gdb_content_deleter<SPAN class="punctuation token">(</SPAN>scr_db<SPAN class="punctuation token">)</SPAN>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 


<SPAN class="keyword token">def</SPAN> <SPAN class="token function">main</SPAN><SPAN class="punctuation token">(</SPAN>cu<SPAN class="punctuation token">,</SPAN> worker<SPAN class="punctuation token">,</SPAN> d_range<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; pool <SPAN class="operator token">=</SPAN> multiprocessing<SPAN class="punctuation token">.</SPAN>Pool<SPAN class="punctuation token">(</SPAN>cu<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; pool<SPAN class="punctuation token">.</SPAN>map<SPAN class="punctuation token">(</SPAN>worker<SPAN class="punctuation token">,</SPAN>d_range<SPAN class="punctuation token">,</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; pool<SPAN class="punctuation token">.</SPAN>close<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; pool<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; 
<SPAN class="keyword token">if</SPAN> __name__ <SPAN class="operator token">==</SPAN> <SPAN class="string token">'__main__'</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; core_usage <SPAN class="operator token">=</SPAN> <SPAN class="number token">5</SPAN> 
&nbsp;&nbsp;&nbsp; chunk_size <SPAN class="operator token">=</SPAN> <SPAN class="number token">1000</SPAN>
&nbsp;&nbsp;&nbsp; needed_cpu <SPAN class="operator token">=</SPAN> int<SPAN class="punctuation token">(</SPAN>round<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">(</SPAN>len<SPAN class="punctuation token">(</SPAN>Temp_Data<SPAN class="punctuation token">)</SPAN><SPAN class="operator token">/</SPAN>chunk_size<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="operator token">+</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; offsetter <SPAN class="operator token">=</SPAN> list<SPAN class="punctuation token">(</SPAN>divmod<SPAN class="punctuation token">(</SPAN>needed_cpu<SPAN class="punctuation token">,</SPAN> core_usage<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; cpu_distribution <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN>core_usage<SPAN class="punctuation token">]</SPAN><SPAN class="operator token">*</SPAN>offsetter<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="operator token">+</SPAN><SPAN class="punctuation token">[</SPAN>offsetter<SPAN class="punctuation token">[</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp; cpu_distribution <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN>cp <SPAN class="keyword token">for</SPAN> cp <SPAN class="keyword token">in</SPAN> cpu_distribution <SPAN class="keyword token">if</SPAN> cp<SPAN class="operator token">!=</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="comment token">#just remove zero</SPAN>
&nbsp;&nbsp;&nbsp; temp_data_range <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN>Temp_Data<SPAN class="punctuation token">[</SPAN>i<SPAN class="punctuation token">:</SPAN>i<SPAN class="operator token">+</SPAN>chunk_size<SPAN class="punctuation token">]</SPAN> <SPAN class="keyword token">for</SPAN> i <SPAN class="keyword token">in</SPAN> range<SPAN class="punctuation token">(</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">,</SPAN>len<SPAN class="punctuation token">(</SPAN>Temp_Data<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN>chunk_size<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">print</SPAN> r"Doing raster math<SPAN class="punctuation token">.</SPAN> It may take upto <SPAN class="number token">3</SPAN><SPAN class="operator token">-</SPAN><SPAN class="number token">7</SPAN> hours even <SPAN class="operator token">and</SPAN> may use your cpu at the heighest<SPAN class="punctuation token">.</SPAN>\
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; So stop using your cpu fo this time<SPAN class="punctuation token">.</SPAN> Go <SPAN class="operator token">and</SPAN> enjoy elsewhere<SPAN class="punctuation token">,</SPAN> let me do the job <SPAN class="keyword token">for</SPAN> you!<SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN>"&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; loopcnt <SPAN class="operator token">=</SPAN> <SPAN class="number token">0</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> cpu <SPAN class="keyword token">in</SPAN> cpu_distribution<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; temp_data_range_splitted <SPAN class="operator token">=</SPAN> temp_data_range<SPAN class="punctuation token">[</SPAN>loopcnt<SPAN class="punctuation token">:</SPAN>loopcnt<SPAN class="operator token">+</SPAN>cpu<SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> len<SPAN class="punctuation token">(</SPAN>temp_data_range_splitted<SPAN class="punctuation token">)</SPAN><SPAN class="operator token">></SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; main<SPAN class="punctuation token">(</SPAN>cpu<SPAN class="punctuation token">,</SPAN> times_worker<SPAN class="punctuation token">,</SPAN> temp_data_range_splitted<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; loopcnt<SPAN class="operator token">+=</SPAN>cpu
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#Cleaning</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>Exists<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"in_memory"</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>Delete_management<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"in_memory"</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; folder_content_deleter<SPAN class="punctuation token">(</SPAN>TEMP_FOLDER_PATH<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#group by year</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">print</SPAN> <SPAN class="string token">"\nGrouping raster math output by year for you. It may take 1-2 hours at best.So stay tuned!........\n"</SPAN>
&nbsp;&nbsp;&nbsp; grouperByYear<SPAN class="punctuation token">(</SPAN>OUTPUT_TEMP_RASTER_FOLDER<SPAN class="punctuation token">,</SPAN> OUTPUT_TEMP_RASTER_FOLDER<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#delete unnecessary folders</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">print</SPAN> <SPAN class="string token">"\nCleaning all unnecessary files........\n"</SPAN>
&nbsp;&nbsp;&nbsp; purge<SPAN class="punctuation token">(</SPAN>OUTPUT_TEMP_RASTER_FOLDER<SPAN class="punctuation token">,</SPAN> r<SPAN class="string token">'RData_[0-9]{8}'</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">print</SPAN> r<SPAN class="string token">"All job finished! Now you are ready for the processing:)"</SPAN>‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

All comments

DanPatterson_Retired

you are being overly optimistic in processing that many rasters within the 4 GB limits without deleting stuff as you go. As a suggestion, I would get the spreadsheet into an alternate readable format and dispense with its overhead. As well you path is long and potentially causing problems... try something simple like c:\test and a shorter grid name... it appears you are using grids which have specific filename requirements, so try 'X' or if you want to use another format 'X.tif'. Keep things simple .... process in smaller batches ... switch to 64 bit background processing if possible or more to python 3.4.x and arcgis pro if you want to use all your available memory rather than tiling or processing in batches.

Try some of the above, monitor your system resources as you go, and you will narrow down your issue. You could profile your script to find out where memory is being consumed (there are modules in the Anaconda distribution for ArcGIS Pro 1.3 that will help with this.)... Good luck.

XanderBakker

I would definitely store the Excel data in a table (fgdb) and also trying to store 15000 grids in a single folder will certainly yield errors. Also avoid long folder structures and name the output raster to be sure that the output name does not exceed limits.

DanPatterson_Retired

The number of rasters isn't the issue ( https://community.esri.com/thread/173806 ).

It is file management, you aren't cleaning up as you go and as Xander points out... storing 15000 grids... do you really need them all and do you need them all at once???

The one thing you failed to mention is whether you ran it in steps... bites, portions... then append the pieces or work with their resultant.

SharifulIslam

Thanks Dan!

At last I tried to implement some of your advice. Is there any problem to read excel into a list and using this list later in the geoprocessing? I transferred all in root folder and tried to delete all the intermediate files after it is being used. I tried to save rasters as tif file and checked if align with the naming convention. In fact each raster name length is 8 e.g. 19120205 .But all with no success.

<SPAN class="keyword token">import</SPAN> arcpy<SPAN class="punctuation token">,</SPAN>os<SPAN class="punctuation token">,</SPAN>sys<SPAN class="punctuation token">,</SPAN>shutil<SPAN class="punctuation token">,</SPAN>time<SPAN class="punctuation token">,</SPAN>csv
<SPAN class="keyword token">from</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>sa <SPAN class="keyword token">import</SPAN> Con
<SPAN class="keyword token">from</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>sa <SPAN class="keyword token">import</SPAN> Raster
<SPAN class="keyword token">from</SPAN> openpyxl <SPAN class="keyword token">import</SPAN> load_workbook
arcpy<SPAN class="punctuation token">.</SPAN>env<SPAN class="punctuation token">.</SPAN>overwriteOutput <SPAN class="operator token">=</SPAN> <SPAN class="token boolean">True</SPAN>
arcpy<SPAN class="punctuation token">.</SPAN>CheckOutExtension<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"spatial"</SPAN><SPAN class="punctuation token">)</SPAN>


INPUT_TEMP_EXCEL_PATH <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">(</SPAN><SPAN class="comment token">#arcpy.GetParameterAsText(0)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; r<SPAN class="string token">"C:\test\Temprt.xlsx"</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="punctuation token">)</SPAN>

INPUT_DEM_RASTER_PATH <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">(</SPAN><SPAN class="comment token">#arcpy.GetParameterAsText(1)&nbsp;&nbsp;&nbsp; </SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; r<SPAN class="string token">"C:\test\dem_clip_11"</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="punctuation token">)</SPAN>
Second_Discrete_variable <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">(</SPAN><SPAN class="comment token">#arcpy.GetParameterAsText(2)&nbsp;&nbsp; </SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="number token">1</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="punctuation token">)</SPAN>

OUTPUT_TEMP_RASTER_FOLDER <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">(</SPAN><SPAN class="comment token">#arcpy.GetParameterAsText(3) </SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; r<SPAN class="string token">"C:\test\myout"</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="punctuation token">)</SPAN>

TEMP_FOLDER_PATH <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">(</SPAN><SPAN class="comment token">#arcpy.GetParameterAsText(4)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; r<SPAN class="string token">"C:\test\mytemp"</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="punctuation token">)</SPAN>

Second_Discrete_variable <SPAN class="operator token">=</SPAN> float<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'%.6f'</SPAN><SPAN class="operator token">%</SPAN>Second_Discrete_variable<SPAN class="punctuation token">)</SPAN>
Temp_Data <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">]</SPAN>
<SPAN class="comment token">#Loading temperature data</SPAN>
temp_wb <SPAN class="operator token">=</SPAN> load_workbook<SPAN class="punctuation token">(</SPAN>filename<SPAN class="operator token">=</SPAN>INPUT_TEMP_EXCEL_PATH<SPAN class="punctuation token">,</SPAN> read_only<SPAN class="operator token">=</SPAN><SPAN class="token boolean">True</SPAN><SPAN class="punctuation token">)</SPAN>
temp_ws <SPAN class="operator token">=</SPAN> temp_wb<SPAN class="punctuation token">[</SPAN>temp_wb<SPAN class="punctuation token">.</SPAN>sheetnames<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">]</SPAN>
<SPAN class="keyword token">for</SPAN> row <SPAN class="keyword token">in</SPAN> temp_ws<SPAN class="punctuation token">.</SPAN>rows<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; d <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> cell <SPAN class="keyword token">in</SPAN> row<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> cell<SPAN class="punctuation token">.</SPAN>value <SPAN class="operator token">==</SPAN> None<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">pass</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">elif</SPAN> cell<SPAN class="punctuation token">.</SPAN>value <SPAN class="operator token">==</SPAN> <SPAN class="number token">0</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; d<SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN><SPAN class="number token">0.000000</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">elif</SPAN> isinstance<SPAN class="punctuation token">(</SPAN>cell<SPAN class="punctuation token">.</SPAN>value<SPAN class="punctuation token">,</SPAN> float<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; d<SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN>round<SPAN class="punctuation token">(</SPAN>cell<SPAN class="punctuation token">.</SPAN>value<SPAN class="punctuation token">,</SPAN><SPAN class="number token">6</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">else</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; d<SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN>cell<SPAN class="punctuation token">.</SPAN>value<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; Temp_Data<SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN>d<SPAN class="punctuation token">)</SPAN>

<SPAN class="comment token">#process collected excel data</SPAN>
Temp_Data<SPAN class="operator token">=</SPAN> Temp_Data<SPAN class="punctuation token">[</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">:</SPAN><SPAN class="punctuation token">]</SPAN>
seen <SPAN class="operator token">=</SPAN> set<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>
Temp_Data <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN>x <SPAN class="keyword token">for</SPAN> x <SPAN class="keyword token">in</SPAN> Temp_Data <SPAN class="keyword token">if</SPAN> x<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="operator token">not</SPAN> <SPAN class="keyword token">in</SPAN> seen <SPAN class="operator token">and</SPAN> <SPAN class="operator token">not</SPAN> seen<SPAN class="punctuation token">.</SPAN>add<SPAN class="punctuation token">(</SPAN>x<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="comment token"># removing duplicate date</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 
<SPAN class="comment token">#Folder content deleter</SPAN>
<SPAN class="keyword token">def</SPAN> <SPAN class="token function">folder_content_deleter</SPAN><SPAN class="punctuation token">(</SPAN>folder_path<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> the_file <SPAN class="keyword token">in</SPAN> os<SPAN class="punctuation token">.</SPAN>listdir<SPAN class="punctuation token">(</SPAN>folder_path<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; file_path <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>folder_path<SPAN class="punctuation token">,</SPAN> the_file<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">try</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>isfile<SPAN class="punctuation token">(</SPAN>file_path<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; os<SPAN class="punctuation token">.</SPAN>unlink<SPAN class="punctuation token">(</SPAN>file_path<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">elif</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>isdir<SPAN class="punctuation token">(</SPAN>file_path<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN> shutil<SPAN class="punctuation token">.</SPAN>rmtree<SPAN class="punctuation token">(</SPAN>file_path<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">except</SPAN> Exception <SPAN class="keyword token">as</SPAN> e<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">pass</SPAN>
 
<SPAN class="comment token">#gdb content deleter</SPAN>
<SPAN class="keyword token">def</SPAN> <SPAN class="token function">gdb_content_deleter</SPAN><SPAN class="punctuation token">(</SPAN>wrkspc<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> r<SPAN class="punctuation token">,</SPAN>d<SPAN class="punctuation token">,</SPAN>fls <SPAN class="keyword token">in</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>da<SPAN class="punctuation token">.</SPAN>Walk<SPAN class="punctuation token">(</SPAN>wrkspc<SPAN class="punctuation token">,</SPAN> datatype<SPAN class="operator token">=</SPAN><SPAN class="string token">"FeatureClass"</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> f <SPAN class="keyword token">in</SPAN> fls<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">print</SPAN> f
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">try</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>Delete_management<SPAN class="punctuation token">(</SPAN>os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>r<SPAN class="punctuation token">,</SPAN>f<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">print</SPAN> <SPAN class="string token">"deleted scratch"</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">except</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">pass</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 
<SPAN class="comment token">#set temporary places, grid format needs a gdb for placing intermediate data</SPAN>
folder_content_deleter<SPAN class="punctuation token">(</SPAN>TEMP_FOLDER_PATH<SPAN class="punctuation token">)</SPAN>
arcpy<SPAN class="punctuation token">.</SPAN>CreateFileGDB_management<SPAN class="punctuation token">(</SPAN>out_folder_path<SPAN class="operator token">=</SPAN>TEMP_FOLDER_PATH<SPAN class="punctuation token">,</SPAN> out_name<SPAN class="operator token">=</SPAN><SPAN class="string token">"SDB_SR"</SPAN><SPAN class="punctuation token">,</SPAN> out_version<SPAN class="operator token">=</SPAN><SPAN class="string token">"CURRENT"</SPAN><SPAN class="punctuation token">)</SPAN>
scr_db <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>TEMP_FOLDER_PATH<SPAN class="punctuation token">,</SPAN><SPAN class="string token">"SDB_SR.gdb"</SPAN><SPAN class="punctuation token">)</SPAN>
arcpy<SPAN class="punctuation token">.</SPAN>env<SPAN class="punctuation token">.</SPAN>scratchWorkspace <SPAN class="operator token">=</SPAN> scr_db

<SPAN class="comment token">#processing Second part</SPAN>
counter <SPAN class="operator token">=</SPAN> <SPAN class="number token">0</SPAN>
<SPAN class="keyword token">for</SPAN> tdata <SPAN class="keyword token">in</SPAN> Temp_Data<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; T1 <SPAN class="operator token">=</SPAN> float<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'%.6f'</SPAN><SPAN class="operator token">%</SPAN>tdata<SPAN class="punctuation token">[</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; T2 <SPAN class="operator token">=</SPAN> float<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'%.6f'</SPAN><SPAN class="operator token">%</SPAN>tdata<SPAN class="punctuation token">[</SPAN><SPAN class="number token">2</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; T3 <SPAN class="operator token">=</SPAN> float<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'%.6f'</SPAN><SPAN class="operator token">%</SPAN>tdata<SPAN class="punctuation token">[</SPAN><SPAN class="number token">3</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; out_folder_name <SPAN class="operator token">=</SPAN> str<SPAN class="punctuation token">(</SPAN>tdata<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">:</SPAN><SPAN class="number token">4</SPAN><SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; out_folder <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>OUTPUT_TEMP_RASTER_FOLDER<SPAN class="punctuation token">,</SPAN>out_folder_name<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> <SPAN class="operator token">not</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>exists<SPAN class="punctuation token">(</SPAN>out_folder<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>os<SPAN class="punctuation token">.</SPAN>mkdir<SPAN class="punctuation token">(</SPAN>out_folder<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>env<SPAN class="punctuation token">.</SPAN>workspace&nbsp; <SPAN class="operator token">=</SPAN> out_folder
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; out_path <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>out_folder<SPAN class="punctuation token">,</SPAN>str<SPAN class="punctuation token">(</SPAN>tdata<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="operator token">+</SPAN><SPAN class="string token">'.tif'</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; output_second <SPAN class="operator token">=</SPAN> Con<SPAN class="punctuation token">(</SPAN>Raster<SPAN class="punctuation token">(</SPAN>INPUT_DEM_RASTER_PATH<SPAN class="punctuation token">)</SPAN><SPAN class="operator token"><</SPAN><SPAN class="number token">2573</SPAN><SPAN class="punctuation token">,</SPAN>T1<SPAN class="punctuation token">,</SPAN>Con<SPAN class="punctuation token">(</SPAN>Raster<SPAN class="punctuation token">(</SPAN>INPUT_DEM_RASTER_PATH<SPAN class="punctuation token">)</SPAN><SPAN class="operator token"><=</SPAN><SPAN class="number token">2754</SPAN><SPAN class="punctuation token">,</SPAN>T2<SPAN class="punctuation token">,</SPAN>T3<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#apply map algebra if feature exists</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">try</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; final_temp_raster <SPAN class="operator token">=</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>gp<SPAN class="punctuation token">.</SPAN>Times_sa<SPAN class="punctuation token">(</SPAN>output_second<SPAN class="punctuation token">,</SPAN> str<SPAN class="punctuation token">(</SPAN>Second_Discrete_variable<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"in_memory\\test_"</SPAN><SPAN class="operator token">+</SPAN>str<SPAN class="punctuation token">(</SPAN>counter<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; qq <SPAN class="operator token">=</SPAN> final_temp_raster<SPAN class="punctuation token">.</SPAN>getoutput<SPAN class="punctuation token">(</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; qq<SPAN class="operator token">=</SPAN>Raster<SPAN class="punctuation token">(</SPAN>qq<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; final_temp_raster <SPAN class="operator token">=</SPAN> qq
&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">except</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">pass</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">try</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<SPAN class="comment token">#check if wait is of use</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; time<SPAN class="punctuation token">.</SPAN>sleep<SPAN class="punctuation token">(</SPAN><SPAN class="number token">15</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; final_temp_raster <SPAN class="operator token">=</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>gp<SPAN class="punctuation token">.</SPAN>Times_sa<SPAN class="punctuation token">(</SPAN>output_second<SPAN class="punctuation token">,</SPAN> str<SPAN class="punctuation token">(</SPAN>Second_Discrete_variable<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"in_memory\\test_"</SPAN><SPAN class="operator token">+</SPAN>str<SPAN class="punctuation token">(</SPAN>counter<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; qq <SPAN class="operator token">=</SPAN> final_temp_raster<SPAN class="punctuation token">.</SPAN>getoutput<SPAN class="punctuation token">(</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; qq<SPAN class="operator token">=</SPAN>Raster<SPAN class="punctuation token">(</SPAN>qq<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; final_temp_raster <SPAN class="operator token">=</SPAN> qq
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">except</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">pass</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">with</SPAN> open<SPAN class="punctuation token">(</SPAN>os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>TEMP_FOLDER_PATH<SPAN class="punctuation token">,</SPAN><SPAN class="string token">'error.csv'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'ab'</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="keyword token">as</SPAN> error_file<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; writr <SPAN class="operator token">=</SPAN> csv<SPAN class="punctuation token">.</SPAN>writer<SPAN class="punctuation token">(</SPAN>error_file<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; writr<SPAN class="punctuation token">.</SPAN>writerow<SPAN class="punctuation token">(</SPAN>tdata<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; counter<SPAN class="operator token">+=</SPAN><SPAN class="number token">1</SPAN>
 
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#save</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; final_temp_raster<SPAN class="punctuation token">.</SPAN>save<SPAN class="punctuation token">(</SPAN>out_path<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#cleaning</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>Exists<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"in_memory"</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>Delete_management<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"in_memory"</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; gdb_content_deleter<SPAN class="punctuation token">(</SPAN>scr_db<SPAN class="punctuation token">)</SPAN>

<SPAN class="comment token">#Cleaning</SPAN>
<SPAN class="keyword token">if</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>Exists<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"in_memory"</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>Delete_management<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"in_memory"</SPAN><SPAN class="punctuation token">)</SPAN>
folder_content_deleter<SPAN class="punctuation token">(</SPAN>TEMP_FOLDER_PATH<SPAN class="punctuation token">)</SPAN><SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

DanPatterson_Retired

/blogs/dan_patterson/2016/08/14/script-formatting

and I can't tell from the code, but grid names can't begin with a number

A copy of the full error message again

There is no indication you have run it on fewer inputs..

DanPatterson_Retired

also... can you add some print statements within the process to monitor what is being used as variable values and filenames

SharifulIslam

Thanks all Dan and Xander,

Al last I solved the problem!

My understanding that solved the problem:

Use separate scratch and output work-space
Using multiprocessing is imperative if raster to process is so many(I am obliged to Dan for this concept)- it is really helpful though slow in comparison to the other multiprocessing tasks- esri engineers can tell more about it.
Be careful about locks.
Always clean the garbage in each work-space.
If given multiple raster to process then arcpy.gp.Times_sa can handle more than the arcpy.sa.Times can handle.Really it is absurd that deprecated tool is more powerful that the new one (i call it a nascent babe or at least an apple of Sodom)

My understanding about arcpy in raster processing at least in their grid format:

Arcpy can not handle task that involves more than 3000 raster at a stretch to process.

Below is my script so far:

<SPAN class="comment token">#!/usr/bin/python</SPAN>
<SPAN class="comment token"># -*- coding: utf-8 -*-</SPAN>
<SPAN class="comment token">#Author:Shariful Islam</SPAN>
<SPAN class="comment token"><SPAN>#Contact: </SPAN><A class="jive-link-email-small" href="mailto:msi_g@yahoo.com" rel="nofollow noopener noreferrer" target="_blank">msi_g@yahoo.com</A></SPAN>

<SPAN class="keyword token">import</SPAN> arcpy<SPAN class="punctuation token">,</SPAN>os<SPAN class="punctuation token">,</SPAN>shutil<SPAN class="punctuation token">,</SPAN>multiprocessing<SPAN class="punctuation token">,</SPAN>re
<SPAN class="keyword token">from</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>sa <SPAN class="keyword token">import</SPAN> Con
<SPAN class="keyword token">from</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>sa <SPAN class="keyword token">import</SPAN> Raster
arcpy<SPAN class="punctuation token">.</SPAN>env<SPAN class="punctuation token">.</SPAN>overwriteOutput <SPAN class="operator token">=</SPAN> <SPAN class="token boolean">True</SPAN>
arcpy<SPAN class="punctuation token">.</SPAN>CheckOutExtension<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"spatial"</SPAN><SPAN class="punctuation token">)</SPAN>
<SPAN class="keyword token">try</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">from</SPAN> openpyxl <SPAN class="keyword token">import</SPAN> load_workbook
<SPAN class="keyword token">except</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">raise</SPAN> Exception<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"install the openpyxl module in your python system"</SPAN><SPAN class="punctuation token">)</SPAN>


<SPAN class="comment token">#================================================================================================================#</SPAN>
<SPAN class="comment token">#change below as it suits your system. try to make short all path below e.g. C:\OutputRsater&nbsp; D:\InputTempFile.xlsx</SPAN>
<SPAN class="comment token">#just change the paths i.e. right side of the equations below nothing else- be careful.</SPAN>



INPUT_TEMP_EXCEL_PATH <SPAN class="operator token">=</SPAN>&nbsp; r<SPAN class="string token">"C:\test\Temprt1.xlsx"</SPAN>
INPUT_DEM_RASTER_PATH <SPAN class="operator token">=</SPAN>&nbsp; r<SPAN class="string token">"C:\test\dem_clip_11"</SPAN>
OUTPUT_TEMP_RASTER_FOLDER <SPAN class="operator token">=</SPAN> r<SPAN class="string token">"C:\test\myout1"</SPAN>
TEMP_FOLDER_PATH <SPAN class="operator token">=</SPAN> r<SPAN class="string token">"C:\test\mytemp"</SPAN>






<SPAN class="comment token">#do not change below from here</SPAN>
<SPAN class="comment token">#================================================================================================================#</SPAN>

Temp_Data <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">]</SPAN>
<SPAN class="comment token">#Loading temperature data</SPAN>
temp_wb <SPAN class="operator token">=</SPAN> load_workbook<SPAN class="punctuation token">(</SPAN>filename<SPAN class="operator token">=</SPAN>INPUT_TEMP_EXCEL_PATH<SPAN class="punctuation token">,</SPAN> read_only<SPAN class="operator token">=</SPAN><SPAN class="token boolean">True</SPAN><SPAN class="punctuation token">)</SPAN>
temp_ws <SPAN class="operator token">=</SPAN> temp_wb<SPAN class="punctuation token">[</SPAN>temp_wb<SPAN class="punctuation token">.</SPAN>sheetnames<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">]</SPAN>
<SPAN class="keyword token">for</SPAN> row <SPAN class="keyword token">in</SPAN> temp_ws<SPAN class="punctuation token">.</SPAN>rows<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; d <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> len<SPAN class="punctuation token">(</SPAN>row<SPAN class="punctuation token">)</SPAN><SPAN class="operator token">></SPAN><SPAN class="number token">3</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> cell <SPAN class="keyword token">in</SPAN> row<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> cell<SPAN class="punctuation token">.</SPAN>value <SPAN class="operator token">==</SPAN> None<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">pass</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">elif</SPAN> cell<SPAN class="punctuation token">.</SPAN>value <SPAN class="operator token">==</SPAN> <SPAN class="number token">0</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; d<SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN><SPAN class="number token">0.000000</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">elif</SPAN> isinstance<SPAN class="punctuation token">(</SPAN>cell<SPAN class="punctuation token">.</SPAN>value<SPAN class="punctuation token">,</SPAN> float<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; d<SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN>round<SPAN class="punctuation token">(</SPAN>cell<SPAN class="punctuation token">.</SPAN>value<SPAN class="punctuation token">,</SPAN><SPAN class="number token">6</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">else</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; d<SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN>cell<SPAN class="punctuation token">.</SPAN>value<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Temp_Data<SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN>d<SPAN class="punctuation token">)</SPAN>

<SPAN class="comment token">#process collected excel data</SPAN>
Temp_Data<SPAN class="operator token">=</SPAN> Temp_Data<SPAN class="punctuation token">[</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">:</SPAN><SPAN class="punctuation token">]</SPAN>
seen <SPAN class="operator token">=</SPAN> set<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>
Temp_Data <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN>x <SPAN class="keyword token">for</SPAN> x <SPAN class="keyword token">in</SPAN> Temp_Data <SPAN class="keyword token">if</SPAN> x<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="operator token">not</SPAN> <SPAN class="keyword token">in</SPAN> seen <SPAN class="operator token">and</SPAN> <SPAN class="operator token">not</SPAN> seen<SPAN class="punctuation token">.</SPAN>add<SPAN class="punctuation token">(</SPAN>x<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="comment token"># removing duplicate date</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 
<SPAN class="comment token">#Folder content deleter</SPAN>
<SPAN class="keyword token">def</SPAN> <SPAN class="token function">folder_content_deleter</SPAN><SPAN class="punctuation token">(</SPAN>folder_path<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> the_file <SPAN class="keyword token">in</SPAN> os<SPAN class="punctuation token">.</SPAN>listdir<SPAN class="punctuation token">(</SPAN>folder_path<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; file_path <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>folder_path<SPAN class="punctuation token">,</SPAN> the_file<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">try</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>isfile<SPAN class="punctuation token">(</SPAN>file_path<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; os<SPAN class="punctuation token">.</SPAN>unlink<SPAN class="punctuation token">(</SPAN>file_path<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">elif</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>isdir<SPAN class="punctuation token">(</SPAN>file_path<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN> shutil<SPAN class="punctuation token">.</SPAN>rmtree<SPAN class="punctuation token">(</SPAN>file_path<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">except</SPAN> Exception <SPAN class="keyword token">as</SPAN> e<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">pass</SPAN>

<SPAN class="comment token">#folder deleter</SPAN>
<SPAN class="keyword token">def</SPAN> <SPAN class="token function">purge</SPAN><SPAN class="punctuation token">(</SPAN>dirpth<SPAN class="punctuation token">,</SPAN> pattern<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> f <SPAN class="keyword token">in</SPAN> os<SPAN class="punctuation token">.</SPAN>listdir<SPAN class="punctuation token">(</SPAN>dirpth<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> re<SPAN class="punctuation token">.</SPAN>search<SPAN class="punctuation token">(</SPAN>pattern<SPAN class="punctuation token">,</SPAN> f<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; pth <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>dirpth<SPAN class="punctuation token">,</SPAN> f<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; shutil<SPAN class="punctuation token">.</SPAN>rmtree<SPAN class="punctuation token">(</SPAN>pth<SPAN class="punctuation token">,</SPAN> ignore_errors<SPAN class="operator token">=</SPAN><SPAN class="token boolean">True</SPAN><SPAN class="punctuation token">)</SPAN>
 
<SPAN class="comment token">#gdb content deleter</SPAN>
<SPAN class="keyword token">def</SPAN> <SPAN class="token function">gdb_content_deleter</SPAN><SPAN class="punctuation token">(</SPAN>wrkspc<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> r<SPAN class="punctuation token">,</SPAN>d<SPAN class="punctuation token">,</SPAN>fls <SPAN class="keyword token">in</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>da<SPAN class="punctuation token">.</SPAN>Walk<SPAN class="punctuation token">(</SPAN>wrkspc<SPAN class="punctuation token">,</SPAN> datatype<SPAN class="operator token">=</SPAN><SPAN class="string token">"FeatureClass"</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> f <SPAN class="keyword token">in</SPAN> fls<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">print</SPAN> f
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">try</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>Delete_management<SPAN class="punctuation token">(</SPAN>os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>r<SPAN class="punctuation token">,</SPAN>f<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">except</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">pass</SPAN> 

<SPAN class="comment token">#copy and group by year</SPAN>
<SPAN class="keyword token">def</SPAN> <SPAN class="token function">grouperByYear</SPAN><SPAN class="punctuation token">(</SPAN>input_folder_path<SPAN class="punctuation token">,</SPAN> output_folder_path<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> dirpath<SPAN class="punctuation token">,</SPAN> dirnames<SPAN class="punctuation token">,</SPAN> filenames <SPAN class="keyword token">in</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>da<SPAN class="punctuation token">.</SPAN>Walk<SPAN class="punctuation token">(</SPAN>input_folder_path<SPAN class="punctuation token">,</SPAN> topdown<SPAN class="operator token">=</SPAN><SPAN class="token boolean">True</SPAN><SPAN class="punctuation token">,</SPAN> datatype<SPAN class="operator token">=</SPAN><SPAN class="string token">"RasterDataset"</SPAN><SPAN class="punctuation token">,</SPAN> type<SPAN class="operator token">=</SPAN><SPAN class="string token">"GRID"</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> filename <SPAN class="keyword token">in</SPAN> filenames<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; out_folder_name <SPAN class="operator token">=</SPAN> re<SPAN class="punctuation token">.</SPAN>findall<SPAN class="punctuation token">(</SPAN>r<SPAN class="string token">'(?<=\g)\d{4}'</SPAN><SPAN class="punctuation token">,</SPAN> filename<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; out_folder_path <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>output_folder_path<SPAN class="punctuation token">,</SPAN>out_folder_name<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> <SPAN class="operator token">not</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>exists<SPAN class="punctuation token">(</SPAN>out_folder_path<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">print</SPAN> <SPAN class="string token">"Creating and populating folder for year %s ......"</SPAN><SPAN class="operator token">%</SPAN>out_folder_name
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; os<SPAN class="punctuation token">.</SPAN>mkdir<SPAN class="punctuation token">(</SPAN>out_folder_path<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; in_data <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>dirpath<SPAN class="punctuation token">,</SPAN>filename<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; ou_feature_name <SPAN class="operator token">=</SPAN> <SPAN class="string token">'g'</SPAN><SPAN class="operator token">+</SPAN>re<SPAN class="punctuation token">.</SPAN>findall<SPAN class="punctuation token">(</SPAN>r<SPAN class="string token">'(?<=\g\d{4})\d{4}$'</SPAN><SPAN class="punctuation token">,</SPAN>filename<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; out_data <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>out_folder_path<SPAN class="punctuation token">,</SPAN>ou_feature_name<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>Copy_management<SPAN class="punctuation token">(</SPAN>in_data<SPAN class="punctuation token">,</SPAN> out_data<SPAN class="punctuation token">)</SPAN>


<SPAN class="comment token">#processing Second part</SPAN>
<SPAN class="keyword token">def</SPAN> <SPAN class="token function">times_worker</SPAN><SPAN class="punctuation token">(</SPAN>times_range_list<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#set temporary places, grid format needs a gdb for placing intermediate data</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#folder_content_deleter(TEMP_FOLDER_PATH)</SPAN>
&nbsp;&nbsp;&nbsp; scratch_db_name <SPAN class="operator token">=</SPAN> <SPAN class="string token">"Scratch_"</SPAN><SPAN class="operator token">+</SPAN>str<SPAN class="punctuation token">(</SPAN>times_range_list<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>CreateFileGDB_management<SPAN class="punctuation token">(</SPAN>out_folder_path<SPAN class="operator token">=</SPAN>TEMP_FOLDER_PATH<SPAN class="punctuation token">,</SPAN> out_name<SPAN class="operator token">=</SPAN>scratch_db_name<SPAN class="punctuation token">,</SPAN> out_version<SPAN class="operator token">=</SPAN><SPAN class="string token">"CURRENT"</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; scr_db <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>TEMP_FOLDER_PATH<SPAN class="punctuation token">,</SPAN>scratch_db_name<SPAN class="operator token">+</SPAN><SPAN class="string token">".gdb"</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>env<SPAN class="punctuation token">.</SPAN>scratchWorkspace <SPAN class="operator token">=</SPAN> scr_db
&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#set output db</SPAN>
&nbsp;&nbsp;&nbsp; out_db_name <SPAN class="operator token">=</SPAN> <SPAN class="string token">"RData_"</SPAN><SPAN class="operator token">+</SPAN>str<SPAN class="punctuation token">(</SPAN>times_range_list<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#arcpy.CreateFileGDB_management(out_folder_path=OUTPUT_TEMP_RASTER_FOLDER, out_name=out_db_name, out_version="CURRENT")</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#out_db = os.path.join(OUTPUT_TEMP_RASTER_FOLDER,out_db_name+".gdb")</SPAN>
&nbsp;&nbsp;&nbsp; out_db <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>OUTPUT_TEMP_RASTER_FOLDER<SPAN class="punctuation token">,</SPAN>out_db_name<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> <SPAN class="operator token">not</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>exists<SPAN class="punctuation token">(</SPAN>out_db<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>os<SPAN class="punctuation token">.</SPAN>mkdir<SPAN class="punctuation token">(</SPAN>out_db<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>env<SPAN class="punctuation token">.</SPAN>workspace&nbsp; <SPAN class="operator token">=</SPAN> out_db
&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> tdata <SPAN class="keyword token">in</SPAN> times_range_list<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; T1 <SPAN class="operator token">=</SPAN> float<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'%.6f'</SPAN><SPAN class="operator token">%</SPAN>tdata<SPAN class="punctuation token">[</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; T2 <SPAN class="operator token">=</SPAN> float<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'%.6f'</SPAN><SPAN class="operator token">%</SPAN>tdata<SPAN class="punctuation token">[</SPAN><SPAN class="number token">2</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; T3 <SPAN class="operator token">=</SPAN> float<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'%.6f'</SPAN><SPAN class="operator token">%</SPAN>tdata<SPAN class="punctuation token">[</SPAN><SPAN class="number token">3</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; out_path <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>out_db<SPAN class="punctuation token">,</SPAN><SPAN class="string token">'g'</SPAN><SPAN class="operator token">+</SPAN>str<SPAN class="punctuation token">(</SPAN>tdata<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; outRast_name <SPAN class="operator token">=</SPAN> <SPAN class="string token">"in_memory\\%s"</SPAN><SPAN class="operator token">%</SPAN>out_db_name
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>MakeRasterLayer_management<SPAN class="punctuation token">(</SPAN>INPUT_DEM_RASTER_PATH<SPAN class="punctuation token">,</SPAN>outRast_name<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; output_second <SPAN class="operator token">=</SPAN> Con<SPAN class="punctuation token">(</SPAN>Raster<SPAN class="punctuation token">(</SPAN>outRast_name<SPAN class="punctuation token">)</SPAN><SPAN class="operator token"><</SPAN><SPAN class="number token">2573</SPAN><SPAN class="punctuation token">,</SPAN>T1<SPAN class="punctuation token">,</SPAN>Con<SPAN class="punctuation token">(</SPAN>Raster<SPAN class="punctuation token">(</SPAN>outRast_name<SPAN class="punctuation token">)</SPAN><SPAN class="operator token"><=</SPAN><SPAN class="number token">2754</SPAN><SPAN class="punctuation token">,</SPAN>T2<SPAN class="punctuation token">,</SPAN>T3<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; final_temp_raster <SPAN class="operator token">=</SPAN> output_second
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#save</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; final_temp_raster<SPAN class="punctuation token">.</SPAN>save<SPAN class="punctuation token">(</SPAN>out_path<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#cleaning</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; gdb_content_deleter<SPAN class="punctuation token">(</SPAN>scr_db<SPAN class="punctuation token">)</SPAN>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 


<SPAN class="keyword token">def</SPAN> <SPAN class="token function">main</SPAN><SPAN class="punctuation token">(</SPAN>cu<SPAN class="punctuation token">,</SPAN> worker<SPAN class="punctuation token">,</SPAN> d_range<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; pool <SPAN class="operator token">=</SPAN> multiprocessing<SPAN class="punctuation token">.</SPAN>Pool<SPAN class="punctuation token">(</SPAN>cu<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; pool<SPAN class="punctuation token">.</SPAN>map<SPAN class="punctuation token">(</SPAN>worker<SPAN class="punctuation token">,</SPAN>d_range<SPAN class="punctuation token">,</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; pool<SPAN class="punctuation token">.</SPAN>close<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; pool<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; 
<SPAN class="keyword token">if</SPAN> __name__ <SPAN class="operator token">==</SPAN> <SPAN class="string token">'__main__'</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; core_usage <SPAN class="operator token">=</SPAN> <SPAN class="number token">5</SPAN> 
&nbsp;&nbsp;&nbsp; chunk_size <SPAN class="operator token">=</SPAN> <SPAN class="number token">1000</SPAN>
&nbsp;&nbsp;&nbsp; needed_cpu <SPAN class="operator token">=</SPAN> int<SPAN class="punctuation token">(</SPAN>round<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">(</SPAN>len<SPAN class="punctuation token">(</SPAN>Temp_Data<SPAN class="punctuation token">)</SPAN><SPAN class="operator token">/</SPAN>chunk_size<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="operator token">+</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; offsetter <SPAN class="operator token">=</SPAN> list<SPAN class="punctuation token">(</SPAN>divmod<SPAN class="punctuation token">(</SPAN>needed_cpu<SPAN class="punctuation token">,</SPAN> core_usage<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; cpu_distribution <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN>core_usage<SPAN class="punctuation token">]</SPAN><SPAN class="operator token">*</SPAN>offsetter<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="operator token">+</SPAN><SPAN class="punctuation token">[</SPAN>offsetter<SPAN class="punctuation token">[</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp; cpu_distribution <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN>cp <SPAN class="keyword token">for</SPAN> cp <SPAN class="keyword token">in</SPAN> cpu_distribution <SPAN class="keyword token">if</SPAN> cp<SPAN class="operator token">!=</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="comment token">#just remove zero</SPAN>
&nbsp;&nbsp;&nbsp; temp_data_range <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN>Temp_Data<SPAN class="punctuation token">[</SPAN>i<SPAN class="punctuation token">:</SPAN>i<SPAN class="operator token">+</SPAN>chunk_size<SPAN class="punctuation token">]</SPAN> <SPAN class="keyword token">for</SPAN> i <SPAN class="keyword token">in</SPAN> range<SPAN class="punctuation token">(</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">,</SPAN>len<SPAN class="punctuation token">(</SPAN>Temp_Data<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN>chunk_size<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">print</SPAN> r"Doing raster math<SPAN class="punctuation token">.</SPAN> It may take upto <SPAN class="number token">3</SPAN><SPAN class="operator token">-</SPAN><SPAN class="number token">7</SPAN> hours even <SPAN class="operator token">and</SPAN> may use your cpu at the heighest<SPAN class="punctuation token">.</SPAN>\
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; So stop using your cpu fo this time<SPAN class="punctuation token">.</SPAN> Go <SPAN class="operator token">and</SPAN> enjoy elsewhere<SPAN class="punctuation token">,</SPAN> let me do the job <SPAN class="keyword token">for</SPAN> you!<SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN>"&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; loopcnt <SPAN class="operator token">=</SPAN> <SPAN class="number token">0</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> cpu <SPAN class="keyword token">in</SPAN> cpu_distribution<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; temp_data_range_splitted <SPAN class="operator token">=</SPAN> temp_data_range<SPAN class="punctuation token">[</SPAN>loopcnt<SPAN class="punctuation token">:</SPAN>loopcnt<SPAN class="operator token">+</SPAN>cpu<SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> len<SPAN class="punctuation token">(</SPAN>temp_data_range_splitted<SPAN class="punctuation token">)</SPAN><SPAN class="operator token">></SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; main<SPAN class="punctuation token">(</SPAN>cpu<SPAN class="punctuation token">,</SPAN> times_worker<SPAN class="punctuation token">,</SPAN> temp_data_range_splitted<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; loopcnt<SPAN class="operator token">+=</SPAN>cpu
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#Cleaning</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>Exists<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"in_memory"</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; arcpy<SPAN class="punctuation token">.</SPAN>Delete_management<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"in_memory"</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; folder_content_deleter<SPAN class="punctuation token">(</SPAN>TEMP_FOLDER_PATH<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#group by year</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">print</SPAN> <SPAN class="string token">"\nGrouping raster math output by year for you. It may take 1-2 hours at best.So stay tuned!........\n"</SPAN>
&nbsp;&nbsp;&nbsp; grouperByYear<SPAN class="punctuation token">(</SPAN>OUTPUT_TEMP_RASTER_FOLDER<SPAN class="punctuation token">,</SPAN> OUTPUT_TEMP_RASTER_FOLDER<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; <SPAN class="comment token">#delete unnecessary folders</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">print</SPAN> <SPAN class="string token">"\nCleaning all unnecessary files........\n"</SPAN>
&nbsp;&nbsp;&nbsp; purge<SPAN class="punctuation token">(</SPAN>OUTPUT_TEMP_RASTER_FOLDER<SPAN class="punctuation token">,</SPAN> r<SPAN class="string token">'RData_[0-9]{8}'</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">print</SPAN> r<SPAN class="string token">"All job finished! Now you are ready for the processing:)"</SPAN>‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

curtvprice

Wanted to add a comment.

Scratch rasters (those created with map algebra and not ".save()d") are piling up. Since all scratch rasters are grids (not .tifs) you are hitting the limit of 10,000 INFO tables (.vat, .bnd, .sta for each grid) in a workspace. I think you're getting around this by putting your scratch workspace into a .gdb, and multiprocessing, which puts every operation's temp rasters in their own workspace. (Setting scratch to file gdb means in many cases more file copying behind the scenes [grid to gdb] every time you create a raster. Even if this wasn't an issue, storing that many datasets of any kind of a folder is going to slow things WAY down as Arcpy needs to locate them whenever you browse the workspace. If your workflow avoided keeping so many temporary files around, I think it would go faster and you'd run into fewer issues.

Also see: The interaction of the Raster object—ArcGIS Help | ArcGIS for Desktop

Únete a la Conversación

Los miembros registrados pueden publicar, seguir actualizaciones y más. ¿Nuevo aquí? Regístrate gratis.

Enlaces Rápidos

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.