Update: I have managed to solve this issue, with help from Dan, colleagues and friends. I'm going to put the final working code at the bottom of this post, so skip to the end if you want to see the "answer".
I have a large radar dataset which contains the rainfall depth across the UK every 5 minutes. I want to determine the total rainfall depth for each day. As there are hundreds of files, I decided to use arcpy to solve this problem, which to be frank, I have little experience in. So firstly, here is an example the format for the radar filenames:
metoffice-c-band-rain-radar_uk_201805312155_5km-composite.gif
So if I take the two characters that indicate day (here being day 31) and use that to group all filenames on day 31 into a list, I can then call a list for each day as an input to cell statistics. I wrote some code (which you can see below), but python crashes everytime I run the code. If I print (Day1) it shows a list of all filenames that indicate Day 1, as desired, so I think this may be a problem with the Arcpy element. I previously used a for loop, but I decided to manaully type in every day so that it was more explicit. Anyway, here is my code:-
import os, sys, arcpy
from collections import defaultdict
from arcpy import env
from arcpy.sa import *
arcpy.env.cartographicCoordinateSystem = arcpy.SpatialReference("British National Grid")
arcpy.env.outputCoordinateSystem = arcpy.SpatialReference("British National Grid")
arcpy.env.extent = r"C:\Users\bwj202\OneDrive - University of Exeter\GIS\Leakage\RuralTrunkMains.shp"
arcpy.CheckOutExtension("Spatial")
env.workspace = os.getcwd()
dictdata = defaultdict(list)
for filename in os.listdir(os.getcwd()):
if filename.endswith('.gif'):
day = filename[37:39]
dictdata[day].append(filename)
Day1 = dictdata["01"]
Day2 = dictdata["02"]
Day3 = dictdata["03"]
Day4 = dictdata["04"]
Day5 = dictdata["05"]
Day6 = dictdata["06"]
Day7 = dictdata["07"]
Day8 = dictdata["08"]
Day9 = dictdata["09"]
Day10 = dictdata["10"]
Day11 = dictdata["11"]
Day12 = dictdata["12"]
Day13 = dictdata["13"]
Day14 = dictdata["14"]
Day15 = dictdata["15"]
Day16 = dictdata["16"]
Day17 = dictdata["17"]
Day18 = dictdata["18"]
Day19 = dictdata["19"]
Day20 = dictdata["20"]
Day21 = dictdata["21"]
Day22 = dictdata["22"]
Day23 = dictdata["23"]
Day24 = dictdata["24"]
Day25 = dictdata["25"]
Day26 = dictdata["26"]
Day27 = dictdata["27"]
Day28 = dictdata["28"]
Day29 = dictdata["29"]
Day30 = dictdata["30"]
Day31 = dictdata["31"]
print(Day1)
outCellStatistics1 = CellStatistics(Day1, "SUM", "NODATA")
outCellStatistics1.save("Day1.tif")
outCellStatistics1 = CellStatistics(Day2, "SUM", "NODATA")
outCellStatistics1.save("Day2.tif")
outCellStatistics1 = CellStatistics(Day3, "SUM", "NODATA")
outCellStatistics1.save("Day3.tif")
outCellStatistics1 = CellStatistics(Day4, "SUM", "NODATA")
outCellStatistics1.save("Day4.tif")
outCellStatistics1 = CellStatistics(Day5, "SUM", "NODATA")
outCellStatistics1.save("Day5.tif")
outCellStatistics1 = CellStatistics(Day6, "SUM", "NODATA")
outCellStatistics1.save("Day6.tif")
outCellStatistics1 = CellStatistics(Day7, "SUM", "NODATA")
outCellStatistics1.save("Day7.tif")
outCellStatistics1 = CellStatistics(Day8, "SUM", "NODATA")
outCellStatistics1.save("Day8.tif")
outCellStatistics1 = CellStatistics(Day9, "SUM", "NODATA")
outCellStatistics1.save("Day9.tif")
outCellStatistics1 = CellStatistics(Day10, "SUM", "NODATA")
outCellStatistics1.save("Day10.tif")
outCellStatistics1 = CellStatistics(Day11, "SUM", "NODATA")
outCellStatistics1.save("Day11.tif")
outCellStatistics1 = CellStatistics(Day12, "SUM", "NODATA")
outCellStatistics1.save("Day12.tif")
outCellStatistics1 = CellStatistics(Day13, "SUM", "NODATA")
outCellStatistics1.save("Day13.tif")
outCellStatistics1 = CellStatistics(Day14, "SUM", "NODATA")
outCellStatistics1.save("Day14.tif")
outCellStatistics1 = CellStatistics(Day15, "SUM", "NODATA")
outCellStatistics1.save("Day15.tif")
outCellStatistics1 = CellStatistics(Day16, "SUM", "NODATA")
outCellStatistics1.save("Day16.tif")
outCellStatistics1 = CellStatistics(Day17, "SUM", "NODATA")
outCellStatistics1.save("Day17.tif")
outCellStatistics1 = CellStatistics(Day18, "SUM", "NODATA")
outCellStatistics1.save("Day18.tif")
outCellStatistics1 = CellStatistics(Day19, "SUM", "NODATA")
outCellStatistics1.save("Day19.tif")
outCellStatistics1 = CellStatistics(Day20, "SUM", "NODATA")
outCellStatistics1.save("Day20.tif")
outCellStatistics1 = CellStatistics(Day21, "SUM", "NODATA")
outCellStatistics1.save("Day21.tif")
outCellStatistics1 = CellStatistics(Day22, "SUM", "NODATA")
outCellStatistics1.save("Day22.tif")
outCellStatistics1 = CellStatistics(Day23, "SUM", "NODATA")
outCellStatistics1.save("Day23.tif")
outCellStatistics1 = CellStatistics(Day24, "SUM", "NODATA")
outCellStatistics1.save("Day24.tif")
outCellStatistics1 = CellStatistics(Day25, "SUM", "NODATA")
outCellStatistics1.save("Day25.tif")
outCellStatistics1 = CellStatistics(Day26, "SUM", "NODATA")
outCellStatistics1.save("Day26.tif")
outCellStatistics1 = CellStatistics(Day27, "SUM", "NODATA")
outCellStatistics1.save("Day27.tif")
outCellStatistics1 = CellStatistics(Day28, "SUM", "NODATA")
outCellStatistics1.save("Day28.tif")
outCellStatistics1 = CellStatistics(Day29, "SUM", "NODATA")
outCellStatistics1.save("Day29.tif")
outCellStatistics1 = CellStatistics(Day30, "SUM", "NODATA")
outCellStatistics1.save("Day30.tif")
outCellStatistics1 = CellStatistics(Day31, "SUM", "NODATA")
outCellStatistics1.save("Day31.tif")
My error message is as follows:
outCellStatistics.save(save)
RuntimeError: ERROR 999998: Unexpected Error.
OK so the main problem was that there were memory issues (which was odd as I was able to run cellstatistics in ArcMap) but I also needed the full path in my output. Some of the other changes are implemented purely for the code to be "cleaner". Here is the code:
import os, sys, arcpy
from collections import defaultdict
from arcpy import env
from arcpy.sa import *
arcpy.env.cartographicCoordinateSystem = arcpy.SpatialReference("British National Grid")
arcpy.env.outputCoordinateSystem = arcpy.SpatialReference("British National Grid")
arcpy.env.extent = r"C:\Users\bwj202\OneDrive - University of Exeter\GIS\Leakage\RuralTrunkMains.shp"
arcpy.env.overwriteOutput = True
arcpy.CheckOutExtension("Spatial")
Data_folder = os.getcwd() + "/Data"
env.workspace = os.getcwd()
print (Data_folder)
dictdata = defaultdict(list)
for filename in os.listdir(Data_folder):
if filename.endswith('.gif'):
day = filename[37:39]
dictdata[day].append(Data_folder + "/" + filename)
Export_folder = os.getcwd() + "/Exports"
if os.path.exists(Export_folder):
pass
else:
arcpy.CreateFolder_management(os.getcwd(), "Exports")
print ("Start loop")
for key, value in dictdata.iteritems():
label = str(key)
outCellStatistics = CellStatistics(value, "SUM",
"DATA")
outCellStatistics.save(Export_folder + "/Day" + label + ".tif")
print ("finished")