Hi Dan,
That's brilliant! It sounds much easier than the previous approach. I have zipped four months' worth of data if you want to play with it. Let me know if you want me to upload some more.
I must say I am not familiar at all with arrays, would you mind pointing me to some additional information ? And possibly copy the start of a script?
Much appreciated!
Ok... just let me get organized today, and I will have a working set for you. Can you articulate what statistics your want, for what time sequences (ie weekly, monthly, running means, annualized) and do you want just the stats and/or maps and or histograms like the following. Which I derived from the average of the 5 days you already posted. So you can at least have a look at the histogram for one of those in symbology and compare it relatively to this one...
Ultimately, I am trying to get an average "growing season temperature or GST" which is an average temperature for each day from 1 October to 30 April, over a 30-year period. It is to characterise the variability within a region so the maps are essential.
My idea was to get the average Maximum temperature for that period and the average Minimum temperature for the same period and then get the average.
A GST per year would be ideal but might be tricky to get as it straddles two years e.g. from 1 October 1961 to 30 April 1962.
I have one month for you to look at the results in the attachment
Currently this is the timing results for what I have so far... reading the monthly data takes 5.5 seconds, calculating the 3 results takes a sip of coffee, and another sip to write them out.
Let me know if the results are correct, I just want to make sure that I calculated the max, avg and min on the correct dimension. I calculated the min, mean and max for each day for the whole area below the time results...
Timing function for... read_folder['mx19610101.txt', 'mx19610102.txt', 'mx19610103.txt', 'mx19610104.txt', 'mx19610105.txt', 'mx19610106.txt', 'mx19610107.txt', 'mx19610108.txt', 'mx19610109.txt', 'mx19610110.txt', 'mx19610111.txt', 'mx19610112.txt', 'mx19610113.txt', 'mx19610114.txt', 'mx19610115.txt', 'mx19610116.txt', 'mx19610117.txt', 'mx19610118.txt', 'mx19610119.txt', 'mx19610120.txt', 'mx19610121.txt', 'mx19610122.txt', 'mx19610123.txt', 'mx19610124.txt', 'mx19610125.txt', 'mx19610126.txt', 'mx19610127.txt', 'mx19610128.txt', 'mx19610129.txt', 'mx19610130.txt', 'mx19610131.txt']Results for... read_folder time taken ...5.534821477e+00 sec.
Timing function for... calc_statsResults for... calc_stats time taken ...5.698581223e-02 sec.
Timing function for... write_fileResults for... write_file time taken ...2.026189804e-01 sec.
>>> s.shape
(691, 886, 31)
>>> s.ndim
3
>>> for i in range(31):
... print("{:>6.3f} {:>6.3f} {:>6.3f}".format(s.min(), s.mean(), s.max()))
...
25.921 30.192 33.996
25.933 30.194 34.139
25.936 30.196 34.311
25.575 30.198 34.483
25.790 30.201 34.655
25.970 30.205 34.776
25.990 30.209 34.897
26.014 30.212 34.986
26.032 30.217 35.077
25.968 30.222 35.137
25.789 30.227 35.196
25.618 30.232 35.236
25.464 30.238 35.285
25.338 30.243 35.313
25.242 30.250 35.342
25.168 30.258 35.370
25.103 30.266 35.379
25.057 30.275 35.396
25.012 30.285 35.394
24.978 30.295 35.391
24.945 30.305 35.378
24.921 30.315 35.364
24.898 30.326 35.318
24.884 30.339 35.191
24.871 30.349 35.017
24.860 30.360 34.833
24.849 30.371 34.661
24.838 30.383 34.519
24.832 30.392 34.396
24.810 30.407 34.308
24.779 30.421 34.307
>>>
PS, can I get permission to use some of this data for writing and teaching purposes?
here is the attachment
Thanks for that. I somehow get different results. Just looking at the first txt file (mx19610101), I get:
min = 7.829 ; mean = 26.725; max = 45.344.
That's just from reading the Statistics of the grid obtained from the txt file and that seems more realistic for the month of January.
Regarding using the data fro writing/teaching, I'll have to get back to you tonight on that one as it is not freely available data and I just need to make sure it's okay to use
Cheers
I branched this to a new discussion for those that are interested in this component and not the original ... or vica versa
I will send stuff later today... that is today Cdn time
so I have the stats reporting in this fashion:
Timing function for... read_folder['mx19610401.txt', 'mx19610402.txt', ... snip ... 'mx19610429.txt', 'mx19610430.txt']Results for... read_folder time taken ...6.003275519e+00 sec.
Timing function for... calc_statsResults for... calc_stats time taken ...5.432958609e-02 sec.
Timing function for... write_statsResults for... write_stats time taken ...2.396773786e-01 sec.
Timing function for... write_all['mx19610401.txt', 'mx19610402.txt', ... snip ... 'mx19610429.txt', 'mx19610430.txt']Results for... write_all time taken ...7.070477038e+00 sec.
Day, min, mean, max
00 6.4 28.2 41.6
01 7.9 28.2 40.3
02 12.6 27.8 40.4
... snip ...
28 8.0 24.8 38.1
29 6.9 24.7 38.1
attached are the rest of the months for checking. If ok, then I will send the scripts ... after I have written the lesson plan for them
ADDENDUM
It isn't letting me save attachments today... you will have to email me at my profile email so I can email you the zips files
Never mind... it is just slow with no progress bar being given
Thanks Dan, it happended to me before with the attachments. I am downloading them right now and will have a look
The results seem correct, Australia is certainly very hot in summer!
good... I will wrap it up... Distorted enough for you
script attached... kept it as verbose as possible
you will have to add batch capabilities and output path generation, and assigning the spatial reference if needed.
I included the prj information within the header plus a bunch of other stuff
If you want to data reclassification into temperature bands, then have a look at my blog post
Reclassify raster data simply
Have fun
link fixed (hopefully)
Thanks a lot for that. I will have a look this week-end and let you know how I go!
Aangemelde leden kunnen berichten plaatsen, updates volgen en meer. Nieuw hier? Registreer een gratis account.
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.
