Understanding .dat-i and .dat-u from server statistics
tl;dr What is the format of .dat-i and .dat-u files in arggisssystem/statistics?
I'm trying to analyse usage patterns for our web services so I can optimise resource allocation (for example, increasing/decreasing running instances). I've tried a few methods that didn't quite work:
- The statistics report tool built into ARCGIS server manager is not suitable for this task because it can only produce pretty limited statistical outputs.
- The export function that is built-in only produces a few data points at a time which makes it a very slow way of extracting the data that sits behind it. (Approx 3 hours to get a sample of 20 .csv's)
- The API works, but only seems able to retrieve total figures. It doesn't seem to have a parameter that lets me specify a date range for the data.
So I've found the place where the statistics are actually saved:
Location of files ^. Although at first menacing, both the .dat-i and the .dat-u files turned out to be structured like csv's.
Sample .dat-i ^
Sample .dat-u ^
So it seems I have managed to find all the statistics I could ever want in a simple machine-readable format that is well suited for statistical analysis. Hurrah! However, I'm not quite sure what most of the fields actually are.
The first field in both is an epoch timestamp in milliseconds. The second is the address of each service.
Then the .dat-i files have 4 other fields:
- The number 60
- The number 0.000
- Either 1.000, 2.000, or 3.000
- Either 0.000, 30.000, 58.000 or 60.000
Wheras, the dat-u files have 7 other fields:
- An integer, mostly under 100
- 0, 1, 2 or 3
- 0
- 0
- An integer, mostly under 100, mostly the same value as field 1.
- A 3 decimal place value, mostly under 1
- A 3 decimal place value, typically higher than field 7.
I have some of my own guesses for some of these, but it would be really helpful if any of you either know for sure what they are, have good guesses, or can point me to the documentation for this. Also, what is the difference between the dat.i and the dat.u files? Thanks for reading!
