Error in Transposed CRF Multidimensional "Sample" Tool Process
Has anyone else seen erroneous export values from a Multidimensional Transposed CRF raster in which the temporal dimension of parameters has different lengths?
The incorrect export is happening when I use the "Sample" tool in ArcGIS Pro 2.6.2 to run an export of pixel values from a Multidimensional Transposed CRF Raster that contains spatio-temporal raster slices for multiple parameters, with the "Multidimensional" option activated.
When I use the “Sample” tool, with the “Multidimensional” option activated, the export gets confused by the fact that the Mosaic Datasets have different numbers of time steps
You can see what happens in the screenshot below, I’ve highlighted the rows where the error occurs:
Column I (SST) is the dataset for which the timeline has the most time slices – so that gets exported correctly – all the values are unique and differ (as they vary over time). All the other preceding (A-H) and succeeding (J) product columns and also the “StdTime” column (K) have repeated values beginning in line 201. The reason for that is that we don’t actually have the timeslices 1/03/2019 0:00 ff. for those parameters.
It appears that the Multidimensional Export from the Transposed CRF just pads the columns with whatever value is the last available (line 201).
Therefore, if you look at the last 5 values for SST and the adjacent KPAR, SST is exported correctly - and KPAR is just 'filled in' with the last available 'true' raster value.
To see the thing in action, with a working set of rasters that have equal numbers of time slices, please check this web app under the URL https://seascoastestuaries.netlify.app/ - only a maximum of 50 pixels can be exported so please zoom....
Reproducing the Error:
I've made a data package to reproduce what I believe to be an error in the Transposed CRF format:
Following is what my package contains (in reality my setup is much bigger with thousands of rasters for about 10 different physico-chemical parameters derived from MODIS satellite imagery):
A ‘correct’ Transposed CRF dataset, where two parameters have the same length on the time dimension:
- I’ve provided rasters for two parameters (CHL, SST), 6 months (timeslices) each – so 12 rasters in total.
- I’ve made 2 Mosaic Datasets, 1 CHL Mosaic Dataset and 1 SST MDS.
- I’ve also added the contents of the above 2 Mosaic Datasets into an empty “Parent” Mosaic Dataset.
- I’ve computed the Multidimensional Info on the Parent Mosaic Dataset using "Build Multidimensional Info".
- I’ve converted/exported the above Parent MDS as a Transposed CRF raster, to allow for very fast temporal profile queries (kind of like here Let’s do data science! Multidimensional analysis in ArcGIS Pro )
A ‘faulty’ Transposed CRF dataset, for comparison, where I’ve used a similar process as above but I’ve made the Sea Surface Temperature (SST) time dimension shorter by just using 3 months for SST (instead of 6 months/slices for CHL/Chlorophyll).
This is how my tool is built. To improve sampling speed, it uses a point feature class where the points are the center points of a raster cell. The first step in the model allows a user to make a selection of points on the screen - those are the raster cell locations which will sampled:
In ArcGIS Pro, the export would kind of work like this:
All of that works very well – particularly when the variables have the same length in the StdTime dimension (as they do in the Web Application referenced above).
When they have different lengths, the tool erroneously writes values from the ‘adjacent’ Variable column into those columns where the Time Dimension is shorter (See the screenshot from Excel above).
For now we’re making sure the the Dimension is always the same length – but that’s hard to enforce in the process we use and we cannot guarantee that in future.
Cheers,
Tilmann
Jian Lange Mansour Raad's Blog Julia Lenhardt, Peter Becker
Accepted Solutions
