Latest Contributions by EricKrause

Hi Suzanne, You're pretty much correct in your first three questions.  I'll add a few notes:    1. The values I see in the voxel layer do come from the EBK3D analysis. That the EBK3D didn't show them to me before is because it has made its (and our) life a bit easier (runs quick) by visualising a triangular grid and 'spreading' the range of values of the original data set over it? Yes.  To build a representative histogram of the output surface, the geostatistical layer would have to make many calculations. By borrowing the histogram of the input points, it can process much more quickly.  However, it is just a rough visualization of what the surface actually looks like.      2. That means that whatever export format I choose for the statistical layer (for example 'to points') I will obtain the values that I now see in my voxel layer?     Yes, all the geostatistical layer does is call a function that returns a prediction (and possibly standard errors) at a coordinate and then writes the value(s) to an output format.  As far as the geostatistical layer is aware, exporting to points, voxels, rasters, etc are all just sets of coordinates used to call this function. After exporting to whatever format, the renderer of the output is always built on the predicted values stored in that format.  So, for example, exporting to a raster will build the render using the entire histogram of the raster.  Similarly with a voxel, the renderer is built on the range of the entire 3D volume (with extremes excluded, as Andrew noted). 3. Some of the values I obtain are nonphysical (i.e. the negative values), does this not mean something is fundamentally wrong about the statistical analysis? Especially if the values are really quite different from the original data set (for example if my data ranges from 0.1 to 80 and the voxel layer shows results between -20 and 60)? Most likely, this behavior indicates that the EBK3D model doesn't fit the data well.  Kriging of all types has a tendency to smooth the input data, where the range of the predictions is narrower than the range of the input data. However, it can also make predictions outside of the range of the input points.  You're actually seeing both at the same time, where both the min and max of the predictions were lower than the min/max of the original data points; it smoothed off the top but extend the bottom of the distribution. I suspect this is due to the shape of the histogram of the original points, which was heavily right-skewed and had constant values.  You should try to identify where the largest deviations are in the 3D volume.  It's possible that you'll find them far off in a corner away from the points, but the still model fits well in the areas where you have data.  If that doesn't work out, I don't have very many recommendations aside from what I already suggested: trying different subset sizes, overlap factors, and transformations.  4. Wouldn't adjusting the min/max range in the above mentioned case from -20 and 60 to 0.1 and 80 be cheating? It seems I would then be ignoring what the geostatistical analyst has told me.. I wouldn't call it cheating as long as you disclose what you're doing.  It's well understood that kriging doesn't understand what the data means (so it won't respect physical impossibilities), and the predictions become unstable when extrapolating.  Simply removing these areas or setting them transparent is very common, but should of course be disclosed. You're not actually changing any data when you do this, just choosing to not display results that you know to be misleading or impossible.  Ideally, you'll just find a perfect EBK3D model that doesn't require this, but assuming this doesn't happen, in my opinion, it's better to not display results you know to be wrong. -Eric ... View more

Hi Suzanne, In addition to the differences in the min/max renderer of the voxel versus the classified renderer of the geostatistical layer, there are also even more fundamental differences. The geostatistical layer actually builds its histogram (used to create the class breaks) on the values of the original points you interpolated.  The voxel layer instead classifies using the values of the netCDF file.  In general, the voxel layer will be a much better representation of what the 3D volume actually looks like.  There is an exactly analogous concept with exporting 2D geostatistical layers to 2D rasters, where the exported raster often looks very different and has different ranges than the geostatistical layer. This is also why the minimum value of the voxel symbology is less than the minimum value of the geostatistical layer symbology.  Some areas of the voxel had predicted values less than the minimum of the original points you interpolated.  This can happen for lots of different reasons, but it's likely related to transformations and trends in the EBK3D model. As for why the geostatistical layer operates like this, it's best to think of the geostatistical layer as a function that does rendering on the fly.  It takes an (x,y) or (x,y,z) coordinate as input and computes the value using references to the interpolation parameters and input points.  Unlike a raster of a voxel layer, it does not have output data written anywhere to disk.  In a sense, the geostatistical layer doesn't know its own values until you do something that requires the calculation.  The filled contours you actually see in the map are just generated by contouring a coarse triangular grid behind the scenes. The benefits of this are that the geostatistical layer calculates and renders very quickly, and it can be used as a model source to easily export to numerous other data formats: rasters, points, contours, voxels, multidimensional rasters, etc.  Hope that made sense, -Eric ... View more

Glad to hear you got the voxel to render!  For your next step, look into the GA Layer To Points tool.  It can be used to export the values of the EBK3D layer directly to 3D point features. ... View more

I suspect the problem is that the coordinate system (both horizonal and vertical) of the scene have to be the same as the Voxel layer, and the coordinate systems of the Voxel layer are driven by the EBK3D layer. Please try adding a brand new Local Scene and immediately try to add the netCDF file as a voxel layer before doing anything else with the scene (specifically, do not add any data to it).  My bet is that it will render correctly.  When you add data to a new map/scene, it will set the coordinate systems to match that data, so the voxel layer and the scene should automatically be put into the same coordinate systems. If this ends up being the issue, you can change the coordinate systems of your other scene to match the coordinate systems of the EBK3D layer, and then you should be able to add the voxel layer. [Edit: I see Andrew beat me to it] ... View more

Hi Suzanne, The isosurface is a 3D contour, so you have to specify which value you want to contour.  My guess is that the default value is producing a very small (or even empty) isosurface.  You can change the isosurface value in the Voxel Exploration pane. I recently wrote a LearnGIS lesson that performs EBK3D and ends with voxel visualization and exploration.  You might find the entire thing useful, but here's the section specifically about creating isosurfaces: Interpolate 3D oxygen measurements in Monterey Bay | Learn ArcGIS  -Eric ... View more

Hi again, I have a lot to say but unfortunately not a lot of time right now.  I'm going to try to cover the most important things, sorry for quickly jumping between topics. The GA Layer 3D To NetCDF tool is used to make the source file for the Voxel layer.  This tool and the voxel layer itself are both new in ArcGIS Pro 2.6, which has only been available for a few months.  If you aren't seeing the option to add a voxel layer in Add Data in a scene view, you probably don't have the most recent version. If you have ArcGIS Pro 2.5, you can use the GA Layer 3D To Multidimensional Raster tool to export the EBK3D layers directly to a multidimensional CRF raster.   I see in your image of cross-validation that many of the points have identical (or very close to identical) values.  These are the horizontal lines of red points in the graph.  Repeated values can be problematic for the Empirical transformation, especially with large gaps between the repeated values (this can be seen in the histogram).  The empirical transformation is essentially trying to fit a smooth curve to the histogram, then uses this curve as a reference to the normal distribution.  However, if the histogram isn't smooth, the curve won't fit the histogram well, and it will likely give strange results in the gaps between the peaks in the histogram. Regarding multivariate normality for the quantile output, this is very hard to explain without giving a 2 hour statistics lecture, so I'll try to keep it short.  Kriging of all types is designed to directly predict the mean and standard error of the true value at a new location. However,  you need a full distribution to estimate quantiles, and the mean and the standard error are not enough to fully reconstruct a distribution.  I could show you many different datasets that all have the same mean and standard error, but they would all have very different quantile values.  So, to calculate quantiles, you have to make an assumption about the predictive distribution, which is almost always a normality assumption.  The reason for this is that if your input data are normally distributed (or transformed to be normal), then the kriging predictions will also be normally distributed.  This is why you check for normality in your input data first so that you don't have to worry about it later. In practice, you do the best that you can to transform the data to be closer to normal, but that is going to be difficult to do for a histogram with repeated values and gaps.  If it were me, I would probably try different EBK3D parameters like subset size and overlap factor and hope that you land on a particular subsetting where the model is most stable.  I would also experiment with not using a transformation at all. To judge how well a particular model is working, I would focus on cross-validation, specifically the RMSE and the Inside 90/95 Percent Interval.  The RMSE is approximately equal to the average difference between the true and predicted value, so it's a quick test to see if a model is too inaccurate to be viable. The two Inside 90/95 statistics build confidence intervals using the normality assumption.  If the values aren't too far off from 90 and 95, it may be safe to assume normality to estimate quantiles. -Eric ... View more

It may not work with your data, but the idea I had was to Empirical Bayesian Kriging with a Probability map output.  If you had a continuous variable with a threshold indicating the cutoff between sand and clay, you could just interpolate the continuous variable with EBK and calculate the probability that each new location exceeds that threshold using the predicted value and the standard error.  The Probability output type does this calculation automatically. The biggest problem with Indicator Kriging is that when you reclassify from a continuous variable to a binary variable, you lose lots of information.  This is why Indicator Kriging usually doesn't perform well.  But if you only have indicator variables to start out and didn't create them with a reclassification, indicator kriging is about the best you can do. And just to add to the complexity, there's another type of kriging called Probability Kriging.  It takes a continuous variable and a threshold.  It then reclassifies the values to 0 and 1 and performs indicator kriging.  However, it also uses the raw values of the continuous variable as a cokriging variable.  This essentially tries to bring back in some of the information that was lost during the reclassification.  In my experience, just using EBK with the Probability output type almost always yields best results (again, if you have a continuous variable), but there are a bunch of options available. ... View more