Latest Contributions by AlbertoAloe

Dan, I did it already.  When I said "even in foreground processing mode " I meant  I switched background gp off. I have another pc with 10.6. Same exact behaviour. Do you have 10.6 to make a simple test like: ext2 = arcpy.Extent(4743000,4518000,5240000,5309000) ras2 = arcpy.sa.CreateRandomRaster(1, 100, ext2) Alberto ... View more

Dan, no news. I was thinking to involve ESRI support and, for this purpose,  I created a script that replicates exactly what I'm doing. Given an input folder containing the 366 rasters, it creates a geodatabase with a mosaic dataset pointing to the rasters and setting all the relevant properties. It also outputs three rasters: ./data/outputMeanRasters/MeanFromCellStats.tif containing the mean we know is correct ./data/outputMeanRasters/MeanFromMosaic.tif containing the mean from the mosaic ./data/outputMeanRasters/PercentVar.tif containing the percent variation The strange thing is that the previous script works with the 'None' mosaic method meaning that rasters are sorted according to ObjectID in the mosaic (although  for the sake of calculating the mean it should be irrelevant). If you try to use 'By Attribute'  using SupportNr field (which is created by the script in a way that the sorting is different from objectID ascending) you get a mean raster from the mosaic which is slightly different from the one gotten before. (??) Zip file in attachment. Unzipping it and double clicking the .py replicates everything (It works with 10.5.1...for older versions the signature of arcpy.SetMosaicDatasetProperties_management may be slightly different. Alberto ... View more

Thanks Dan. Have a nice day Alberto ... View more

Dan, still I cannot figure out what is going on.... I tried to check mosaic dataset properties in order to find the reason for observed difference among mean calculated with mosaic operator and cell statistics on source rasters I'am attaching a python script  with the geoprocessing chain I run in ArcGIS 10.5.1 for mosaic creation and setting.  It can be synthesized as follows: Create a mosaic dataset inside a file geodatabase Load the mosaic pointing to the 366 rasters I sent you before (previous attachment) Set the mosaic properties through gp function (mosaic method to none, operator to mean and other stuff.....) Then, I input mosaic dataset to "copy raster" (or alternatively I can make a mosaic layer keeping the default properties) which should output  a raster that SHOULD BE IDENTICAL to the one created with cell statistics. As we have seen this is not the case.  A note on ArcGIS Pro: I get the same exact raster I get in ArcGIS 10.5.1 when running copy raster against the mosaic dataset.  Another detail: I'm attaching the output from copy raster Yr2008TestRasterMosMeanV2.tif (based on these settings) that is slightly different from the one I sent you before (Yr2008TestRasterMosMean.tif). I found out that the reason for this is the mosaic method : When set to None you would get Yr2008TestRasterMosMeanV2.tif. In this case the sorting of the rasters are based on the ObjectID When set to "By Attribute" with StdTime field and base value '2008/01/01' you would get a raster that is identical to Yr2008TestRasterMosMean.tif Given that: For the sake of calculating our mean the sorting is irrelevant  I purposely scrambled ObjectID and StdTime  in a way that sorting by ObjectID is different from sorting by StdTime ....there must be something wrong in the mosaic dataset Alberto P.S. I gotta solve this otherwise I cannot sleep !!  ... View more

Dan, thanks for taking care of this.. I'm still out of office (the 8th of December is bank holiday in Italy). On Monday I can make other tests... Out of the many I already did one was with ArcGIS Pro 2 getting (if I remember correctly) the same issues. I agree that float32 cannot be an issue. You can try to use dtype 64 when averaging over axis = 0 and you would get irrelevant differences. Also ArcGIS Pro is 64 bit application but I think that if the rasters (and the mosaic dataset) are 32 bit nothing should change except the ability to reference more than 2 GB of memory (but this is not the case given the very small size of the test data). I'll keep you posted Thanks again Alberto ... View more

Dan, I think that if you made r_m.mean() on the temporal axis (axis = 0) and converting back to raster you should get Yr2008TestRasterMean. We know that this works. The problem is the mosaic dataset that is supposed to properly calculate the mean of a pile of rasters as an aggregation on the temporal dimension (or whatever we want to call it). Thanks again for your help Alberto ... View more

Dan, 2008 was a leap year...366 are days. Alberto ... View more

Dan, I'm out of office... I can only see at your figures but it completely makes sense for me. Your r0m.max() and r01m.max() are  exactly the values at the location with max discharge (po river outlet) In my case I did a per cell percent variation calculating a raster like this: perc var = abs(Yr2008TestRasterMosMean - Yr2008TestRasterMean) / Yr2008TestRasterMean Statistics on percvar are a mean of 3% and a max of 10%. You see at cell with max it should be 1670 but I get 1615. Why ? Why Yr2008TestRasterMosMean is different from Yr2008TestRasterMean (which I know is correct ) ? What have I done wrong when obtaining the mean mosaic layer ? I do not need the mosaic dataset to do these things but it's a convenient way to expose netcdf and multidimensional stuff to people that do not know how to use numpy, netcdf4 and so on...Besides, ESRI is advertizing it this way. Thanks Dan Alberto ... View more

Dan, Year2008TestRasterMean.tif (output from cell statistics) is OK because it's exactly what I get using other methods (Aggregation of the numpy array from the original netcdf, using sample in sa...). The problem is in the mosaic dataset. It just references the same rasters....I cannot get it !! Basically I set the mosaic as follows: Mosaic dataset in a file geodatabase with ETRS_89_LAEA projection "Add rasters", pointing to the folder where I have the 366 rasters I gave you , without building overviews I then set the mosaic operator to Mean I set "MaximumNumber of Rasters per Mosaic" = 400 to include them all. This property if not set correctly would have an impact on the mosaic operator limiting the number of rasters used After I make a mosaic layer (dragging and dropping in ArcMap or using the gp tool) double checking that the default mosaic operator still is 'mean'. I run copy raster....and I get what you saw. If you had time to try something I would ask you to make a mosaic dataset out of the 366 rasters and see if you can get the mosaic mean that matches the one from cell stats (or one that matches mine). Thanks again Alberto P.S. If somebody else would be able to confirm my output from the mosaic dataset I would start to think that there is something weird in the mosaic algorithm  ... View more

Dan, thanks for your reply.  Actually I took into account that when running cell statistics. Also when getting the mean from the mosaic I've been also carefull to mosaic properties that can prevent the use of all the rasters. Basically the mosaic is a cube of 366 overlapping rasters. I just want the mosaic to return the mean of the 366 rasters......it works but when I make a percent variation out of cell stats versus mosaic layer I have an average of 3% and a maximum of 10%. I'm attaching a simple test scenario with the 366 rasters (File Year2008TestRaster.zip) as well as what I get from running mean cell statistics (file Yr2008TestRasterMean.zip) and what I get from the mosaic (pointing to the same set of 366 rasters) with the operator mean (File Yr2008TestRasterMosMean.zip).  If you (or somebody else..) have few minutes to tell me how to get the same values from the mosaic....... There must be something I misunderstood about mosaic datasets Thanks a lot Alberto ... View more

Supriya, I forgot to restart it !! I did  it and now it works. Thanks !! Alberto ... View more

Supriya, it did not work. I installed the old ArcSDE installation files (10.2.2) but the result is the same. I did everything as sysadmin (os and sql server). I Also did the following: I tried to use older libst_raster_sql.dll and createAssembly.sql (coming from ArcGIS 10.4 installation instead of 10.5.1). For consistency I used 10.4 client  when using "Create Raster Type" and also the geodatabase was 10.4 schema (even if probably does not make any difference) I double checked .NET requirements on the server and they should be OK especially because there is an installation of ArcGIS Server (4.5 or superior required) I'm using SQL Server 2014 on Windows Server 2012 R2 Standard. The server (staging machine) also has an installation of ArcGIS Desktop & Server 10.4. Can you confirm that what I'm experiencing is not a bug of the ST_Raster assembly ? Consider that the installation seems succesfull (I get 1000 from .st_raster_util_getVersion()) and I'm able to load a raster as ST_Raster (importing it from arcgis using a conf keyword) and using properties of the ST_Raster. When I try to go for pixel data I get the described trouble. Alberto ... View more