POST
|
Dan, I did it already. When I said "even in foreground processing mode " I meant I switched background gp off. I have another pc with 10.6. Same exact behaviour. Do you have 10.6 to make a simple test like: ext2 = arcpy.Extent(4743000,4518000,5240000,5309000) ras2 = arcpy.sa.CreateRandomRaster(1, 100, ext2) Alberto
... View more
05-18-2018
01:04 AM
|
0
|
4
|
610
|
POST
|
Hi everybody. I have a strange issue here when running arcpy.sa.CreateRandomRaster depending on the size of the extent. I'm running ArcGIS Desktop 10.6 with 64bit geoprocessing in Windows 10 if I run this code in the python window: ext1 = arcpy.Extent(4054000,2329000,4260000,2597000) ras1 = arcpy.sa.CreateRandomRaster(1, 100, ext1) everything works as expected and I get my raster but.... when I run: ext2 = arcpy.Extent(4743000,4518000,5240000,5309000) ras2 = arcpy.sa.CreateRandomRaster(1, 100, ext2) I get a popup window saying that the background processing has encountered a serious error. Obviously I get the same issue running the tool from the sa toolbox even in foreground processing mode. A colleague here with ArcGIS 10.5.1 with 64bit geoprocessing in Windows 10 does not absolutely have this issue. Any idea ? Alberto
... View more
05-17-2018
06:22 AM
|
0
|
6
|
715
|
POST
|
Dan, after contacting ESRI support they recognized it as a bug for both ArcGIS and ArcGIS Pro. BUG-000113559 : Mosaic operator ‘Mean’ produces the incorrect results in ArcMap BUG-000113661 : Mosaic operator ‘Mean’ produces the incorrect results in ArcGIS Pro It should be fixed in following releases of the software Ciao Alberto
... View more
05-03-2018
07:33 AM
|
1
|
2
|
2015
|
POST
|
Dan, no news. I was thinking to involve ESRI support and, for this purpose, I created a script that replicates exactly what I'm doing. Given an input folder containing the 366 rasters, it creates a geodatabase with a mosaic dataset pointing to the rasters and setting all the relevant properties. It also outputs three rasters: ./data/outputMeanRasters/MeanFromCellStats.tif containing the mean we know is correct ./data/outputMeanRasters/MeanFromMosaic.tif containing the mean from the mosaic ./data/outputMeanRasters/PercentVar.tif containing the percent variation The strange thing is that the previous script works with the 'None' mosaic method meaning that rasters are sorted according to ObjectID in the mosaic (although for the sake of calculating the mean it should be irrelevant). If you try to use 'By Attribute' using SupportNr field (which is created by the script in a way that the sorting is different from objectID ascending) you get a mean raster from the mosaic which is slightly different from the one gotten before. (??) Zip file in attachment. Unzipping it and double clicking the .py replicates everything (It works with 10.5.1...for older versions the signature of arcpy.SetMosaicDatasetProperties_management may be slightly different. Alberto
... View more
01-22-2018
07:34 AM
|
0
|
0
|
723
|
POST
|
Dan, still I cannot figure out what is going on.... I tried to check mosaic dataset properties in order to find the reason for observed difference among mean calculated with mosaic operator and cell statistics on source rasters I'am attaching a python script with the geoprocessing chain I run in ArcGIS 10.5.1 for mosaic creation and setting. It can be synthesized as follows: Create a mosaic dataset inside a file geodatabase Load the mosaic pointing to the 366 rasters I sent you before (previous attachment) Set the mosaic properties through gp function (mosaic method to none, operator to mean and other stuff.....) Then, I input mosaic dataset to "copy raster" (or alternatively I can make a mosaic layer keeping the default properties) which should output a raster that SHOULD BE IDENTICAL to the one created with cell statistics. As we have seen this is not the case. A note on ArcGIS Pro: I get the same exact raster I get in ArcGIS 10.5.1 when running copy raster against the mosaic dataset. Another detail: I'm attaching the output from copy raster Yr2008TestRasterMosMeanV2.tif (based on these settings) that is slightly different from the one I sent you before (Yr2008TestRasterMosMean.tif). I found out that the reason for this is the mosaic method : When set to None you would get Yr2008TestRasterMosMeanV2.tif. In this case the sorting of the rasters are based on the ObjectID When set to "By Attribute" with StdTime field and base value '2008/01/01' you would get a raster that is identical to Yr2008TestRasterMosMean.tif Given that: For the sake of calculating our mean the sorting is irrelevant I purposely scrambled ObjectID and StdTime in a way that sorting by ObjectID is different from sorting by StdTime ....there must be something wrong in the mosaic dataset Alberto P.S. I gotta solve this otherwise I cannot sleep !!
... View more
12-12-2017
10:02 AM
|
0
|
7
|
652
|
POST
|
Dan, thanks for taking care of this.. I'm still out of office (the 8th of December is bank holiday in Italy). On Monday I can make other tests... Out of the many I already did one was with ArcGIS Pro 2 getting (if I remember correctly) the same issues. I agree that float32 cannot be an issue. You can try to use dtype 64 when averaging over axis = 0 and you would get irrelevant differences. Also ArcGIS Pro is 64 bit application but I think that if the rasters (and the mosaic dataset) are 32 bit nothing should change except the ability to reference more than 2 GB of memory (but this is not the case given the very small size of the test data). I'll keep you posted Thanks again Alberto
... View more
12-08-2017
10:58 AM
|
0
|
0
|
652
|
POST
|
Dan, I think that if you made r_m.mean() on the temporal axis (axis = 0) and converting back to raster you should get Yr2008TestRasterMean. We know that this works. The problem is the mosaic dataset that is supposed to properly calculate the mean of a pile of rasters as an aggregation on the temporal dimension (or whatever we want to call it). Thanks again for your help Alberto
... View more
12-07-2017
02:48 PM
|
0
|
1
|
652
|
POST
|
Dan, I'm out of office... I can only see at your figures but it completely makes sense for me. Your r0m.max() and r01m.max() are exactly the values at the location with max discharge (po river outlet) In my case I did a per cell percent variation calculating a raster like this: perc var = abs(Yr2008TestRasterMosMean - Yr2008TestRasterMean) / Yr2008TestRasterMean Statistics on percvar are a mean of 3% and a max of 10%. You see at cell with max it should be 1670 but I get 1615. Why ? Why Yr2008TestRasterMosMean is different from Yr2008TestRasterMean (which I know is correct ) ? What have I done wrong when obtaining the mean mosaic layer ? I do not need the mosaic dataset to do these things but it's a convenient way to expose netcdf and multidimensional stuff to people that do not know how to use numpy, netcdf4 and so on...Besides, ESRI is advertizing it this way. Thanks Dan Alberto
... View more
12-07-2017
02:23 PM
|
0
|
1
|
1292
|
POST
|
Dan, Year2008TestRasterMean.tif (output from cell statistics) is OK because it's exactly what I get using other methods (Aggregation of the numpy array from the original netcdf, using sample in sa...). The problem is in the mosaic dataset. It just references the same rasters....I cannot get it !! Basically I set the mosaic as follows: Mosaic dataset in a file geodatabase with ETRS_89_LAEA projection "Add rasters", pointing to the folder where I have the 366 rasters I gave you , without building overviews I then set the mosaic operator to Mean I set "MaximumNumber of Rasters per Mosaic" = 400 to include them all. This property if not set correctly would have an impact on the mosaic operator limiting the number of rasters used After I make a mosaic layer (dragging and dropping in ArcMap or using the gp tool) double checking that the default mosaic operator still is 'mean'. I run copy raster....and I get what you saw. If you had time to try something I would ask you to make a mosaic dataset out of the 366 rasters and see if you can get the mosaic mean that matches the one from cell stats (or one that matches mine). Thanks again Alberto P.S. If somebody else would be able to confirm my output from the mosaic dataset I would start to think that there is something weird in the mosaic algorithm
... View more
12-07-2017
01:00 PM
|
0
|
3
|
1292
|
POST
|
Dan, thanks for your reply. Actually I took into account that when running cell statistics. Also when getting the mean from the mosaic I've been also carefull to mosaic properties that can prevent the use of all the rasters. Basically the mosaic is a cube of 366 overlapping rasters. I just want the mosaic to return the mean of the 366 rasters......it works but when I make a percent variation out of cell stats versus mosaic layer I have an average of 3% and a maximum of 10%. I'm attaching a simple test scenario with the 366 rasters (File Year2008TestRaster.zip) as well as what I get from running mean cell statistics (file Yr2008TestRasterMean.zip) and what I get from the mosaic (pointing to the same set of 366 rasters) with the operator mean (File Yr2008TestRasterMosMean.zip). If you (or somebody else..) have few minutes to tell me how to get the same values from the mosaic....... There must be something I misunderstood about mosaic datasets Thanks a lot Alberto
... View more
12-07-2017
10:46 AM
|
0
|
0
|
1292
|
POST
|
Hi guys. I have the following problem (probably due to my lack of confidence when using the mosaic dataset): I have a set of 366 rasters (one for each day in 2008) with the following specs: 1000 columns, 950 rows 1 band Cell size is 5 km TIF format 32 bit floating point Coordinate system is ETRS_1989_LAEA The question is : Why am I getting different results when calculating mean with spatial analyst cell statistics versus a mosaic dataset loaded with the same set of rasters ? There may be a difference of almost 2% Cell statistics seems to be the correct one because I'm getting the same values in other ways. Rasters are coming from a big netcdf and I verified the avg values of the mean of the same slices using numpy (as well as other tools like sample and multi dimension ones..) I'm using ArcGIS 10.5.1 on Windows 8.1 workstation with 16 GB of RAM and I've been very carefull to set up the mosaic dataset with the correct properties(pixel depth of the mosaic is 32 bit and maximum size of requests are properly set). I also created a mosaic dataset based on the original netcdf (9131 temporal steps) getting the same results of the tif based mosaic when making a query definition for 2008 year and settimg operator to mean. Values for the rasters are in range 0 - 10000 more or less Thanks a lot to whoever is able to give me a hint Alberto
... View more
12-06-2017
09:08 AM
|
0
|
23
|
3396
|
POST
|
Supriya, I forgot to restart it !! I did it and now it works. Thanks !! Alberto
... View more
09-12-2017
08:46 AM
|
0
|
0
|
1217
|
POST
|
Supriya, it did not work. I installed the old ArcSDE installation files (10.2.2) but the result is the same. I did everything as sysadmin (os and sql server). I Also did the following: I tried to use older libst_raster_sql.dll and createAssembly.sql (coming from ArcGIS 10.4 installation instead of 10.5.1). For consistency I used 10.4 client when using "Create Raster Type" and also the geodatabase was 10.4 schema (even if probably does not make any difference) I double checked .NET requirements on the server and they should be OK especially because there is an installation of ArcGIS Server (4.5 or superior required) I'm using SQL Server 2014 on Windows Server 2012 R2 Standard. The server (staging machine) also has an installation of ArcGIS Desktop & Server 10.4. Can you confirm that what I'm experiencing is not a bug of the ST_Raster assembly ? Consider that the installation seems succesfull (I get 1000 from .st_raster_util_getVersion()) and I'm able to load a raster as ST_Raster (importing it from arcgis using a conf keyword) and using properties of the ST_Raster. When I try to go for pixel data I get the described trouble. Alberto
... View more
09-12-2017
01:47 AM
|
1
|
2
|
1217
|
Title | Kudos | Posted |
---|---|---|
1 | 08-21-2019 05:51 AM | |
4 | 09-06-2017 01:15 AM | |
1 | 09-12-2017 01:47 AM | |
1 | 05-03-2018 07:33 AM | |
3 | 07-14-2017 07:20 AM |
Online Status |
Offline
|
Date Last Visited |
3 weeks ago
|