POST
|
Using either of ArcMap 10.5.1 and ArcGIS Pro 2.0.1 I am trying to keep a Mosaic Dataset updated with the contents of a folder of NetCDF files - where new NetCDF files are constantly being added -- but "Synchronize" doesn't seem to be doing what I expect as it fails to add new items: For my testing, I've created a new FileGeodatabase and a Mosaic Data set inside it. I've added a few NetCDF files to the MDS using the NetCDF raster type- works fine (using "Add rasters" Tool) I then move a few more NetCDF files into the directory where the first NetCDFs live I then run "Synchronize Mosaic Dataset" - hoping that both the first and the added NetCDF files would appear as references in my attribute table - but it's still showing just the original ones, that were added using "Add rasters". When I run synchronize I use the "Update With New Items" Option. This is the command expression in Python - here with a SQL filter, but I've tried it with and without that with no difference: arcpy.management.SynchronizeMosaicDataset(r"D:\OneDrive - NIWA\GISDATA\ArcGISPro_Projects\NetCDF_Synchronizing_CSM\NetCDFs.gdb\syncWithNoFilterNoDatafileExtension", r"Name LIKE '%nwpsfc\_2017%' ESCAPE '\'", "UPDATE_WITH_NEW_ITEMS", "SYNC_STALE", "UPDATE_CELL_SIZES", "UPDATE_BOUNDARY", "NO_OVERVIEWS", "NO_PYRAMIDS", "NO_STATISTICS", "NO_THUMBNAILS", "NO_ITEM_CACHE", "REBUILD_RASTER", "UPDATE_FIELDS", "CenterX;CenterY;Dimensions;GroupName;ProductName;Raster;Shape;StdTime;Tag;Variable;ZOrder", "UPDATE_EXISTING_ITEMS", "IGNORE_BROKEN_ITEMS", "OVERWRITE_EXISTING_ITEMS", "NO_REFRESH_INFO", "NO_STATISTICS") I've tried a couple of different versions of the above process - using a SQL clause as mentioned by Jake Skinner in another GeoNet post here: https://community.esri.com/thread/121638 As I'm using Windows 7 I've tried to disable Background Geoprocessing as advised here http://support.esri.com/en/bugs/nimbus/TklNMDYwNjAz - this had made no difference. Is it the fact that I'm dealing with NetCDF files in the synchronize process that makes it fail to update the Mosaic Dataset? Anyone else had this? Sounds like the above mentioned old thread is about the same or a similar issue - but from the description of the synchronize tool it can be used to "add new items" which is exactly what I need to do on a regular basis - but it fails. Using the manual "Add rasters" tool instead doesn't sound like a valid option? mosaicdataset netcdf synchronize This is the output from one of my runs: Start Time: Friday, 13 October 2017 1:26:17 p.m. 2017-10-13T13:26:18.275: Synchronizing items associated with raster type instance 'NetCDF' [ID: 1]. 2017-10-13T13:26:18.297: Completed synchronization: 29 items selected, 0 items synchronized. 2017-10-13T13:26:18.308: Adding new data source items associated with the raster type instance 'NetCDF' [ID: 1] last used on 2017/10/13 13:19:18.127 (local). 2017-10-13T13:26:18.309: Loading raster datasets 2017-10-13T13:26:18.324: Completed crawling 0 data source items. Added 0 mosaic dataset items. 2017-10-13T13:26:18.326: Error: 8004205f: No new mosaic dataset item was added. Edit: Thinking that it is the NetCDF file format which doesn't synchronize properly: The next output is after adding some other (non-NetCDF based) rasters to the same Mosaic Dataset and running a synchronize with some more of those afterwards - as you can see the NetCDF sync fails with "Error: 8004205f: No new mosaic dataset item was added." but synchronizing the other rasters is successful. Executing: SynchronizeMosaicDataset "D:\OneDrive - NIWA\GISDATA\ArcGISPro_Projects\NetCDF_Synchronizing_CSM\NetCDFs.gdb\syncWithNoFilterNoDatafileExtension" # UPDATE_WITH_NEW_ITEMS SYNC_STALE UPDATE_CELL_SIZES UPDATE_BOUNDARY NO_OVERVIEWS NO_PYRAMIDS NO_STATISTICS NO_THUMBNAILS NO_ITEM_CACHE REBUILD_RASTER UPDATE_FIELDS # IGNORE_EXISTING_ITEMS IGNORE_BROKEN_ITEMS SKIP_EXISTING_ITEMS NO_REFRESH_INFO NO_STATISTICS Start Time: Fri Oct 13 15:26:46 2017 2017-10-13T15:26:46.101: Adding new data source items associated with the raster type instance 'NetCDF' [ID: 1] last used on 2017/10/13 14:39:54.022 (local). 2017-10-13T15:26:46.102: Loading raster datasets 2017-10-13T15:26:46.115: Completed crawling 0 data source items. Added 0 mosaic dataset items. 2017-10-13T15:26:46.117: Error: 8004205f: No new mosaic dataset item was added. 2017-10-13T15:26:46.118: Adding new data source items associated with the raster type instance 'Raster Dataset' [ID: 2] last used on 2017/10/13 15:10:06.575 (local). 2017-10-13T15:26:46.120: Loading raster datasets 2017-10-13T15:26:46.149: Completed crawling 25 data source items. Added 25 mosaic dataset items. 2017-10-13T15:26:46.150: Synchronizing crawled data source items 2017-10-13T15:26:46.158: Synchronizing items associated with raster type instance 'Raster Dataset' [ID: 2]. 2017-10-13T15:26:48.083: Completed synchronization: 25 items selected, 25 items synchronized. 2017-10-13T15:26:48.084: Processing duplicate mosaic dataset items 2017-10-13T15:26:48.105: Removed 13 duplicate mosaic dataset items. 2017-10-13T15:26:48.109: Computing cell size levels 2017-10-13T15:26:48.110: Computing unique cell size values 2017-10-13T15:26:48.189: Computing maximum cell size values 2017-10-13T15:26:48.253: Updating maximum cell size values 2017-10-13T15:26:48.261: Computing minimum cell size values 2017-10-13T15:26:48.381: Updating visibility values of selected items 2017-10-13T15:26:48.400: Updating visibility values of selected items 2017-10-13T15:26:48.406: Computing maximum cell size for mosaic dataset 2017-10-13T15:26:48.458: Completed computing cell size ranges. 2017-10-13T15:26:48.515: Completed building boundary. Succeeded at Fri Oct 13 15:26:48 2017 (Elapsed Time: 2.49 seconds)
... View more
10-12-2017
06:07 PM
|
0
|
1
|
1133
|
POST
|
That looks like a nice way of scripting it. Have you seen the "Space Time Pattern Mining" Toolbox though? toolboxes\system toolboxes\space time pattern mining tools.pyt\utilities It contains tools to create a space time cube by converting your rasters into a multidimensional NetCDF file with a time dimension, and visualise that in time and space - I imagine you should be able to use that at least for visualisation of your time series - though I'm not sure that you can actually get a tabular (or other) output of the deltas between the timesteps. There is an "Emerging Hot Spot Analysis" tool in the toolbox which can be used to visualise statistically significant trend/change areas over time.
... View more
08-03-2017
05:55 PM
|
1
|
0
|
286
|
Title | Kudos | Posted |
---|---|---|
1 | 08-03-2017 05:55 PM |
Online Status |
Offline
|
Date Last Visited |
11-11-2020
02:24 AM
|