Latest Contributions by NicholasO_Connor

Hello Peter First and foremost thank you very much for following-up on this point and yes, “2 - Generate Pyramids for the data” worked. Perfect. Secondly, to aid my understanding, you wrote as an explanation, “I would typically recommend using pyramids so as to reduce the number of overviews and enables a user to lock the mosaic dataset to a specific raster and see the raster at all scales.” Up until now I have not generated “any” pyramids just “overviews” which has always produced good quality results and good performance. Your answer to generate pyramids gives the “good quality results” part but would it have any impact on the performance? I am still a little confused when are pyramids being used or when overviews are being used in ArcMap? I now see the Catalog table (name, objectid, minPS, MaxPs...) and when I had just Overviews it was simple to see the MinPS and MaxPS and calculate the item visibility  and thus the scales if tifs or overviews are being used. How do the Pyramids fit into this table? Finally, once again thank you for your help. ... View more

Hello Peter Thank you once again for following up on my issue. Very much appreciated. I did the following: For each 20 tifsI made a copy.... into a seperate Folder with a rename Executing:CopyRaster x:\test\1602\109M.tif x:\test\1602_copy\2.tif # # 256 NONE NONE # NONE NONE Executing: AddRastersToMosaicDataset x:\test\fGDB\test.gdb\Raster_1602_copy "Raster Dataset"  UPDATE_CELL_SIZES UPDATE_BOUNDARY NO_OVERVIEWS # 0 1500 # # SUBFOLDERS ALLOW_DUPLICATES NO_PYRAMIDS NO_STATISTICS NO_THUMBNAILS # NO_FORCE_SPATIAL_REFERENCE Executing: DefineOverviews x:\test\fGDB\test.gdb\Raster_1602_copy x:\test\overviews_quality\1602_copy\overviews # "620000 170000 690000 242000" 1.25 12 5120 5120 2 NO_FORCE_OVERVIEW_TILES BILINEAR LZW 80 Executing: BuildOverviews x:\test\fGDB\test.gdb\Raster_1602_copy # DEFINE_MISSING_TILES GENERATE_OVERVIEWS GENERATE_MISSING_IMAGES REGENERATE_STALE_IMAGES The result was the same poor quality overview images. To answer your questions abount the copied raster data: >>Check in the properties of the rasters (in catalog), does it state the pyramids exist. No >>If you add the rasters to ArcMap does it prompt to generate pyramids? Yes >>It is possible that if the original data is read only that the pyramids have been put in a cache directory No Any further help? Perhaps I could send you directly my tif data for testing? ... View more

Hello Peter I have uploaded two images (copyright © Data: Federal Office of Topography Swisstopo) – as screenshots from ArcMap at 1:25,000. Both sets of Overviews were generated in the same way but from different year tifs 2011 and 2015. See workflow below with details on how I defined the Overviews: arcpy.CreateMosaicDataset_management arcpy.AddRastersToMosaicDataset_management arcpy.DefineOverviews_management DefineOverviews x:\test\test.gdb\Raster_1101 x:\test\overviews # "620000 170000 690000 242000" 1.25 # 5120 5120 2 NO_FORCE_OVERVIEW_TILES BILINEAR LZW 80 arcpy.BuildOverviews_management arcpy.CalculateCellSizeRanges_management 1101_good_1to25k.jpg = good quality overviews (1st picture) 1602_bad_1to25k.jpg = poor quality overviews (2nd Picture) I hope you can see the difference? ... View more

Hello Peter Once again thank you for your feedback so far changing the resampling type has made no difference. Some more information that maybe helpful. Below shows the details of the tifs from 2015 which have poor overviews generated. The ones we had delivered in 2011 the overviews once generated are very good. The extent of the tifs from 2011 and 2015 is the same. The difference being in Source Type, NoData Value, Compression, Mensuration and Statistics. Could any of these play a role in the quality of the overview generation? 2015 (poor overviews) Each tif: Number of bands: 3 Cell size 1.25 x 1.25 Format: TIFF Source Type: Thematic Pixel Type: unsigned integer Pixel Depth: 8 bit NoData Value 256,256,256 ColourMap: absent Pyramids: absent Compression: LWZ Mensuration: None Statistices : not calculated 2011 (good overviews) Each tif: Number of bands: 3 Cell size 1.25 x 1.25 Format: TIFF Source Type: Generic Pixel Type: unsigned integer Pixel Depth: 8 bit NoData Value ColourMap: absent Pyramids: absent Compression: PACKBITS Mensuration: Basic Statistices : calculated ... View more

Hello Peter Thank you very much for your replay – greatly appreciated. Sorry I should have been a little more clear. The Raster Mosiac Datasets are being accessed only by ArcMap Users connecting to SDE 10.2.2 (Oracle 11.2) – with the actual tifs and overviews stored on the file system. I only generate overviews, I do not create any pyramids. Performance is excellent – the only issue is with the overview quality from certain tifs we have delivered. An example being one in the details above. Question: Do you feel that the Source Type in the original tif here Thematic (or Generic) plays a role combined with the Resampling Method BILINEAR and Compression as LWZ? ...Sorry I assume it is difficult to answer this without actually testing the data. However I will also follow-up too on some of your suggestions to see if we can improve our workflow. Kind Regards ... View more

Hi Gabriel, Larry + Everyone else who has helped. Finally got round to further testing. Both methods suggested by Gabriel (build overviews which are stale) and Larry (delete all overviews and build) worked with no active connections to the data. RemoveRastersFromMosaicDataset: "OBJECTID = x" for the Primary Rasters AddRastersToMosaicDataset: Add new Primary Rasters BuildOverviews: Generate Overviews + Regenerate Stale Overview Images Only checked Question to Gabriel you wrote,  "..I don't think you will need to use a query since the affected overviews have been marked while removing the primary rasters".. After I have deleted the Primary Rasters - do I see in any Table which overviews have been marked for removal? I still do not see the link at the database Level between the Primary Rasters and Overviews. Kind Regards Nicholas ... View more

Thanks for your inputs too Gabriel. Much appreciated. Since I am the SDE Administrator / Oracle DBA - I can make sure no one is accessing the data while I perform my "other job" as Raster Administrator 😉 I will try your and Larry's workflows and report back my findings. Once again thank you both for your feedback and I hope this forum posting helps others too. ... View more

Thanks Larry - time to start testing - I will report back as soon as I have some results. ... View more

Hello Larry – thank you for the feedback. As I understand it you suggest the following approach: [1] add the new rasters first (AddRastersToMosaicDataset) [2] rebuild the footprints of those 4 new additional rasters.( BuildFootprints_management) [3] select and delete the 4 “old” rasters (primary) as you have in the graphic (RemoveRastersFromMosaicDataset_management) [4] select all Overviews under Category, and delete ... (RemoveRastersFromMosaicDataset_management) [5] Define Overviews and rebuild ...(DefineOverviews_management / BuildOverviews_management) Is the tool in [2] correct? ... View more

Hello Gabriel and Larry – again thank you for your answers – I think the recipe for my mosaic dataset cookbook is almost ready. Sorry just some follow-up points. Firstly, some additional background we do a lot of pre-processing of our tifs. So there is no overlapping of images. Each tif is a rectangle, all the same size, same properties and they fit together “like squares on a chessboard”. Hopefully that is a ok analogy. Ok back to the 4 points. And again not real numbers just for the example. In our Footprint Table for AREA1 the OBJECTIDs for the original 1000 tifs stored in D:\RASTER\tifs\*.tif go from 1 to 1001. The OBJECTIDs for the tifs I wish to replace are 500,501,502,503 The OBJECTIDs for the Overviews in the Footprint Table go from 1002 to 40000 [1] When I use the “Remove Rasters From Mosaic Dataset” tool how would I set up the parameters? (CHECK or unchecked). Mosiac dataset: AREA1 Query Definition: where objectid in(500,501,502,503) Delete Overview Images: ? Delete Item Cache: <we do not generate a cache> Advanced Update Cell Size Ranges: ? Update Boundary: ? Marked affected overviews: ? Remove Mosaic dataset items: ? I am a little confused after reading the documentation on how/where the relationship between the original tiffs and the overviews are stored. So that if you delete the original tiffs, the associated overviews are also deleted. [2] ok – so the new tifs in the Footprint Table have OBJECTIDs 40001, 40002, 40003, 40004 [3] ok – hopefully 😉 [4] run the Build Overviews tool Mosaic dataset: AREA1 Query definition: Do I need to give the new OBJECTIDs here i.e. 40001, 40002, 40003, 40004? Define missing overview tiles: unchecked Generate overviews: CHECK Overview generation options Generate missing overview images only: unchecked Regenerate stale overviews images only: CHECK Is the above correct? ... View more

Hello Firstly, thank you both (Larry) for the feedback. Greatly appreciated. 1. Did you configure the cell size ranges (i.e., MinPS and MaxPS) or allow the software to do it? After loading the tifs and building the overviews I used the CalculateCellSizeRanges tool. 2. Did you configure anything in the Define Overviews tool or accept the defaults? I defined them myself – through trial and error. 3. Do you intend to update the affected overviews in addition to replacing the source images? Yes. 4. Are you OK with removing the old tiffs from the folder location where are the imagery is stored? Yes. ... View more

Cheers Kim - now all is clear. Thank you. Perhaps it was too many late nights migrating SDE to 10.2.2. I just needed fresh pair of eyes. ... View more