Community
Select to view content in your preferred language

Missing parts of overview layers

476
2
12-20-2024 07:38 AM
JenniferBorlick
by
Frequent Contributor
‎12-20-2024 07:38 AM

My end goal is to have an image service that has downloadable tiles in our open data portal. This dataset will also serve as our imagery basemap. (Unless that's a bad idea to have the open data and the basemap from the same service. If so, let me know, please and I'll duplicate this or the service.) I haven't ever done this before. The total area is 720 sq miles. Most of it is 6" resolution and there is a 3" resolution inset up the center (inside the black outline.) The imagery is 4-band MrSID, but I have it designated as Natural Color. I haven't done statistics or analyzed it yet. Other things to note, I am using ArcGIS Pro 3.3.1 and our server is 10.9.1. I do have access to Pro 2.9.x, and we are upgrading our server next month to 11.3 or .4. (I can't remember which. whatever the latest not ".0" version is. That's IT's call.)

I know there are overviews drawing for that area at larger scales. It looks like they pop in at about 1:41000. What can I do to get them to draw at the same levels as the rest of the dataset? I would appreciate any help or advice.

 

JenniferBorlick_1-1734706357700.png

I used this workflow:

  • Created the mosaic dataset
  • Defined Overviews
  • Added my 6" tiles
  • Added my 3" tiles
  • Defined Overviews to set the output location. I left all other parameters as their default settings.

 

 

=====================
Parameters

Mosaic Dataset ...
Output Location ...
Extent from Dataset Imagery.ARCGIS.Ortho_2024_Combined\Boundary
Extent 399704.915003829 199130.546969488 560326.695137579 329572.761806324
Pixel Size
Number Of Levels
Number Of Rows 5120
Number Of Columns 5120
Overview Sampling Factor 3
Force Overview Tiles NO_FORCE_OVERVIEW_TILES
Resampling Method BILINEAR
Compression Method JPEG
Compression Quality 80
Updated Mosaic Dataset ...
=====================
Messages

Start Time: Thursday, December 19, 2024 3:58:44 PM
2024-12-19T15:58:55.848: Defining overviews
2024-12-19T15:58:55.852: Computing default overview pixel size
2024-12-19T15:58:56.026: Defining overviews for pixel size: 0.4444444444444444
2024-12-19T15:58:58.556: Defining overviews for pixel size: 1.333333333333333
2024-12-19T15:59:00.631: Defining overviews for pixel size: 4
2024-12-19T15:59:01.833: Defining overviews for pixel size: 12
2024-12-19T15:59:02.044: Defining overviews for pixel size: 36
2024-12-19T15:59:02.151: Defining overviews for pixel size: 108
2024-12-19T15:59:02.268: Defining overviews for pixel size: 324
2024-12-19T15:59:02.456: Defining overviews for pixel size: 972
2024-12-19T15:59:02.562: Identified 6 overview levels at cell sizes: { 4 12 36 108 324 972 }
2024-12-19T15:59:03.087: Loading raster datasets
2024-12-19T15:59:03.405: Constructing overview items: region 1 of 1 at cell size 4
2024-12-19T15:59:04.168: Constructing overview items: region 1 of 1 at cell size 12
2024-12-19T15:59:04.320: Constructing overview items: region 1 of 1 at cell size 36
2024-12-19T15:59:04.374: Constructing overview items: region 1 of 1 at cell size 108
2024-12-19T15:59:04.394: Constructing overview items: region 1 of 1 at cell size 324
2024-12-19T15:59:04.417: Constructing overview items: region 1 of 1 at cell size 972
2024-12-19T15:59:04.536: Completed crawling 82 data source items. Added 82 mosaic dataset items.
2024-12-19T15:59:04.858: Processing duplicate mosaic dataset items
2024-12-19T15:59:06.500: Removed 0 duplicate mosaic dataset items.
2024-12-19T15:59:08.260: Synchronizing items associated with raster type instance 'Overview Instance 23' [ID: 23].
2024-12-19T15:59:14.032: Completed synchronization: 82 items selected, 82 items synchronized.
2024-12-19T15:59:14.058: Computing cell size levels
2024-12-19T15:59:14.061: Computing unique cell size values
2024-12-19T15:59:22.611: Computing maximum cell size values
2024-12-19T15:59:23.113: Updating maximum cell size values
2024-12-19T15:59:23.189: Computing minimum cell size values
2024-12-19T15:59:25.076: Updating visibility values of selected items
2024-12-19T15:59:25.196: Updating visibility values of selected items
2024-12-19T15:59:25.233: Computing maximum cell size for mosaic dataset
2024-12-19T15:59:25.280: Completed computing cell size ranges.
2024-12-19T15:59:25.282: Completed defining overviews
Succeeded at Thursday, December 19, 2024 3:59:39 PM (Elapsed Time: 55.19 seconds)

 

 

  •  Built Overviews using only the defaults. I did this several times because it never generated overviews over the whole area. The first two times I had an error and the areas that didn't generate were different. Please note that line 62 I cut out all of the successfully generated overviews.

 

 

Build Overviews
=====================
Parameters

Mosaic Dataset     ...
Query Definition     
Define Missing Overview Tiles     DEFINE_MISSING_TILES
Generate Overviews     GENERATE_OVERVIEWS
Generate Missing Overview Images Only     GENERATE_MISSING_IMAGES
Regenerate Stale Overview Images Only     REGENERATE_STALE_IMAGES
Updated Mosaic Dataset     ...
=====================
Messages

Start Time: Thursday, December 19, 2024 4:00:50 PM
2024-12-19T16:00:59.415: Defining overviews
2024-12-19T16:00:59.673: Defining overviews for pixel size: 0.4444444444444444
2024-12-19T16:01:02.123: Defining overviews for pixel size: 1.333333333333333
2024-12-19T16:01:04.561: Defining overviews for pixel size: 4
2024-12-19T16:01:06.283: Defining overviews for pixel size: 12
2024-12-19T16:01:06.631: Defining overviews for pixel size: 36
2024-12-19T16:01:06.880: Defining overviews for pixel size: 108
2024-12-19T16:01:07.164: Defining overviews for pixel size: 324
2024-12-19T16:01:07.317: Defining overviews for pixel size: 972
2024-12-19T16:01:07.571: Identified 6 overview levels at cell sizes: { 4 12 36 108 324 972 }
2024-12-19T16:01:07.685: Removed 0 mosaic dataset items and 0 unreferenced raster type instances.
2024-12-19T16:01:09.190: Loading raster datasets
2024-12-19T16:01:09.877: Constructing overview items: region 1 of 1 at cell size 4
2024-12-19T16:01:10.670: Constructing overview items: region 1 of 1 at cell size 12
2024-12-19T16:01:10.826: Constructing overview items: region 1 of 1 at cell size 36
2024-12-19T16:01:10.892: Constructing overview items: region 1 of 1 at cell size 108
2024-12-19T16:01:10.918: Constructing overview items: region 1 of 1 at cell size 324
2024-12-19T16:01:10.947: Constructing overview items: region 1 of 1 at cell size 972
2024-12-19T16:01:11.005: Completed crawling 82 data source items. Added 82 mosaic dataset items.
2024-12-19T16:01:11.415: Processing duplicate mosaic dataset items
2024-12-19T16:01:12.347: Removed 82 duplicate mosaic dataset items.
2024-12-19T16:01:15.485: Synchronizing items associated with raster type instance 'Overview Instance 23' [ID: 23].
2024-12-19T16:01:25.856: Completed synchronization: 82 items selected, 82 items synchronized.
2024-12-19T16:01:25.888: Computing cell size levels
2024-12-19T16:01:25.891: Computing unique cell size values
2024-12-19T16:01:36.989: Computing maximum cell size values
2024-12-19T16:01:37.486: Updating maximum cell size values
2024-12-19T16:01:37.873: Computing minimum cell size values
2024-12-19T16:01:39.718: Updating visibility values of selected items
2024-12-19T16:01:39.799: Updating visibility values of selected items
2024-12-19T16:01:39.826: Computing maximum cell size for mosaic dataset
2024-12-19T16:01:39.864: Completed computing cell size ranges.
2024-12-19T16:01:39.867: Completed defining overviews
2024-12-19T16:01:39.932: Marking stale and missing overview items
2024-12-19T16:01:41.235: Overview items marked for generation: 82
2024-12-19T16:01:41.532: Distributing mosaic dataset operation across 6 parallel instances on the specified host: [...].
2024-12-19T16:01:56.797: Error: 80048191: Unable to process remotely generated overview item. [-1]
2024-12-19T16:01:56.810: Error: 80048199: Failures reported while processing specified command item. [1553]
2024-12-19T16:01:56.818: Error: 8004021b: Chained Error ID: 0X8004021B
2024-12-19T16:01:56.827: Error: 8004021b: This release of the GeoDatabase is either invalid or out of date.
2024-12-19T16:01:56.833: Error: 80040351: DBMS table not found [42S02:[Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Invalid object name 'Imagery.dbo.GDB_Release'.] [Imagery.dbo.GDB_Release]
2024-12-19T16:01:57.846: Error: 80048191: Unable to process remotely generated overview item. [-1]
2024-12-19T16:01:57.848: Error: 80048199: Failures reported while processing specified command item. [1554]
2024-12-19T16:01:57.850: Error: 8004021b: Chained Error ID: 0X8004021B
2024-12-19T16:01:57.852: Error: 8004021b: This release of the GeoDatabase is either invalid or out of date.
2024-12-19T16:01:57.855: Error: 80040351: DBMS table not found [42S02:[Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Invalid object name 'Imagery.dbo.GDB_Release'.] [Imagery.dbo.GDB_Release]
...

2024-12-19T16:14:48.441: Completed synchronization: 61 items selected, 61 items synchronized.
2024-12-19T16:14:48.503: Distributing mosaic dataset operation across 6 parallel instances on the specified host: [...].
2024-12-19T16:14:52.151: Error: 80048191: Unable to process remotely generated overview item. [-1]
2024-12-19T16:14:52.154: Error: 80048199: Failures reported while processing specified command item. [1618]
2024-12-19T16:14:52.156: Error: 8004021b: Chained Error ID: 0X8004021B
2024-12-19T16:14:52.158: Error: 8004021b: This release of the GeoDatabase is either invalid or out of date.
2024-12-19T16:14:52.160: Error: 80040351: DBMS table not found [42S02:[Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Invalid object name 'Imagery.dbo.GDB_Release'.] [Imagery.dbo.GDB_Release]
2024-12-19T16:14:52.174: Error: 80048191: Unable to process remotely generated overview item. [-1]
2024-12-19T16:14:52.176: Error: 80048199: Failures reported while processing specified command item. [1615]
2024-12-19T16:14:52.179: Error: 8004021b: Chained Error ID: 0X8004021B
2024-12-19T16:14:52.182: Error: 8004021b: This release of the GeoDatabase is either invalid or out of date.
2024-12-19T16:14:52.184: Error: 80040351: DBMS table not found [42S02:[Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Invalid object name 'Imagery.dbo.GDB_Release'.] [Imagery.dbo.GDB_Release]
2024-12-19T16:14:52.215: Error: 80048191: Unable to process remotely generated overview item. [-1]
2024-12-19T16:14:52.218: Error: 80048199: Failures reported while processing specified command item. [1617]
2024-12-19T16:14:52.220: Error: 8004021b: Chained Error ID: 0X8004021B
2024-12-19T16:14:52.222: Error: 8004021b: This release of the GeoDatabase is either invalid or out of date.
2024-12-19T16:14:52.224: Error: 80040351: DBMS table not found [42S02:[Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Invalid object name 'Imagery.dbo.GDB_Release'.] [Imagery.dbo.GDB_Release]
2024-12-19T16:14:52.242: Error: 80048191: Unable to process remotely generated overview item. [-1]
2024-12-19T16:14:52.245: Error: 80048199: Failures reported while processing specified command item. [1613]
2024-12-19T16:14:52.251: Error: 8004021b: Chained Error ID: 0X8004021B
2024-12-19T16:14:52.253: Error: 8004021b: This release of the GeoDatabase is either invalid or out of date.
2024-12-19T16:14:52.255: Error: 80040351: DBMS table not found [42S02:[Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Invalid object name 'Imagery.dbo.GDB_Release'.] [Imagery.dbo.GDB_Release]
2024-12-19T16:14:55.339: Error: 80048191: Unable to process remotely generated overview item. [-1]
2024-12-19T16:14:55.341: Error: 80048199: Failures reported while processing specified command item. [1622]
2024-12-19T16:14:55.344: Error: 8004021b: Chained Error ID: 0X8004021B
2024-12-19T16:14:55.350: Error: 8004021b: This release of the GeoDatabase is either invalid or out of date.
2024-12-19T16:14:55.353: Error: 80040351: DBMS table not found [42S02:[Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Invalid object name 'Imagery.dbo.GDB_Release'.] [Imagery.dbo.GDB_Release]
2024-12-19T16:15:04.638: Generated item [69 of 82]: 'Ov_i17_L02_R00000313_C0000012A.tif'
2024-12-19T16:15:24.883: Generated item [70 of 82]: 'Ov_i17_L02_R00000311_C0000012D.tif'
2024-12-19T16:15:34.490: Generated item [71 of 82]: 'Ov_i17_L02_R00000313_C0000012C.tif'
2024-12-19T16:15:42.298: Generated item [72 of 82]: 'Ov_i17_L02_R00000313_C0000012D.tif'
2024-12-19T16:15:42.943: Warning: The specified participating mosaic dataset item is a partial or unprocessed overview. [Name: Ov_i17_L01_R00000934_C00000383.tif; ID: 1553]
2024-12-19T16:15:42.951: Generated item [73 of 82]: 'Ov_i17_L02_R00000311_C0000012B.tif'
2024-12-19T16:15:42.959: Warning: Generated partial overview image [73 of 82]: Ov_i17_L02_R00000311_C0000012B.tif. One or more participating rasters under this overview image could not be loaded.
2024-12-19T16:15:48.336: Generated item [74 of 82]: 'Ov_i17_L02_R00000312_C0000012D.tif'
2024-12-19T16:15:57.135: Generated item [75 of 82]: 'Ov_i17_L02_R00000312_C0000012C.tif'
2024-12-19T16:15:57.616: Synchronizing items associated with raster type instance 'Overview Instance 23' [ID: 23].
2024-12-19T16:15:58.638: Completed synchronization: 7 items selected, 7 items synchronized.
2024-12-19T16:15:58.700: Distributing mosaic dataset operation across 6 parallel instances on the specified host: [...].
2024-12-19T16:16:05.885: Warning: The specified participating mosaic dataset item is a partial or unprocessed overview. [Name: Ov_i17_L02_R00000313_C0000012B.tif; ID: 1622]
2024-12-19T16:16:05.888: Generated item [76 of 82]: 'Ov_i17_L03_R00000106_C00000063.tif'
2024-12-19T16:16:05.891: Warning: Generated partial overview image [76 of 82]: Ov_i17_L03_R00000106_C00000063.tif. One or more participating rasters under this overview image could not be loaded.
2024-12-19T16:16:06.788: Warning: The specified participating mosaic dataset item is a partial or unprocessed overview. [Name: Ov_i17_L02_R00000312_C0000012B.tif; ID: 1618]
2024-12-19T16:16:06.790: Generated item [77 of 82]: 'Ov_i17_L03_R00000105_C00000064.tif'
2024-12-19T16:16:06.793: Warning: Generated partial overview image [77 of 82]: Ov_i17_L03_R00000105_C00000064.tif. One or more participating rasters under this overview image could not be loaded.
2024-12-19T16:16:07.604: Warning: The specified participating mosaic dataset item is a partial or unprocessed overview. [Name: Ov_i17_L02_R00000312_C0000012A.tif; ID: 1617]
2024-12-19T16:16:07.611: Generated item [78 of 82]: 'Ov_i17_L03_R00000105_C00000063.tif'
2024-12-19T16:16:07.618: Warning: Generated partial overview image [78 of 82]: Ov_i17_L03_R00000105_C00000063.tif. One or more participating rasters under this overview image could not be loaded.
2024-12-19T16:16:18.171: Warning: The specified participating mosaic dataset item is a partial or unprocessed overview. [Name: Ov_i17_L02_R00000313_C0000012B.tif; ID: 1622]
2024-12-19T16:16:18.173: Generated item [79 of 82]: 'Ov_i17_L03_R00000106_C00000064.tif'
2024-12-19T16:16:18.175: Warning: Generated partial overview image [79 of 82]: Ov_i17_L03_R00000106_C00000064.tif. One or more participating rasters under this overview image could not be loaded.
2024-12-19T16:16:18.681: Synchronizing items associated with raster type instance 'Overview Instance 23' [ID: 23].
2024-12-19T16:16:19.408: Completed synchronization: 4 items selected, 4 items synchronized.
2024-12-19T16:16:19.496: Generating item [80 of 82]: 'Ov_i17_L04_R00000057_C00000021.tif'
2024-12-19T16:16:19.873: Warning: The specified participating mosaic dataset item is a partial or unprocessed overview. [Name: Ov_i17_L03_R00000106_C00000063.tif; ID: 1627]
2024-12-19T16:16:23.083: Warning: Generated partial overview image [80 of 82]: Ov_i17_L04_R00000057_C00000021.tif. One or more participating rasters under this overview image could not be loaded.
2024-12-19T16:16:23.407: Synchronizing items associated with raster type instance 'Overview Instance 23' [ID: 23].
2024-12-19T16:16:23.827: Completed synchronization: 1 items selected, 1 items synchronized.
2024-12-19T16:16:23.911: Generating item [81 of 82]: 'Ov_i17_L05_R0000001D_C0000000B.tif'
2024-12-19T16:16:24.269: Warning: The specified participating mosaic dataset item is a partial or unprocessed overview. [Name: Ov_i17_L04_R00000057_C00000021.tif; ID: 1629]
2024-12-19T16:16:24.887: Warning: Generated partial overview image [81 of 82]: Ov_i17_L05_R0000001D_C0000000B.tif. One or more participating rasters under this overview image could not be loaded.
2024-12-19T16:16:25.147: Synchronizing items associated with raster type instance 'Overview Instance 23' [ID: 23].
2024-12-19T16:16:25.559: Completed synchronization: 1 items selected, 1 items synchronized.
2024-12-19T16:16:25.651: Generating item [82 of 82]: 'Ov_i17_L06_R00000009_C00000003.tif'
2024-12-19T16:16:25.986: Warning: The specified participating mosaic dataset item is a partial or unprocessed overview. [Name: Ov_i17_L05_R0000001D_C0000000B.tif; ID: 1630]
2024-12-19T16:16:26.387: Warning: Generated partial overview image [82 of 82]: Ov_i17_L06_R00000009_C00000003.tif. One or more participating rasters under this overview image could not be loaded.
2024-12-19T16:16:26.775: Synchronizing items associated with raster type instance 'Overview Instance 23' [ID: 23].
2024-12-19T16:16:27.225: Completed synchronization: 1 items selected, 1 items synchronized.
2024-12-19T16:16:28.151: Completed generating overviews: 82 items selected, 82 eligible items processed, 75 items updated.
Succeeded at Thursday, December 19, 2024 4:16:46 PM (Elapsed Time: 15 minutes 56 seconds)
​

 

 

  • Then I tried Generating Missing Overviews only a couple of times the first result was:

 

Build Overviews
=====================
Parameters

Mosaic Dataset     ...
Query Definition     
Define Missing Overview Tiles     DEFINE_MISSING_TILES
Generate Overviews     NO_GENERATE_OVERVIEWS
Generate Missing Overview Images Only     IGNORE_MISSING_IMAGES
Regenerate Stale Overview Images Only     IGNORE_STALE_IMAGES
Updated Mosaic Dataset     ...
=====================
Messages

Start Time: Thursday, December 19, 2024 4:29:11 PM
2024-12-19T16:29:20.417: Defining overviews
2024-12-19T16:29:20.744: Defining overviews for pixel size: 0.4444444444444444
2024-12-19T16:29:23.031: Defining overviews for pixel size: 1.333333333333333
2024-12-19T16:29:25.237: Defining overviews for pixel size: 4
2024-12-19T16:29:26.700: Defining overviews for pixel size: 12
2024-12-19T16:29:26.873: Defining overviews for pixel size: 36
2024-12-19T16:29:26.960: Defining overviews for pixel size: 108
2024-12-19T16:29:27.068: Defining overviews for pixel size: 324
2024-12-19T16:29:27.151: Defining overviews for pixel size: 972
2024-12-19T16:29:27.239: Identified 6 overview levels at cell sizes: { 4 12 36 108 324 972 }
2024-12-19T16:29:27.351: Removed 0 mosaic dataset items and 0 unreferenced raster type instances.
2024-12-19T16:29:27.938: Loading raster datasets
2024-12-19T16:29:28.384: Constructing overview items: region 1 of 1 at cell size 4
2024-12-19T16:29:29.170: Constructing overview items: region 1 of 1 at cell size 12
2024-12-19T16:29:29.325: Constructing overview items: region 1 of 1 at cell size 36
2024-12-19T16:29:29.379: Constructing overview items: region 1 of 1 at cell size 108
2024-12-19T16:29:29.405: Constructing overview items: region 1 of 1 at cell size 324
2024-12-19T16:29:29.429: Constructing overview items: region 1 of 1 at cell size 972
2024-12-19T16:29:29.456: Completed crawling 82 data source items. Added 82 mosaic dataset items.
2024-12-19T16:29:29.713: Processing duplicate mosaic dataset items
2024-12-19T16:29:30.332: Removed 82 duplicate mosaic dataset items.
2024-12-19T16:29:32.391: Synchronizing items associated with raster type instance 'Overview Instance 23' [ID: 23].
2024-12-19T16:29:46.686: Completed synchronization: 82 items selected, 82 items synchronized.
2024-12-19T16:29:46.719: Computing cell size levels
2024-12-19T16:29:46.722: Computing unique cell size values
2024-12-19T16:29:55.683: Computing maximum cell size values
2024-12-19T16:29:56.141: Updating maximum cell size values
2024-12-19T16:29:56.194: Computing minimum cell size values
2024-12-19T16:29:58.004: Updating visibility values of selected items
2024-12-19T16:29:58.098: Updating visibility values of selected items
2024-12-19T16:29:58.148: Computing maximum cell size for mosaic dataset
2024-12-19T16:29:58.204: Completed computing cell size ranges.
2024-12-19T16:29:58.207: Completed defining overviews
Succeeded at Thursday, December 19, 2024 4:30:14 PM (Elapsed Time: 1 minutes 2 seconds)
​

 

  • The second result was:

 

Build Overviews
=====================
Parameters

Mosaic Dataset     ...
Query Definition     
Define Missing Overview Tiles     NO_DEFINE_MISSING_TILES
Generate Overviews     GENERATE_OVERVIEWS
Generate Missing Overview Images Only     GENERATE_MISSING_IMAGES
Regenerate Stale Overview Images Only     IGNORE_STALE_IMAGES
Updated Mosaic Dataset     ...
=====================
Messages

Start Time: Thursday, December 19, 2024 4:31:31 PM
2024-12-19T16:31:39.965: Marking stale and missing overview items
2024-12-19T16:31:43.255: Overview items marked for generation: 0
Succeeded at Thursday, December 19, 2024 4:31:57 PM (Elapsed Time: 26.44 seconds)
​

This is where I am now.

 

 

2 Replies
JenniferBorlick
by
Frequent Contributor
‎01-07-2025 11:47 AM

Found a solution:

Coming back to this question. I was able to fix it by defining overviews with -1 set as the Number of Layers. Then I built the overviews with Define missing overview unchecked.
Found the solution here.
AmyD
by
Emerging Contributor
‎01-29-2025 10:53 AM

Thank you for replying with your fix and saving me from chucking my PC out the window! I was missing the # of layers step myself because the tooltip said it was optional. 

Deeds will not be less valiant because they are unpraised.