Select to view content in your preferred language

Bag Bathymetric Mosaic dataset recently became limited to 20000 x 20000 pixels (heigh/width) to export.

1209
2
10-22-2023 02:16 PM
SonarBATStructureMAPS
New Contributor III

Hello,

In 2015, Esri support showed me how to create and export hi-res bathy mosaics built with hi-res BAG elevation survey data mosaic with Bag Bathymetry Imagery Layers saved from AGOL, then exported from ArcGIS Pro using the Copy Raster tool. I've used the same source Bag Bathymetry Mosaic Image Service (Image Collection) since day one since some of the other data sets that Bag Bathymetry Image Service limited exported data to maximum of 20000 pixels width and 20000 pixels height.

I've always used Esri (old) Map viewer and the following Bag Bathymetry Image ServiceImage Collection source to filter/extract all the footprints for a specific survey, then save the Image layer to my AGOL account.

{https://gis.ngdc.noaa.gov/arcgis/services/bag_bathymetry/ImageServer/WMSServer?}}

In ArcGIS Pro, I'd import survey specific Imagery Layers that were saved in my AGOL account in, then run the Mosaic Raster tool with Bag Bathy survey elevation data on the bottom and the hi-res Image Layer on top. The new mosaic would then be exported to stored somewhere outside the ArcGIS managed space. This worked great for many years and the mosaiced survey data was used to create hi-res seafloor maps that could be used in many different types of GIS applications as online data and offline source data for onboard nav systems. Recently, I found that the performing aforementioned steps terminates the Copy Raster Geoprocess shortly after it started, the message is created by the Copy Raster tool alerting the user of excessive use (over 20,000 pixels w/h.)

Exporting raster mosaics that are smaller than 20,000 x 20,000 px would require the very Copy Raster Geoprocess to run “a lot of times’ and consume a copious amount of time just to complete the export process for just survey that had an average area of extent.  Then you have 50 small sized raster datasets which would have to be mosaicked back into one raster file. It’s a ridiculous endeavor to even consider.

How are people working with hi-res bathy survey data now, i.e., creating the hi-res mosaic since then requirements to create mosaic persists, yet the data resources (that I used use) no longer support any sort of export process.

It's kind of frustrating that once post survey analysis is complete, and the data is not considered confidential, NOAA is mandated to provide free and open access to survey data for download and use by the public at large. However, time and time again, there are access issues which are intentionally implemented by NOAA  (or responsible organizations) which blocks or impedes full and open access to survey data. Making all the bag bathymetry mosaics services basically read-only, while only permitting tiny areas to be downloaded at one time serves what purpose?

 

Another example is NOAA's habit of posting (typically offshore) survey data as being open and available for public use, and processed to 'level 1' standards, which means anyone could download it and the data is processed into commonly used file formats (kmz, tif, ascii, etc). These file formats are supported for use in many well-known, inexpensive (or free) GIS applications like Google Earth or ArcGIS Online.  

 

The truth is that offshore survey data is routinely posted as 'level 1’ when it’s only processed to ‘level 0’. The entire collection of multibeam survey data that the user links to exists in vendor formats like raw.all.mb58, all.mb58, raw.all.56, all.mb56, mb162, etc. For John-Q public to open sonar data in vendor formats referenced above, the following would have to be true:

  1. Be an experienced software programmer with proficiency in Linux coding methods (and) be capable of leveraging Linux programming skills to execute complex code-based post processing on survey data using a hydrographic survey data processing program named MB-System. MB-System is designed specifically for programmers that code in Linux, and btw, MB-System only runs on Apple machines, so you’ll need to buy one of those if you want to use the "level 0" data released by NOAA. I'm not a programmer, so I only got a few pages deep in the MB-System cookbook before I realized I'd need to dedicate a few years learning how to read, write and speak in code before I could start processing survey data using MB-System. Not an option. 

 

or...

 

  1. Have the available financial resources to invest several thousands of dollars (each year) for yearly subscriptions to Fledermaus software tailored to meet the business requirements and solve data management problems faced bt hydrographic survey organizations. I was interested in the Fledermanus tools, so I reached out to their sales department. I found the quote for yearly subscriptions to be more than $5000 a year. I could only imagine the audible snorts of derision that could be heard from the salesman when he learned I was an in individual performing self-research and not a part a well-funded commercial entity seeking comprehensive software configurations for long term projects.
0 Kudos
2 Replies
MeredithPayne1
Esri Contributor

According to our raster team, the Copy Raster tool does not have a pixel limitation for the source data. However, it is possible that it is an issue stemming from the image service published from the mosaic dataset.  If the publisher has set a height/width limitation on the mosaic dataset those height/width limitations will pass to the image service. This will affect the Copy Raster tool to be limited to the same pixel size.

As the following screenshot shows, the pixel limitation of the mosaic dataset is 4100 in rows and 15000 in columns. This is the default pixel limitation, and it can be edited to larger number of pixels here.

MeredithPayne1_0-1698100478624.png

Then after published to image service, the limitation will show in the properties of the layer:

MeredithPayne1_1-1698100478628.png

This limitation will impact the usage of Copy Raster tool.

To solve this issue, the publisher would need adjust the setting in the source mosaic dataset. It might be worthwhile to contact NCEI to discover if this could be the case.

0 Kudos
SonarBATStructureMAPS
New Contributor III

Thanks for the feedback Merideth. Clipping the hi-res Imagery Layer from AGOL Map Viewer then creating a mosaic with Bag elevation band (.tif) in ArcGIS Pro, then exporting the mosaic using Copy Raster is the only way I've known for so many years. Seems like nobody is that concerned about the controls on exporting using Image Service layers. For me, a crucial resource is no longer available, so I cannot create mosaics moving forwards.  Everyone must be creating their mosaic datasets using differerent steps than I am. 

Any ideas? 

 

Thanks. 

 

0 Kudos