POST
|
James, I am far from being a NetCDF specialist, but the issues you face sound familiar. If you read this page of the ArcGIS Pro Doc: https://pro.arcgis.com/en/pro-app/latest/tool-reference/multidimension/essential-netcdf-vocabulary.htm you will find a hint regarding NetCDF conventions ... and that within ArcGIS we currently support the Climate and Forecast (CF) and the Cooperative Ocean/Atmosphere Research Data Service (COARDS) conventions. It also states "others might work" ... but to my experience this is exactly where the problems with coordinates and reading data/dimensions come from. The link above also has additional links regarding conventions in it. If you want to dig in deeper, ArcGIS Pro today comes with the pre-installed python library netcdf4 which might be a way to get to the data you want ... Regards Guenter
... View more
09-15-2023
02:05 AM
|
0
|
0
|
58
|
POST
|
Megan, To achive this, you have to enable "Individual Image Search", specify the Sentinel-Layer and the AquisitionDate Field. I have created and attached a short video that might be easier to follow than a lengthy text. Plus the dummy app shown in the video will be available for a few days. The control with the 3 vertical bars opens the UI to select a specific date available in the extent chosen. Hope this helps! G.
... View more
09-14-2023
12:12 AM
|
1
|
1
|
1550
|
BLOG
|
Sources and further reading The data I use for the blog is open government data from Vienna, Austria. Find it at Offene Daten Österreich | data.gv.at. This site is in German. See the help topic Use mosaic dataset items in raster function templates for raster function templates (RFT) that work on an Item Group level. I will only reference the UI and workflows in ArcGIS Pro. The template would work in ArcMap the same way. nDSM—Two model suggestions At any location, the Normalized Digital Surface Model (nDSM) is the difference between the built-up/overgrown surface, or DSM, and the bare ground, or DTM, elevation. In this calculation, the bare ground is 0 everywhere by definition. Objects like trees or buildings will be represented as absolute heights above local terrain. The simplest raster function to create nDSM from DSM and DTM is to only use the Arithmetic function (available as a raster core function): However, elevation datasets may contain errors. For DSM data derived from lidar, this risk is quite high (birds, planes). QC of the data is a prerequisite. In cities, the highest buildings are usually known; with vegetation, this might become more difficult. See topics like Locate outliers (3D Analyst) for that, or fixing elevation dataset voids using the Elevation Void Fill function. In other cases, the DSM might be more correct (a recent excavation site is correct in the DSM but not in the [maybe older] DTM). This is easier to fix—assuming an nDSM can never be below 0. This fix could be built into the raster function chain, using the Less Than and the Con functions (both available with ArcGIS Spatial Analyst and/or ArcGIS Image Analyst licenses): This version of the nDSM raster function chain will substract the DTM from the DSM in the Arithmetic function and send the result into the Less Than function. The second raster for the Less Than function is a constant raster of 0. Less Than is a Boolean function: If the elevation value is less than the constant value (of 0), the condition is True, else False. This Boolean result is handed over to the Con function, which performs a conditional if-then-else and will either assign the (positive) value of the Arithmetic function to the output cell, or the value of 0. In the two use cases below, I will use the second model and explain all the settings. If you build on top of this, you can customize it for further needs. Two use cases—Many data sources The following description is basically independent of the data sources used as the elevation data source. It could be ASCII point data, lidar, or the result of drone processing, for example. The only important prerequisite is that you need to have DSM and DTM data of an overlapping area. This is where the two use cases differ: Use case—Separate MDs: DSM and DTM have different origin/tiling/schema/dates ==> are in different MDs. Use case—One MD: DSM and DTM have essentially a 1:1 relationship (like the same tiling schema and dates) ==> and here can be held in the same MD. Use case: Separate MDs When collecting DSM and DTM datasets over time and adding them to an (always improving/recent) MD, there might be different datasets coming into both from the sources mentioned above or even from a land survey. You can maintain them independently and create the nDSM as a function, available on one of them or both—you decide. Following is the workflow to add the nDSM raster function to the DSM MD created: Here are the two source MDs side by side: At left, the DSM (central Vienna, Stephansdom area); at right, the DTM with the rasters listed below in TableView. In the Catalog window, it looks like this: In this Catalog view, right-click your DSM layer and select the Manage Processing Templates entry: If there are no processing templates yet, or to create the one you need, add a new template: The Function Editor will open with an empty template called None. Open the properties first: This will display the Edit Properties form—see the important settings noted in red: This is the name displayed later in the UI for selecting this template—choose one that is unique and self-explanatory. The function template you create will apply to the mosaic as such—this is important. Think about creating and adding a visual for the function—not mandatory, though—then click OK to confirm. The (still) empty Function Editor appears—time to add the functions. Open the Raster Functions pane: In the Search box of the Raster Functions pane, enter "Arith" to quickly find the Arithmetic function. Drag it into the Function Editor: It now looks like this: Double-click the Arithmetic function to open the settings, and fill in accordingly: No need to do something here—this references the current mosaic. Navigate to your DTM mosaic here and select it. This is the place to select Minus as the mathematical operator— leave the other defaults and click OK. The Function Editor now looks something like this. Click the Save button to confirm: Once clicking Save, the properties will appear again. Just click OK.[Edit OK?] As a result, the name of the function template should have changed to the one you gave it: Don't worry: There is no dataset connected to the raster input yet. Once closed/reopened, it will display. If there is no error message, the function template is okay and can actually be used (this is the simple version described above). But now you'll improve it with the model setting that avoids values below 0. Open the Raster Functions pane again, and first search for "Less", then for "Con", and add both to the Function Editor: Once they are added, you will need to make the necessary connections. Start with connecting the output of the Arithmetic function with the Less Than function. You will want the output of the Arithmetic function to be the first raster in the Less Than function: As Raster2, the second raster, is the one to check against with the Less Than function, you need a raster that has only 0 values. You get this by creating it in the Function Editor, using the Add Constant button: As this is by default 0, you only have to connect this 0 raster to Raster2 of the Less Than function: Now, you'll have to connect the three inputs that the Con function needs. These are called Raster, True Raster, and False Raster—and those three have to be connected like this: Raster is the output of the Less Than function, which is Boolean (either true or false). True Raster is what needs to be done when Less Than has returned True, so this needs to be connected to the 0 raster (again). False Raster is what needs to be added when Less Than has returned False. The output of the Arithmetic function needs to be connected here (again). Here's how it looks once you are done connecting: You can clean it up using the Auto Layout button, then it should look like this: Save and close and reopen once: Now you'll also see the reference (shown in green) to the DSM mosaic itself, as mentioned above: You can now use this template in the UI of ArcGIS Pro and ArcMap or through services: Use case: One MD From the perspective of an MD, this is the more advanced and interesting one, and it is applicable very often as map sheets or common tiling schemes or similar processing is applied to elevation data. While the Raster function as such is not different from the first use case, the way to specify the data is different and shows the flexibility of the mosaic dataset. The DSM and DTM datasets have to be filtered and then assigned as input. You'll start like in the first use case and see the result of a query on the map and table of the MD: In this dataset, for any selected tile in the map, two records will be found. In the attribute table Name will show the property the datasets share, here, the tile (could also be the date). GroupName in this example uses the built-in field to specify whether this is the DEM or the DSM dataset. In other workflows, a separate field called DEM Type is used for this. Important: What needs to be fulfilled here is that every request for the common item (here, Name equals Tile) returns just one item of the specified group (here, GroupName equals DSM or DEM). They do not necessarily need to overlap 100 percent. Every request can contain more than one of the (here) tiles—like in the screenshot above to the left, nine tiles would be returned! This is the needed data and table structure to use the second version of a Raster function template, which will use the records within the MD to do the nDSM calculation. Before going through the workflow – another hint: To just use DEM or DSM from this dataset and base whatever function template on those is very easy, just use a definition query: In this example, the definition query (highlighted) would filter out only the DSM datasets for use with this Raster function. But back to the workflow for your use case One MD and nDSM: This dataset with DSM and DTM in it looks like this in the Catalog window: Right-click this one and select the Manage Processing Templates entry: If there are no processing templates yet, or to create the one you need, add a new template again: The Function Editor will open with an empty template called None. Open the properties first: This will display the Edit Properties form—see the important settings highlighted that differ from the first example: Choose a different name and description (and thumbnail) and make sure to adjust these important settings (noted in red): Change the Type parameter to work on Item Group now (instead of Mosaic previously). Type the field name here that reflects the common group identifier (in this example, the tile name). Type the field name here that contains the dataset type identifier (in this example, DSM and DTM). Then click OK to confirm. The (still) empty Function Editor reappears, like in use case 1 above. As previously, open the Raster Functions pane: In the Search box of the Raster Functions pane, enter "Arith" again to quickly find the Arithmetic function. Drag it into the Function Editor: It will now look like this: Double-click the Arithmetic function to open the properties, and fill in differently this time: The Minus operator in the Parameters section is going to be similar. But now you will switch to the Variables section: This is where it all happens now! Remember, in the properties of the Raster Functions pane, you had specified in the Name field where the dataset type was to be found. Now, here you have to enter the values in this field that are to be used: In the Name column for the parameter Raster, specify DSM. In the Name column for the parameter Raster2, specify DEM Then click OK to confirm. After these settings to the Arithmetic function, the remaining procedure is identical to the function chain above. So the following steps repeated here are just for your convenience—no change. ... Open the Raster Functions pane again, and first search for "Less", then for "Con", add both to the Function Editor: Once they are in, you need to make the necessary connections. Start with connecting the output of the Arithmetic function with the Less Than function. You want the output of the Arithmetic function to be the first Raster in the Less Than function: As Raster2, the second raster, is the one to check against with the Less Than function, you need a raster that has only 0 values. You get this by creating it in the Function Editor, using the Add Constant button: As this is by default 0, you only have to connect this 0 raster to Raster2 of the Less Than function: Now, you have to connect the three inputs that the Con function needs. These are called Raster, True Raster, and False Raster, and these three have to be connected like this: Raster is the output of the Less Than function, which is Boolean (either true or false). True Raster is what needs to be done when Less Than has returned True, so this needs to be connected to the 0 raster (again). False Raster is what needs to be added when Less Than has returned False. The output of the Arithmetic function needs to be connected here (again). Here's how it looks once you are done connecting: You can clean it up by using the Auto Layout button, then it should look like this: Finally, save this function chain. You can now use this template in the UI of ArcGIS Pro and ArcMap or through services: The two resultant RFT templates have been exported and are attached to this blog article so you can look at them in detail. Please do not forget to adjust according to your MD field names/data structure!
... View more
08-03-2021
04:43 AM
|
1
|
1
|
4121
|
BLOG
|
Hi Vinay, I had a mail contact with Rohit already on the PointCloud model. How to use it really needs a Notebook to understand. It can only be used with the API and how to reference the base data and which properties to use should be made clear by a notebook G.
... View more
10-14-2020
09:13 AM
|
0
|
0
|
1923
|
POST
|
Hey Sorato, publishing 2.5 TB of Data through properly structured MosaicDatasets/ImageServices is not a problem as such. (Just an example: The Sentinel-2 Service from the Living Atlas is a MosaicDataset with 4 Mio records, so referencing 4 Mio Sentinel-2 scenes with 0.5 to 1 GB each, updated every night). The performance will mainly be defined by the data structure of your Images and the storage access speed (storage/network) available. If you have never dealt with those topics, you will find good guidance in the help system and even more so in the ArcGIS Imagery Workflows | Documentation. There is a What are Imagery Best Practices?—Imagery Workflows | Documentation section about formats and organization. You might also contact your local Distributor for assistance. Regards Guenter
... View more
10-11-2020
11:50 PM
|
1
|
0
|
4383
|
POST
|
Dont you see THIS in Pro? Thats what you are looking for ...
... View more
09-22-2020
05:08 AM
|
0
|
1
|
1733
|
POST
|
Laurent, ArcGIS has settings to take care of this resolution issue: As a property of many band related functions you can specify your choice for the output cellsize type using the selector: This is part of all math functions, too. If you have simplified the process and use the "user defined" Method of the Band Arithmetic function (which your question indicates), your mix of band-resolutions will be resolved as the "best available". I tested the line that I have suggested with Sentinel2 in "Band Arithmetic" and it works just fine: RGB (4/3/2) BandArithmetic Function: (B12 / B13) + (B8 / B13) + (B8 / B3) Sorry - some city-patch - not ideal for regiolith G.
... View more
08-25-2020
02:21 AM
|
0
|
0
|
3242
|
POST
|
Hi Wade, to answer this one first: In a MosaicDataset you can create "custom" band definitions: which lead to these custom band names being accessible in UI ... Raster Functions ... Python code .. The gp-tool "arcpy.SetMosaicDatasetProperties_management()" will allow you to also SET those definitions, but personally I never used that G.
... View more
07-24-2020
01:42 AM
|
0
|
1
|
4503
|
POST
|
Best: Create a MosaicDataset (where in the definition of the Mosaic, you can define Band Names, bandwidth, ...) and then load your dataset into it ... also will allow you to define proper processing templates for suitable band combinations G.
... View more
07-22-2020
11:31 PM
|
0
|
0
|
4504
|
POST
|
Nada, no sure what this xml is you get - when I try (with Chrome) in EarthObservation Explorer I do get a TIFF. A few thoughts/questions: Why do you download Sentinel-2 data? Cant you use the same ArcGIS Online Services in ArcGIS Pro? See Sentinel-2 Views in the Living Atlas! Have you logged in to Earth Observation Explorer? Which kind of download did you try? Regards Guenter
... View more
07-22-2020
11:30 PM
|
1
|
1
|
5205
|
POST
|
Zoe, I highly recommend looking at this workflow and probably testing the free Toolbox offered here: Managing High-Resolution Satellite Imagery—Imagery Workflows | Documentation This is based on MosaicDatasets - go for it! G.
... View more
06-17-2020
09:55 AM
|
1
|
1
|
2246
|
POST
|
Laurent, what is your issue here? Not the results you expect? Without checking the literature: Does (B11 / B12) + (B8 / B12) + (B8 / B3) refer to the band naming/order in the Sentinel doc? Or is it (which would be crutial!)the band order in the dataset when in ArcGIS? Please keep in mind that there is a Band 8A in sentinel - and counting this is band 9 - so all following are one number up! So as a simple example: If you would need a formula like "NarrowNIR" / "Cirrus" in the sense of Sentinel doc this would be "8A" / "10" , but in the sense of band order in ArcGIS this would be "B9" / "B11" !! If that "error" happend on your formula above, the real formula most likely should be (B12 / B13) + (B8 / B13) + (B8 / B3) Regards Guenter
... View more
06-02-2020
12:30 AM
|
0
|
2
|
3242
|
POST
|
Did you use Pro 2.5 and the Raster Type for Sentinel 3 available since this release? Or did you try to use the native NetCDF data-files from the Scene-Folder (which wont work)?
... View more
05-19-2020
10:43 PM
|
0
|
1
|
1506
|
POST
|
Hi Benas, your inline question above on "Best practices" for Caching is best answered with a reference to the Image Management Workflows section of the Web doc. There is one workflow description: Serving cached Imagery that has a lot of helpful resources linked into it. It also contains a link to a little toolbox that is meant to help creating caches from Mosaic Datasets in a simplified way. Check this out first, please. Hope your NoData topic is solved. If not, feel free to send me access to one of the images. G.
... View more
05-01-2020
05:59 AM
|
1
|
1
|
1270
|
Title | Kudos | Posted |
---|---|---|
1 | 09-14-2023 12:12 AM | |
2 | 04-27-2020 01:37 AM | |
1 | 08-03-2021 04:43 AM | |
2 | 02-07-2019 01:42 AM | |
1 | 10-05-2018 12:42 AM |
Online Status |
Offline
|
Date Last Visited |
09-02-2024
10:29 AM
|