Latest Contributions by GuenterDoerffel

Sources and further reading The data I use for the blog is open government data from Vienna, Austria. Find it at Offene Daten Österreich | data.gv.at. This site is in German.    See the help topic Use mosaic dataset items in raster function templates for raster function templates (RFT) that work on an Item Group level. I will only reference the UI and workflows in ArcGIS Pro. The template would work in ArcMap the same way.   nDSM—Two model suggestions At any location, the Normalized Digital Surface Model (nDSM) is the difference between the built-up/overgrown surface, or DSM, and the bare ground, or DTM, elevation. In this calculation, the bare ground is 0 everywhere by definition. Objects like trees or buildings will be represented as absolute heights above local terrain. The simplest raster function to create nDSM from DSM and DTM is to only use the Arithmetic  function (available as a raster core function): GuenterDoerffel_0-1627989680460.png     However, elevation datasets may contain errors. For DSM data derived from lidar, this risk is quite high (birds, planes). QC of the data is a prerequisite. In cities, the highest buildings are usually known; with vegetation, this might become more difficult. See topics like Locate outliers (3D Analyst) for that, or fixing elevation dataset voids using the Elevation Void Fill function. In other cases, the DSM might be more correct (a recent excavation site is correct in the DSM but not in the [maybe older] DTM). This is easier to fix—assuming an nDSM can never be below 0. This fix could be built into the raster function chain, using the Less Than and the Con functions (both available with ArcGIS Spatial Analyst and/or ArcGIS Image Analyst licenses): GuenterDoerffel_1-1627989680476.png   This version of the nDSM raster function chain will substract the DTM from the DSM in the Arithmetic function and send the result into the Less Than function. The second raster for the Less Than function is a constant raster of 0. Less Than is a Boolean function: If the elevation value is less than the constant value (of 0), the condition is True, else False. This Boolean result is handed over to the Con function, which performs a conditional if-then-else and will either assign the (positive) value of the Arithmetic function to the output cell, or the value of 0.   In the two use cases below, I will use the second model and explain all the settings. If you build on top of this, you can customize it for further needs.   Two use cases—Many data sources The following description is basically independent of the data sources used as the elevation data source. It could be ASCII point data, lidar, or the result of drone processing, for example. The only important prerequisite is that you need to have DSM and DTM data of an overlapping area. This is where the two use cases differ: Use case—Separate MDs: DSM and DTM have different origin/tiling/schema/dates ==> are in different MDs. Use case—One MD: DSM and DTM have essentially a 1:1 relationship (like the same tiling schema and dates) ==> and here can be held in the same MD.   Use case: Separate MDs When collecting DSM and DTM datasets over time and adding them to an (always improving/recent) MD, there might be different datasets coming into both from the sources mentioned above or even from a land survey. You can maintain them independently and create the nDSM as a function, available on one of them or both—you decide. Following is the workflow to add the nDSM raster function to the DSM MD created: GuenterDoerffel_2-1627989680533.png    Here are the two source MDs side by side: At left, the DSM (central Vienna, Stephansdom area); at right, the DTM with the rasters listed below in TableView. In the Catalog window, it looks like this: GuenterDoerffel_3-1627989680535.png   In this Catalog view, right-click your DSM layer and select the Manage Processing Templates entry: GuenterDoerffel_4-1627989680536.png     If there are no processing templates yet, or to create the one you need, add a new template: GuenterDoerffel_5-1627989680539.png   The Function Editor will open with an empty template called None. Open the properties first: GuenterDoerffel_6-1627989680541.png   This will display the Edit Properties form—see the important settings noted in red: GuenterDoerffel_7-1627989680549.png    This is the name displayed later in the UI for selecting this template—choose one that is unique and self-explanatory.  The function template you create will apply to the mosaic as such—this is important.  Think about creating and adding a visual for the function—not mandatory, though—then click OK to confirm.   The (still) empty Function Editor appears—time to add the functions. Open the Raster Functions pane: GuenterDoerffel_8-1627989680550.png   In the Search box of the Raster Functions pane, enter "Arith" to quickly find the Arithmetic function. Drag it into the Function Editor: GuenterDoerffel_9-1627989680552.png   It now looks like this: GuenterDoerffel_10-1627989680555.png   Double-click the Arithmetic function to open the settings, and fill in accordingly: GuenterDoerffel_11-1627989680557.png    No need to do something here—this references the current mosaic.  Navigate to your DTM mosaic here and select it.  This is the place to select Minus as the mathematical operator— leave the other defaults and click OK.   The Function Editor now looks something like this. Click the Save button to confirm: GuenterDoerffel_12-1627989680560.png   Once clicking Save, the properties will appear again. Just click OK.[Edit OK?] As a result, the name of the function template should have changed to the one you gave it:   GuenterDoerffel_13-1627989680562.png   Don't worry: There is no dataset connected to the raster input yet. Once closed/reopened, it will display.   If there is no error message, the function template is okay and can actually be used (this is the simple version described above). But now you'll improve it with the model setting that avoids values below 0.   Open the Raster Functions pane again, and first search for "Less", then for "Con", and add both to the Function Editor: GuenterDoerffel_14-1627989680564.png   Once they are added, you will need to make the necessary connections. Start with connecting the output of the Arithmetic function with the Less Than function. You will want the output of the Arithmetic function to be the first raster in the Less Than function: GuenterDoerffel_15-1627989680567.png   As Raster2, the second raster, is the one to check against with the Less Than function, you need a raster that has only 0 values. You get this by creating it in the Function Editor, using the Add Constant button: GuenterDoerffel_16-1627989680568.png   As this is by default 0, you only have to connect this 0 raster to Raster2 of the Less Than function: GuenterDoerffel_17-1627989680571.png   Now, you'll have to connect the three inputs that the Con function needs. These are called Raster, True Raster, and False Raster—and those three have to be connected like this: Raster is the output of the Less Than function, which is Boolean (either true or false).  True Raster is what needs to be done when Less Than has returned True, so this needs to be connected to the 0 raster (again). False Raster is what needs to be added when Less Than has returned False. The output of the Arithmetic function needs to be connected here (again).   Here's how it looks once you are done connecting: GuenterDoerffel_18-1627989680575.png   You can clean it up using the Auto Layout button, then it should look like this: GuenterDoerffel_19-1627989680579.png   Save and close and reopen once: Now you'll also see the reference (shown in green) to the DSM mosaic itself, as mentioned above: GuenterDoerffel_20-1627989680583.png     You can now use this template in the UI of ArcGIS Pro and ArcMap or through services: GuenterDoerffel_21-1627989680587.png     Use case: One MD From the perspective of an MD, this is the more advanced and interesting one, and it is applicable very often as map sheets or common tiling schemes or similar processing is applied to elevation data. While the Raster function as such is not different from the first use case, the way to specify the data is different and shows the flexibility of the mosaic dataset. The DSM and DTM datasets have to be filtered and then assigned as input. You'll start like in the first use case and see the result of a query on the map and table of the MD: GuenterDoerffel_22-1627989680647.png   In this dataset, for any selected tile in the map, two records will be found. In the attribute table Name will show the property the datasets share, here, the tile (could also be the date). GroupName in this example uses the built-in field to specify whether this is the DEM or the DSM dataset. In other workflows, a separate field called DEM Type is used for this. Important: What needs to be fulfilled here is that every request for the common item (here, Name equals Tile) returns just one item of the specified group (here, GroupName equals DSM or DEM). They do not necessarily need to overlap 100 percent. Every request can contain more than one of the (here) tiles—like in the screenshot above to the left, nine tiles would be returned! This is the needed data and table structure to use the second version of a Raster function template, which will use the records within the MD to do the nDSM calculation.   Before going through the workflow – another hint: To just use DEM or DSM from this dataset and base whatever function template on those is very easy, just use a definition query:   GuenterDoerffel_23-1627989680656.png   In this example, the definition query (highlighted) would filter out only the DSM datasets for use with this Raster function. But back to the workflow for your use case One MD and nDSM: This dataset with DSM and DTM in it looks like this in the Catalog window: GuenterDoerffel_24-1627989680658.png   Right-click this one and select the Manage Processing Templates entry: GuenterDoerffel_25-1627989680659.png     If there are no processing templates yet, or to create the one you need, add a new template again: GuenterDoerffel_26-1627989680662.png   The Function Editor will open with an empty template called None. Open the properties first: GuenterDoerffel_27-1627989680664.png   This will display the Edit Properties form—see the important settings highlighted that differ from the first example: GuenterDoerffel_28-1627989680676.png   Choose a different name and description (and thumbnail) and make sure to adjust these important settings (noted in red): Change the Type parameter to work on Item Group now (instead of Mosaic previously). Type the field name here that reflects the common group identifier (in this example, the tile name). Type the field name here that contains the dataset type identifier (in this example, DSM and DTM).  Then click OK to confirm.   The (still) empty Function Editor reappears, like in use case 1 above. As previously, open the Raster Functions pane: GuenterDoerffel_29-1627989680678.png   In the Search box of the Raster Functions pane, enter "Arith" again to quickly find the Arithmetic function. Drag it into the Function Editor: GuenterDoerffel_30-1627989680680.png   It will now look like this: GuenterDoerffel_31-1627989680682.png   Double-click the Arithmetic function to open the properties, and fill in differently this time:   GuenterDoerffel_32-1627989680686.png   The Minus operator in the Parameters section is going to be similar. But now you will switch to the Variables section: GuenterDoerffel_33-1627989680690.png   This is where it all happens now! Remember, in the properties of the Raster Functions pane, you had specified in the Name field where the dataset type was to be found. Now, here you have to enter the values in this field that are to be used: In the Name column for the parameter Raster, specify DSM. In the Name column for the parameter Raster2, specify DEM Then click OK to confirm.   After these settings to the Arithmetic function, the remaining procedure is identical to the function chain above. So the following steps repeated here are just for your convenience—no change. ...   Open the Raster Functions pane again, and first search for "Less", then for "Con", add both to the Function Editor: GuenterDoerffel_34-1627989680692.png   Once they are in, you need to make the necessary connections. Start with connecting the output of the Arithmetic function with the Less Than function. You want the output of the Arithmetic function to be the first Raster in the Less Than function: GuenterDoerffel_35-1627989680694.png   As Raster2, the second raster, is the one to check against with the Less Than function, you need a raster that has only 0 values. You get this by creating it in the Function Editor, using the Add Constant button: GuenterDoerffel_36-1627989680696.png   As this is by default 0, you only have to connect this 0 raster to Raster2 of the Less Than function: GuenterDoerffel_37-1627989680698.png   Now, you have to connect the three inputs that the Con function needs. These are called Raster, True Raster, and False Raster, and these three have to be connected like this: Raster is the output of the Less Than function,  which is Boolean (either true or false).  True Raster is what needs to be done when Less Than has returned True, so this needs to be connected to the 0 raster (again). False Raster is what needs to be added when Less Than has returned False. The output of the Arithmetic function needs  to be connected here (again).   Here's how it looks once you are done connecting: GuenterDoerffel_38-1627989680701.png   You can clean it up by using the Auto Layout button, then it should look like this: GuenterDoerffel_39-1627989680704.png     Finally, save this function chain. You can now use this template in the UI of ArcGIS Pro and ArcMap or through services:   GuenterDoerffel_40-1627989680717.png     The two resultant RFT templates have been exported and are attached to this blog article so you can look at them in detail. Please do not forget to adjust according to your MD field names/data structure! ... View more