ArcGIS Spatial Analyst Blog - Page 2

cancel
Showing results for
Show  only  | Search instead for
Did you mean:

(34 Posts)

Unleash the power of RasterCellIterator to perform custom raster analysis

by
New Contributor II

If you’ve read through the Introducing the Raster Cell Iterator blog, you’ll be familiar with the basic concepts of the Raster Cell Iterator (RCI), a new ArcPy class added in ArcGIS Pro 2.5 in the Spatial Analyst module that allows you to iterate over raster cells, query and modify individual cell values.

In this blog, let’s explore how RCI can be used to customize your analysis on problems that cannot be easily solved using the existing out-of-the-box geoprocessing tool. In the first example, we will use RCI to create a new raster with a specific spatial pattern – checkerboard. And in two other examples, we will use RCI to customize two focal operations: 1) count the number of neighboring cells that have a different value from the center cell, and 2) calculate the minimum slope from a DEM.

Create a new raster with the checkerboard pattern

In atmospheric modeling, a heterogenous surface raster represented by a checkerboard pattern can be used as an initial condition for idealized model simulations. However, there is not an existing geoprocessing tool with which you can create such a raster directly. In earlier ArcGIS releases, you can accomplish this task by using NumPy, but it would require a multi-step workflow to do so.  In ArcGIS Pro 2.5 with RCI, this task has become much easier! You simply need to assign a binary value to each cell based on the current row and column while iterating through a raster.  Following is a simple code sample which does exactly that.

import json
from arcpy.sa import RasterInfo, Raster

# Create an empty RasterInfo object
myRasterInfo = RasterInfo()

# Load raster info from a Python dictionary
myRasterInfoData = {
'bandCount': 1,
'extent': {
'xmin': -107.0,
'ymin': 38.0,
'xmax': -104.0,
'ymax': 40.0,
'spatialReference': {'wkid': 4326},
},
'pixelSizeX': 0.01,
'pixelSizeY': 0.01,
'pixelType': 'U8',
}

# Convert myRasterInfoData to a JSON string and load into myRasterInfo
myRasterInfo.fromJSONString(json.dumps(myRasterInfoData))

# Create a new Raster object based on myRasterInfo
myRaster = Raster(myRasterInfo)

for (r, c) in myRaster:
# Checkerboard with 10 pixels width
if r % 20 < 10 and c % 20 < 10 or r % 20 >= 10 and c % 20 >= 10:
myRaster[r, c] = 1
else:
myRaster[r, c] = 0

myRaster.save(r 'C:\output\checkerboard.tif')  ‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

From the code snippet above, lines 8-23 show the steps to construct a RasterInfo object from scratch. Line 26 creates an empty raster based on the raster info specified in the previous step. Lines 28-33 show the main logic of creating a checkerboard pattern raster by assigning binary values to the cell based on the current row and column. An implicit way of using Raster Cell Iterator is demonstrated by calling a simple For loop to go through all the cells. Since there is only one raster involved in this analysis, you don’t need to create a RasterCellIterators object explicitly. It is recommended to follow this pattern when you are working with only one raster. Finally, at line 35, we are saving the output raster. Figure 1 shows the output raster with the checkerboard pattern.

Figure 1. Output raster with the checkerboard pattern

Custom Focal Operations

Focal operations produce an output raster dataset in which the output value at each cell is a function of the input cell value at that location and its neighboring cell values. With RCI, you can customize these kinds of focal operations with more flexibility.

Count the number of neighboring cells that have a different value from the center cell

Stathakis & Tsilimigkas (2015) proposed several metrics to calculate the compactness of cities using land use adjacency information. To derive the compactness metric, one of the pre-processing steps is to iterate through the land use data and count the number of neighboring cells that have a value different to the center cell. This focal operation process is illustrated in Figure 2.

Figure 2. Examples of input and output with the operation counting the number of neighboring cells that have a value different to the center cell.

RCI makes it much easier to iterate over all cells and compare each cell value with its neighbors using a simple if logic. Here’s how you can do it yourself:

from math import isnan
from arcpy.sa import Raster, RasterCellIterator

landu = Raster(r"C:\sample_data\landuse.tif")
# Create a temporary output raster based on the raster info of landuse
output = Raster(landu.getRasterInfo())

with RasterCellIterator({'rasters':[landu, output]}) as rci:
for i,j in rci:
count = 0
# Assign NoData value to the output if the input is NoData
if isnan(landu[i,j]):
output[i,j] = math.nan
continue
# Create a moving window
for x in [-1,0,1]:
for y in [-1,0,1]:
# Count the number of adjacent cells with a different value
if not isnan(landu[i+x,j+y]) and landu[i+x,j+y] != landu[i,j]:
count+=1
output[i,j] = count
output.save(r"C:\output\landuse_diff_count.tif")   ‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

In line 6, we are creating a new empty raster using the land use raster as an input template. Line 8 creates a RasterCellIterator object explicitly using input and output. In lines 9-21, we are looping through all cells in the input raster, and for each cell comparing its value to each of its immediate neighbors based on its relative position.

While performing the analysis, it is required to check for the NoData cells in the input, so that an appropriate value can be assigned to the output raster.  In lines 12 and 19, we are using isnan() function from the math module to check if the cell value in the input raster is NoData.  Line 13 uses nan from the same math module to represent NoData. In this example, NoData values are not considered while iterating through the cells and they are excluded from the neighbor cells when counting. The input land use data and the final output raster used in the sample code are shown in Figure 3.

Figure 3. The figure on the left shows the input land use raster dataset. The figure on the right shows the output raster with values ranging from 0 to 8, which counts the number of neighboring cells that have a different value from the center cell.

Calculate minimum slope from a DEM

For the Slope geoprocessing tool, the slope value of each cell is determined by the rates of change of the elevation in the horizontal and vertical directions from the center cell (How slope works). In some cases, you might be interested in calculating the slope using a different equation. For an example, when building a fish trawlability model, an analyst might want to use a slope of the seabed that is based on the minimum change of elevation across each cell. By creating a custom focal operation with RCI, the following example shows how to calculate the minimum slope for your bathymetric DEM:

from arcpy.sa import Raster, RasterCellIterator

dem = Raster(r'C:\sample_data\demtif.tif')
cell_x = dem.meanCellHeight
cell_y = dem.meanCellWidth
raster_info = dem.getRasterInfo()

# Set the output pixel type to be float 32
raster_info.setPixelType('F32')
min_slope = Raster(raster_info)
with RasterCellIterator({'rasters':[dem, min_slope]}) as rci:
for r,c in rci:
slopes = []
# Iterate through 8 neighbors
for x,y in [(-1,-1),(-1,0),(1,0),(0,-1),(0,1),(1,1),(-1,1),(1,-1)]:
# Calculate the slope from center cell to each neighbor
slope = abs(dem[r,c]-dem[r+x,c+y])/math.sqrt(cell_x**2+cell_y**2)
# Add all the slope values to a list
slopes.append(slope)
# Assign the minimum slope to the output cell value
min_slope[r,c] = min(slopes)
min_slope.save(r'C:\output\min_slope.tif')‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

In the code sample, lines 13-21 implement the main logic for calculating the minimum slope. For each cell from DEM, we iterate through its eight neighbors and calculate the rate of change. Then we find the minimum slope and assign that value to the output cell. Note that in line 9, the pixel type is set as ‘F32’, which means that the output is expected to be a 32-bit floating raster. In this example, the raster info of the slope output is inherited from the input DEM data. If the pixel type of your DEM data is integer, you need this step to make sure that the slope value of each cell is calculated as a floating-point value instead of an integer.

Summary

In this blog, we went through three different examples of using Raster Cell Iterator in ArcPy, each of them showing how easy it is to perform custom raster analysis. We hope you start using this simple but powerful capability to extend your own raster analysis capabilities.  Please let us know if you have any questions or want to see other examples and use cases.  You can also pass along any cool applications you come up with, and we can share your solutions with the community at large!

Reference:

Stathakis, D., & Tsilimigkas, G. (2015). Measuring the compactness of European medium-sized cities by spatial metrics based on fused data sets. International Journal of Image and Data Fusion6(1), 42-64.

The original blog post Unleash the power of RasterCellIterator to perform custom raster analysis has been written by Hao Hu and Yu Wang.

more
1 1 601

What’s New for Spatial Analyst and Raster Analysis in ArcGIS Pro 2.5

Esri Contributor

Do you use Spatial Analyst extension for performing raster analysis?  With the release of ArcGIS Pro 2.5, we have made significant changes in the areas of distance analysis, multidimensional raster analysis, and zonal analysis. There is also whole host of improvements across many other areas.  If you are comfortable in ArcPy, there is an exciting new way to perform custom raster analysis.  Also in ArcPy, many new functions are available for analyzing, visualizing, and managing raster data using Python.

We will have many more blog posts coming that go into much more detail on some of these changes, and they will be linked to below.

Where do I get it?

ArcGIS Pro 2.5 was released of in February 6 of this year.

For a quick summary of all the changes that have been made for these releases, have a look at the What's New topic (and video!) here:

ArcMap Desktop 10.8 and ArcGIS Enterprise 10.8 will be released very soon.  Watch this space, or your usual channels, for more details once they become available.

What's changed for Spatial Analyst?

Here are the main categories of the changes.

1. Distortion free distance analysis
2. Multidimensional raster analysis
3. Multiband raster analysis
4. Zonal analysis
5. General tool and function enhancements
6. Iterating over raster cells using ArcPy
7. Modifying raster cell values using ArcPy
8. Enhanced multidimensional raster processing with ArcPy
9. Additional raster functions available in ArcPy

1.  Distortion Free Distance Analysis

With ArcGIS Pro 2.5, Spatial Analyst sees a significant change in distance analysis capabilities.  First and foremost, a new algorithm for cost-based distance analysis provides more accurate and precise results than before.  By avoiding the distortion in output that was caused by the previously-used network model of cell connectivity, several benefits are achieved:

• Cost accumulation is measured the same way in all directions. An important special case is that cost distance with a constant cost surface now produces the same output as Euclidean distance mapping.
• Surface distance over a digital elevation model is computed much more accurately.
• Paths around barriers are followed more accurately.
• Cost distance accumulation and allocation analysis have the option to perform the analysis using either a planar or a geodesic method.

As part of making this improved capability available, the distance toolset has been reorganized into fewer but more capable tools.  The Corridor tool is carried over as-is. The new tools are:

Rest assured, all the tools that existed in the previous release are still available; they have simply been moved into a new sub-toolset.  We have a help topic that well covers the migration from legacy to distortion free distance tools.

To complement the new GP tools, two new raster functions are available in the Distance group:

Updates have been made to the following existing raster functions:

• The Least Cost Path function has been updated to use the Distance Accumulation tool to perform distortion-free distance analysis.
• The Cost Path function has been updated with a new parameter that allows you to determine a hydrological flow path.

Related blog:

In ArcMap 10.8, the Distance toolset remains basically the same as before.  The only exception is that the Cost Path and Cost Path As Polyline tools have been updated with a new parameter which provides the option to make those tools treat the input backlink raster as a flow direction raster instead.

2.  Multidimensional Raster Analysis

As part of the increased focus on multidimensional analysis in the ArcGIS platform, a number of tools in various toolsets have been updated to directly support multidimensional raster data. There is also a new pathway for accessing multidimensional analytical tools directly from the ArcGIS Pro interface.

In the Multidimensional Analysis toolset, the following tools have improvements:

In the Extraction toolset, the Sample tool has several new parameters to process multidimensional data.  When processing this type of data, the tool has options to specify the following:

• The output table layout, either as rows or columns
• The time, depth, or other acquisition information associated with the location features to extract information for a given time or calculate statistics within a period of the observation time
• Lines and polygons as the input location, in addition to points and rasters
• The Parallel Processing Factor environment

In the Zonal toolset, the Zonal Statistics tool, the Zonal Statistics as Table tool, and the Zonal Statistics raster function now have the Process as multidimensional parameter available, which allows you to calculate various statistics on each slice of a multidimensional value raster.

In ArcGIS Pro, for the Raster Layer tab, the new Multidimensional tab is enabled when a multidimensional raster layer or a multidimensional mosaic layer is selected in the Contents pane. It provides tools and functionality for working with multidimensional raster data.

3.  Multiband Raster Analysis

Not to be confused with multidimensional rasters, multiband rasters are raster datasets that are composed of two or more raster bands, or layers, where each band is a measure of a single characteristic. The following tools have changes pertaining to how multiband raster data is handled.

• The Aggregate Multidimensional Raster tool can now do band-wise aggregation for multiband multidimensional input.
• The Pick and Cell Statistics tools have a new parameter that allows you more control over how multiband input rasters are processed.
• The Cell Statistics raster function can calculate single-band or multiband output based on the multiband processing type.

4.  Zonal Analysis

In response to an often-requested capability, the Zonal Statistics as Table tool now processes overlapping polygon zones. The statistics for each zone are calculates separately.

5.  General tool and function enhancements

In addition to the changes described above, some new other new tools and tool enhancements are available.

• In the Raster segmentation and classification toolset, a new tool, Linear Spectral Unmixing, has been added to perform subpixel classification and calculate the fractional abundance of different land cover types for individual cells. The Export Training Data For Deep Learning tool has four new parameters.
• For generalizing rasters, the Aggregate tool now supports the Parallel Processing Factor environment. The Median option for the Aggregation technique now only produces a float output raster, regardless of the input type. The new Aggregate raster function dynamically generalizes a raster in the display at a reduced resolution. When the output from the function is persisted, it will be generated at the full resolution.
• A new Random raster function is available for creating a dynamic raster on the fly with random cell values.

6.  Iterating over raster cells using ArcPy

With ArcGIS Pro 2.5, a powerful new capability to perform custom raster analysis is available to you in ArcPy.  Two classes are introduced that allow you iterate over rasters programmatically.

The RasterCellIterator class in the Spatial Analyst module allows you to visit each cell location in a raster. The iterator makes it easy to inspect cell values at each location, as well as neighboring locations. While iterating over a raster, you can read and write cell values.

The RasterInfo class in the ArcPy classes describes a set of raster properties that facilitate the creation of a new raster dataset using the Raster object. This class has several methods and properties available.

Related blogs:

7. Modifying raster cell values using ArcPy

The Raster object now allows you to read cell values using a [row, column] index notation (also known as neighborhood notation).  In addition to reading values, you can also assign a new value to a particular raster cell, thereby modifying the Raster object. Once the values of raster cells have been modified through an assignment, you can persist these changes by calling the save() method on the Raster object.

8. Enhanced multidimensional raster processing using ArcPy

The Raster object has more capabilities for processing multidimensional rasters.  The additional properties and methods are:

• Properties—bandNames, blockSize, noDataValues, properties, and readOnly
• Methods—addDimension, exportImage, getProperty, getRasterBands, getRasterInfo, read, removeVariables, renameVariable, setProperty, and write

9. Additional raster functions available in ArcPy

In the Spatial Analyst module, many new ArcPy functions are available for analyzing, visualizing, and managing raster data using Python.  To keep things organized, they are grouped into functional categories, which you can see from Overview page.

New resources

Do you have our Spatial Analyst resources blog post bookmarked?

Not only does it include a list of links to informative content, we also work to keep it updated as new resources become available.  For example, here is some of the material that was recently added:

Summary

That about covers what's changed since the last release.  Once you've downloaded and installed the latest versions of the software, give it a try.  We hope you will enjoy the improvements and enhancements.

more
1 0 595

Introducing the Raster Cell Iterator

New Contributor III

Have you ever wished you could easily interact with raster datasets in ArcGIS Pro at an individual cell level? Be able to read and write cell values directly, and create your own custom analysis routines? Look no further, as the Raster Cell Iterator (RCI) in ArcGIS Pro 2.5, lets you do just that. RCI is available through the Spatial Analyst module, an extension of the ArcPy Python site package. With RCI, visit each cell location in a Raster object, all in a Python environment. The iterator makes it easy to query and modify cell values at each cell location and its neighboring cell locations. Iterable access to raster cells enables you to write custom raster analysis scripts and combine them with the existing suite of Spatial Analysis geoprocessing tools, expanding your analytical capabilities immensely!

Here is a short video that helps you get started with RCI and shows how to use it for custom raster analysis.

As you can see in this video, using RCI is very simple and requires few lines of coding. You can invoke RCI on a single raster object or multiple raster objects to iterate through their row and column indices. Using these indices, you can query a cell value at a given cell location. You can also query neighborhood cell values by using relative indexing. Additionally, you can use index notation to write cell values to an output raster.

Be sure to download the latest version of ArcGIS Pro 2.5 to try out RCI and explore its capabilities.

Read A quick tour of using Raster Cell Iterator to get started with RCI in ArcGIS Pro.

Read the blog Unleash the power of RasterCellIterator to perform custom raster analysis, which illustrates how to apply RCI to solve a read-world problem.

more
3 1 1,435

What's New in the Spatial Analyst Distance Toolset in Pro 2.5

Esri Contributor

The Spatial Analyst team has made some important improvements in the Distance toolset for ArcGIS Pro 2.5.
We have significantly improved the precision of the geoprocessing tools in the Distance toolset and the corresponding raster functions. We have also provided additional capabilities which let you answer some new “how much/how many/how far” questions.

These improvements allows us to remove the strong distinction between the “Euclidean” and “Cost” Distance tools. The number of tools has been reduced from 17 to 6. The new user experience is now simplified to:

1. Enter Sources

2. Do you have barriers?

3. Do you have a surface?

4. Do you have a cost surface?

5. Do you need to consider direction of motion and other source characteristics?

The new toolset organization is shown in Figure 1 below. The new tools are:

Figure 1 - Organization of the Distance toolset at ArcGIS Pro 2.5

The original Distance tools that you are familiar with are available in the  Legacy sub-toolset.
The Spatial Analyst team recommends that you move your Euclidean and Cost Distance analysis work to the new tools.

Why should I use the new tools?

At ArcGIS Pro 2.5, we’ve uniformly adopted a new algorithm [4] for all distance mapping tools. This algorithm removes the distortion in outputs caused by using a network model of cell connectivity, as shown in Figure 2 on the left below. This has the following benefits:

• Cost distance is now measured the same way in all directions. An important special case is cost distance with a constant cost surface now produces the same output as the Euclidean distance tools.
• Surface distance over a digital elevation model can now be precisely computed.
• Paths around barriers are now followed precisely, as shown in Figure 3.
 Figure 2- Output of legacy Cost Distance tool using a constant cost input with barriers encoded as ndoata, along with output of legacy Cost Path As Polyline tool. Figure 3 – Output of new Distance Accumulation tool with barrier inputs, along with output of new Optimal Path As Line tool.

At ArcGIS Pro 2.4, we introduced this improvement in a limited context: Euclidean distance with barriers. ArcGIS Pro 2.5 generalizes and extends this work.

What else is new?

Along with this fundamental change to the behavior of the cost distance algorithm, several additional capabilities have been added.

Figure 4- Geodesic flight paths from some Northern European airports to LAX, but avoiding paths over southern Greenland.

• As a side effect of the improved algorithm, you can now create buffer regions whose distance is measured over the surface of a digital elevation model. This isn’t really a separate capability, but it’s important enough to mention explicitly. In the example below, the study area is Lake Wakatipu near Queenstown, New Zealand – a high relief area. Notice how, in such areas, the output of the Distance Accumulation tool (gray gradient) with a maximum accumulation cutoff of 1500m “pulls away” from the output of the vector Buffer tool that used a buffer distance of 1500m. This surface-aware distance computation can be done in either planar or geodesic mode.

Figure 5 – Comparison of planar (blue) and surface distance (gray gradient) buffers around Lake Wakatipu in the Queenstown area of New Zealand. Both buffers extend to a distance of 1500m, but the gray gradient output measures distance over the surface of a DEM instead of distance in the plane of the data’s projected coordinate system.

• Using the new Optimal Path as Raster tool, you can count how many least cost paths pass through a cell en route from some destinations to a source.

Figure 6 – Optimal Path As Raster output from ridgeline hiking trail in SW to access road in NE. Green is less intense usage, red is more intense.

• Using the new source_location output from Distance Accumulation and Distance Allocation, you can do intensity mapping of the boundary of sources, showing where most cost paths in the study area would end up, without plotting all cost paths. The RasterCellIterator python object, also new at ArcGIS Pro 2.5, makes this type of intensity calculation very easy to perform. See A quick tour of using Raster Cell Iterator to know more about it.
• Figure 7 – Lake Wakatipu, New Zealand. Hot spots (red and blue fuzzy dots), computed from Distance Accumulation output, show locations of most intense drainage into lake from surrounding watershed.

Which geoprocessing tools and raster functions should I start using?

For each legacy distance tool that you are currently using, table 1 shows you which new tool to switch to.

Table 1 – Upgrade guide for distance mapping tools
If you usedThen switch to
Cost DistanceDistance Accumulation
Cost BacklinkDistance Accumulation with back direction output
Cost AllocationDistance Allocation
Euclidean DistanceDistance Accumulation
Euclidean AllocationDistance Allocation
Euclidean DirectionDistance Accumulation with source direction output
Path DistanceDistance Accumulation
Path BacklinkDistance Accumulation with back direction output
Path AllocationDistance Allocation
Cost PathOptimal Path As Raster
Cost Path As PolylineOptimal Path As Line
Cost ConnectivityOptimal Region Connections

For new workflows, the back direction raster replaces the back link raster. The Optimal Path as Raster/Line tools no longer accept a backlink raster as input (they will still accept a flow direction raster). If you were using the backlink raster in a workflow that involves something other than the Cost Path or Cost Path As Polyline tools, then the Spatial Analyst team would like to hear from you. The Euclidean distance blog mentioned above discusses the backdirection raster in more detail.If you used the maximum_distance parameter in the legacy cost tools, you should now use the Maximum Accumulation parameter in the “Characteristics of the sources” parameter group.

These are the new raster functions.

The functions have the same parameters as the tools, with an important difference: the functions optionally emit multi-band outputs. This avoids having to run these (possibly expensive) global functions twice to get closely related outputs (like accumulated cost surface and back direction raster).

Also, the Least Cost Path raster function has been updated to use the Distance Accumulation tool behind the scenes.

Finally, although not directly related to these enhancements, your advanced use of the Distance toolset needs to consider if direction of motion between sources and destinations is important. If so, please check out this blog on Creating accumulative cost surfaces using directional data.

References

1) Goodchild, M. F. (1977). An Evaluation of Lattice Solutions to the Problem of Corridor Location. Environment and Planning A. 9. 727-738. 10.1068/a090727.
2) Sethian, J. A. (1997). Tracking Interfaces with Level Sets: An “act of violence” helps solve evolving interface problems in geometry, fluid mechanics, robotic navigation and materials sciences. American Scientist Vol. 85, No. 3 (MAY-JUNE 1997), pp. 254-263
3) Sethian, J. A. (1999). Level set methods and fast marching methods: evolving interfaces in computational geometry, fluid mechanics, computer vision, and materials science (2nd edition). Cambridge University Press.
4) Zhao, H. (2005). A fast sweeping method for eikonal equations. Mathematics of computation, 74(250), 603-627.

The original blog post has been written by James TenBrink, and is available here, What’s New in the Spatial Analyst Distance Toolset in Pro 2.5James TenBrink is a senior software engineer in the raster analysis group at Esri. He has a Bachelor of Science degree in Computer Science from Hope College in Holland, Michigan, and a Master of Science degree in Computer Science from Rensselaer Polytechnic Institute in Troy, New York. He has been with Esri 28 years and has had an office in almost every building, on almost every floor, of the Esri campus.

more
2 0 1,154

Creating accumulative cost surfaces using directional data

Esri Contributor

The following question was recently posted to GeoNet:

“I want to add wind direction and strength to a cost distance surface. How do I create a raster surface which incorporates these two factors…?“

In this blog post, the spatial analyst team responds to this user’s question.

The Path Distance tool in the Spatial Analyst toolbox can do part of this. It can be used to create an accumulative cost surface from costs that are determined by the direction of the wind. Unfortunately, Esri doesn’t yet have a way to make cost of motion vary based on both the direction and speed of the wind.

Also, when using Path Distance, it is extremely important to know in advance whether you intend to move from source cells to other cells, or vice versa. You will see that changing this intent changes the result dramatically.

If you decide to use Path Distance to incorporate only wind direction into your accumulative cost surface, here is what we recommend:

You can encode wind direction azimuths (clockwise 0-360 from north, both 0 and 360 are treated as “north”) in the input horizontal raster. In figure 1 below, a 10 x 10 horizontal raster is shown. It has a constant cell value of 90 and represents wind blowing to the east with some uniform speed over the study area.

You should not use wind speed as the friction cost surface, because that cost is paid regardless of which direction you travel through a cell. In fact, Path Distance doesn’t need a cost friction surface at all (nor do the other Path Distance tools: allocation, backlink).

Figure 1-A 10 x 10 horizontal direction raster used as input to Path Distance. Each cell has a value of 90 (wind blowing from west to east). Cell centers are shown in yellow, along with a vector field renderer (blue arrows).

Case 1: Motion from source

Let’s first assume that:

• You are sitting on the single source point (the selected blue point in the center of the raster in figure 2).
• That its easiest to move with the wind and impossible to move either against it or at right angles to it.
• That you want to calculate accumulated cost from your source cell to every other cell in the study area.

For this case, you could use a horizontal_factor parameter value of Horizontal Forward Function AND (this is important) a source_direction parameter value of “FROM_SOURCE”. The result is shown in figure 2. You can see that your accumulated cost surface is only defined in areas that can be reached travelling with (or somewhat with) the wind.

Figure 2- The results of using Path Distance with the horizontal raster from figure 1 and the horizontal forward function. The single source is the selected point in the center and we want to know how expensive it is to reach other cells in the study area. The accumulated cost surface shows that we are only allowed to move with the wind, and its easiest to move directly east. For this result, we use the horizontal forward function AND a source_direction parameter value of motion from source.

Lets go into a bit of detail on how Path Distance turns wind direction into a travel cost (also called a weight). We zoom in on the single source cell and its 8 neighbors:

Figure 3 - Detail of source cell (marked with an S) and its 8 neighbors. Each cell has a wind direction.

From that source, we can move in 8 directions to reach a neighbor cell and we need to figure out what the step cost for that move will be.

Figure 4 - You can move 8 ways from the source cell to its neighbors. Each choice makes a different angle relative to each cell's wind direction.

In our case, the only thing that affects the step cost is wind direction. Each of those 8 step directions makes some angle with the preferred moving direction, which in this case is the wind direction.

HRMA calculation

We call this angle the horizontal relative moving angle (HRMA) because it is your direction of motion relative to an azimuth value stored in your horizontal raster. Each horizontal factor function turns this HRMA into a weight. In this case, we’ve used the Horizontal Forward Function. See this help topic for more details on HRMA.

The HRMA and horizontal factor function are evaluated twice for each of the 8 possible steps: once for the wind direction at the source cell and once for the direction at the neighbor cell. The two results are then averaged to get the horizontal weight to be used when calculating the cost of a step into the neighbor cell.

Case 2: Motion To Source

Now let’s assume that the single source point represents a location that you want to reach from other points in the study area. In this case, all inputs to Path Distance remain the same, except you now choose a source _direction parameter value of “TO_SOURCE”. Figure 5 shows the result. As you can see, the results are very different.

Figure 5 -The result of using Path Distance to find accumulated cost of travelling to the single source point from other cells in the study area: source_direction is set to “ motion to source”. All other inputs are the same as for figure 2.

Why are these results so different? In both cases, the accumulative cost surface is grown outward starting from the source and visiting all other cells; but in the TO_SOURCE case, the calculation of the HRMA is different, so the output of the horizontal factor function applied to that HRMA will also be different. In the TO_SOURCE case the HRMA is the geometric supplement of the HRMA calculated in the FROM_SOURCE case, as shown in figure 6.

HRMA value changes based on direction of motion

To summarize:

Path distance can adjust the cost of motion through a cell based on input data that describes, for each cell, the easiest direction to move through the cell. This calculation can produce very different results, based on whether you want to move from a source to other cells, or from other cells back to the source.

We haven’t explicitly talked about the vertical raster and vertical factor functions, but the reasoning process is basically the same as described above. However, instead of accounting for horizontal influences, the vertical factor determines the cost to overcome the changes in elevations (the slopes) between the cells.

The original blog content has been written by James TenBrink and Kevin Johnston and is available here: https://www.esri.com/arcgis-blog/products/arcgis-pro/analytics/creating-accumulative-cost-surfaces-u...

James TenBrink is a senior software engineer in the raster analysis group at esri. He has a Bachelor of Science degree in Computer Science from Hope College in Holland, Michigan, and a Master of Science degree in Computer Science from Rensselaer Polytechnic Institute in Troy, New York. He has been with esri 28 years and has had an office in almost every building, on almost every floor, of the esri campus.

Kevin Johnston has been a Product Engineer on the Spatial Analyst development team for over 29 years. He has degrees in Landscape Architect from Harvard and in environmental modeling from Yale. At Esri, Kevin’s current focus is developing suitability and connectivity tools. He hopes the tools that he works on can help users make more informed decisions.

more
0 0 368

Doing more with Euclidean Distance: Barriers and Paths

Esri Contributor

Currently, Euclidean Distance Mapping geoprocessing tools can be used to assign distance properties to raster cells. Example applications include distance from runways used as part of an airport noise model, or distance from streams used as a criterion layer in a habitat suitability model. From ArcGIS Pro 2.4 and ArcMap 10.7.1, you can now  use the Euclidean distance tool to consider an important use case: generating precise shortest paths around barriers. You can also use barrier-aware distance and allocation maps  to replace their corresponding cost distance versions as input layers in suitability models.

Consider these related use cases posted to geonet:

• “Aim: To determine the shortest path from a nest to marine foraging locations for an individual bird. The bird is restricted to traveling along water only (i.e., freshwater and ocean).” [1]
• “I have been running some least cost path analyses on manatee GPS telemetry data using ArcGIS 9.3.1.  The goal of this analysis is to create the shortest path between points without the path crossing land. [2]

Prior to ArcGIS Pro 2.4/ArcMap 10.7.1, the only way to plot shortest paths around barriers was to use the Cost Distance (CD) tool using a constant cost surface with a small, positive, non-zero cost for the areas where movement was allowed and either a high cost or nodata where movement was prohibited. As the geonet posters point out, the cost paths obtained from this approach are unsatisfactory. To drive this point home, lets recreate the seabird flight paths (using completely fake nesting and feeding site data) using the CD + constant cost surface approach. The study area is Vancouver, British Columbia.

Figure 1: Sea bird over-water flight paths created using the CostDistance + constant cost surface approach. The Cost Distance sources are the feeding sites in the northwest corner and the Cost Path As Polyline destinations are the nesting locations in the southeast corner.

Goodchild [3] discusses why these paths deviate. The underlying cost distance algorithm is based on an 8-way connected graph, which restricts the directions that can be considered, and the absence of meaningful variation in the cost surface (other than barriers) means that the CostPath/CostPath As Polyline operation doesn’t know which of many equal length routes to choose from.

At 2.4/10.7.1, we can do better, as shown below in figure 2 in green, using enhancements to the Euclidean Distance (ED) and Cost Path As Polyline GP tools.

Figure 2: Sea bird over-water flight paths (in green) created using barrier-aware Euclidean Distance and Cost Path As Polyline tools.

Figure 3: Detail showing Euclidean distance (in green) and Cost Distance (in blue) flight paths.

The CD paths in this example are only about 1% longer than the ED paths, but their deviations from the geometric shortest paths are significant.

There is also a geodesic option for ED, but we haven’t used it in this example because we wanted to compare results with CD.

Creating Shortest Paths With Barriers

To generate these improved paths, use the following workflow:

1. Create a barrier dataset. It can be any kind of feature class or a raster. In the example above, the barrier dataset is a polyline feature class representing coastlines. If it’s a raster, DATA cells act as barriers.

2. Use the Euclidean Distance tool instead of the Cost Distance tool. Specify values for the new optional input parameter “Input raster or feature barrier data” and the new optional output parameter “Out back direction raster”, as shown below:

Figure 4: Euclidean distance tool user interface

3. Use the Cost Path As Polyline tool to create the paths from the destinations (Nesting sites) back to the sources (Feeding Sites). This tool has been enhanced to accept the new back direction output from Euclidean distance:

Figure 5: Cost Path As Polyline tool user interface

Simple!

The Back Direction Raster

The new back direction raster, created by the Euclidean toolset, encodes a floating point azimuth value in the range (0.0, 360.0] degrees, with zero itself reserved for source cell locations. This azimuth points in the direction of the shortest path as it passes through the cell. This is different than the “Output direction raster” parameter, which uses the 1-360 azimuth value (rounded to an integer) to point directly at the closest source cell (the cell where the shortest path will eventually end).

You can always ask for the back direction raster to be generated whether or not barriers are specified. This provides a convenient way to use Cost Path As Polyline tool to identify which source cell is closest to a given destination cell (it’s the cell on which the polyline ends) even if you’re not interested in barriers or specific pathways. Locating this cell was difficult in earlier versions of the Euclidean toolset: For planar Euclidean distance, you could have done some math with the distance and direction outputs, but that wouldn’t work for geodesic output.

Barrier-aware Distance and Allocation Outputs

So far, we’ve emphasized the use of the new functionality in generating better shortest paths. For this application, the most important output was the new back direction raster, which gave Cost Path as Polyline the information it needed to plot those paths step-by-step.

The distance and allocation maps may also be directly useful. As an example, compare the euclidean and cost distance versions of the barrier-aware distance map in figures 6 and 7 below. The euclidean version avoids the distortions present in the cost distance version.

Figure 6: Barrier-aware Euclidean distance map used to compute the sea bird over-water flight paths.

Figure 7: Cost distance map produced from encoding coastline as nodata into a constant cost surface

Use of Distance and Allocation Outputs in Suitability Models

Let’s repeat that the difference between CostDistance and Euclidean Distance with barriers is that the former performs its computation using a network distance metric and the latter uses a euclidean distance metric. This is something to be aware of when using the outputs of cost distance as layers in a suitability model. It is probably the case that some other layers in that model assume a Euclidean distance metric, so the mixture of distance metrics in a single suitability model will introduce some amount of error into the model results. For example, if both the Cost allocation and Euclidean allocation datasets shown in the figure below were used in the same model, then the classification of cells near an allocation boundary becomes more uncertain, as shown below.

Figure 8: Comparison between cost allocation and barrier-aware euclidean allocation zones. Note the zig-zag distortion in the cost allocation boundary.

The only reason to mix distance metrics like this is because, up until now, there hasn’t been a good alternative. Also, the distance distortion itself is bounded by a factor of about 1.1 ([3, Table 2], so that may be good enough for many applications. However, now we have a better alternative for the special case of barriers on a constant cost surface. For a distortion free solution for more general cost distance problems, we’ll have to wait for an upcoming ArcGIS platform release.

Summary

This blog has shown you how to generate shortest paths around barriers, using the versions of the Euclidean Distance and Cost Path as Polyline tools available in ArcGIS Pro 2.4 and ArcMap 10.7.1. Also, if you are using cost distance tools with a constant cost raster (containing some nodata cells) to generate inputs for a suitability model, you should consider using the barrier-aware euclidean tools instead.

References

[1] Sea bird geonet posting

[2] ManateeGISDB geonet posting

[3] Description of length and direction distortions in current cost distance implementation

Goodchild, M. F. (1977). An Evaluation of Lattice Solutions to the Problem of Corridor Location. Environment and Planning A. 9. 727-738. 10.1068/a090727.

[4] Readable introduction to barrier-aware euclidean distance method

Sethian, J. A. (1997). Tracking Interfaces with Level Sets: An “act of violence” helps solve evolving interface problems in geometry, fluid mechanics, robotic navigation and materials sciences. American Scientist Vol. 85, No. 3 (MAY-JUNE 1997), pp. 254-263

The original blog is available at https://www.esri.com/arcgis-blog/products/spatial-analyst/analytics/doing-more-with-euclidean-distan...

The content of the blog has been prepared by James TenBrink, a senior software engineer in the raster analysis group at esri. He has a Bachelor of Science degree in Computer Science from Hope College in Holland, Michigan, and a Master of Science degree in Computer Science from Rensselaer Polytechnic Institute in Troy, New York. He has been with Esri 28 years and has had an office in almost every building, on almost every floor, of the Esri campus.

more
2 1 478

Raster Analysis and Spatial Analyst session and events at UC 2019

Esri Contributor

The 2019 Esri User Conference (July 8-12) is almost here. We here on the Raster Analysis team are busy practicing and polishing our presentations and are looking forward to showing you what we’ve been working on since last summer.  If you do any kind of raster or multidimensional analysis, please look at the guide below to help you plan out your time with us.

Expo hall

Visit the UC Expo to meet with us and with several amazing exhibitors. The UC Expo Hall has extended hours on Thursday this year, meaning more time is available for you to come by and get your questions answered.

Tuesday, July 9th         9:00 AM – 6:00 PM

Wednesday, July 10th  9:00 AM – 6:00 PM

Thursday, July 11th      9:00 AM – 4:00 PM

Technical Workshops and Demos

To help target your areas of interest, we’ve categorized workshops and demos relevant to raster analysis with Spatial Analyst into different groups.  In each group, you will find the general purpose of the sessions within the group, followed by the specific session titles in chronological order. Repeat presentations are also listed, in case you are not able to attend the first offering. Finally, remember to check out the Road Ahead sessions for a peek into the future.

1.     Raster Analysis

The ArcGIS platform offers many capabilities for raster analysis, data management, and spatial modeling.  Attend the introductory sessions to get a broad overview of the capabilities for raster analysis.  If your interest is more for specific types of analysis, such as suitability modeling, cost distance modeling, or hydrological modeling, we offer many sessions more specifically focused on those areas.

 Raster Analysis Tuesday, July 9 10:00 AM Spatial Analyst: An Introduction SDCC – Room 30 B 10:00 AM Hydrologic Analysis and Applications in ArcGIS SDCC – Room 33 C 11:15 AM Working With Living Atlas Data: Habitat Modeling SDCC – Expo Demo Theater 11 1:00 PM Interpolating Surfaces in ArcGIS SDCC – Room 33 C 1:15 PM Map Algebra: Getting Started SDCC – Expo Demo Theater 10 1:15 PM Working With Living Atlas Data: Elevation Analysis SDCC – Expo Demo Theater 11 4:00 PM Finding the Best Locations Using Suitability Modeling SDCC – Room 33 C Wednesday, July 10 8:30 AM Cost Distance Analysis: Identifying the Best Path SDCC – Room 30 C 11:15 AM Surface Interpolation in ArcGIS SDCC – Expo Demo Theater 10 1:15 PM Bobcat Suitability Modeling SDCC – Expo Demo Theater 10 2:30 PM Spatial Analyst: An Introduction SDCC – Room 32 A/B Thursday, July 11 2:30 PM Finding the Best Locations Using Suitability Modeling SDCC – Room 15 A 4:00 PM Cost Distance Analysis: Identifying the Best Path SDCC – Room 17 A 4:00PM Interpolating Surfaces in ArcGIS SDCC – Room 15 A 4:00 PM Hydrologic Analysis and Applications in ArcGIS SDCC – Room 14 A
 Raster Data Management Tuesday, July 9 10:00 AM Basics of creating a Mosaic Dataset SDCC Demo Theater 2 Thursday, July 11 11:15 AM Managing and Serving Elevation and Lidar Data SDCC Demo Theater 2

2. Raster Analysis using Python

The integration of Map Algebra with Python via the Spatial Analyst ArcPy module offers a broad range of capabilities for raster analysis, data management and automation workflows. With the arcgis.raster module in the ArcGIS Python API, you can perform raster analysis on very large raster datasets in the server environment. Various Python packages also let you extend your analytical power though custom tools, as well as machine and deep learning.

 Tuesday, July 9 1:15 PM ArcGIS API for Python: Integrating Machine Learning and Deep Learning SDCC – Expo Demo Theater 08 2:30 PM Python: Raster Analysis SDCC – Room 30 A Wednesday, July 10 10:00 AM ArcGIS API for Python: Imagery Analysis SDCC – Room 16 A 4:00 PM Python: Raster Analysis SDCC – Room 30 A

3.     Raster Analytics using ArcGIS Image Server

The Raster Analytics capabilities offered by ArcGIS Image Server can be used to enable cloud computing, which applies distributed computing and distributed storage technology to analyze and process large raster data collections.  The following sessions will introduce the capabilities of this product, describe the architecture, detail the deployment options, and identify the performance benefits offered by distributed processing.

 Tuesday, July 9 8:30 AM ArcGIS Enterprise: Raster Analytics in Image Server SDCC – Room 33 A/B 1:00 PM ArcGIS Enterprise: Managing and Serving Imagery in the Cloud using ArcGIS Image Server SDCC – Room 15 B Wednesday, July 10 10:00 AM ArcGIS Enterprise: Sharing Imagery SDCC Demo Theater 2 12:30 PM Working with OGC WCS Services SDCC Standards & Interoperability Spotlight Theater 4:00 PM ArcGIS Enterprise: Managing and Serving Imagery in the Cloud using ArcGIS Image Server SDCC – Room 05 B Thursday, July 11 8:30 AM ArcGIS Enterprise: Deploying Distributed Raster Analytics SDCC – Room 05 A 2:30 PM ArcGIS Enterprise: Raster Analytics in Image Server SDCC – Room 08

4.     Scientific Multidimensional Data Analysis

The ArcGIS platform has great capabilities for the ingesting, management, visualization, analysis, and sharing of scientific multidimensional data.  If you would like to use netCDF, HDF or GRIB data for doing spatial and temporal analysis, the following sessions will be of interest to you.

 Wednesday, July 10 2:30 PM Analyzing Multidimensional Scientific Data in ArcGIS SDCC – Room 33 C Thursday, July 11 10:00 AM Analyzing Multidimensional Scientific Data in ArcGIS SDCC – Room 10 10:00 AM Scientific and Multidimensional Raster Support in ArcGIS SDCC – Room 05 A 10:30 AM Working with Scientific Data in ArcGIS Platform SDCC – Expo – Open Platform: Standards & Interoperability Spotlight Theater

Come join us for the Road Ahead sessions to find out what enhancements have been made in spatial analytics and what we are planning for the upcoming year, especially in the areas of machine learning, big data and real time analysis.

Special Interest Group Meetings

Special Interest Groups – or SIGs—are communities within multiple organizations with a shared interest in advancing a specific area of knowledge, learning, or technology. Every year, the Esri UC supports dozens of SIG meetups, both technical and non-technical. SIGs are a great way to network with your peers and learn something new. We encourage you to check out and participate in any of several SIG meetups highlighted below.

 Sunday, July 07 7:30 am – 5:00 pm Imagery Summit Marriott – Marina Foyer 8:30 am – 5:00 pm GeoSpatial Deep Learning SDCC – Room 32 A/B 9:00 am – 4:30 pm Esri GIS Hydro Meeting Marriott – Santa Rosa Tuesday, July 9 11:30 am – 12:30 pm Ocean SIG SDCC – Room 26 A Thursday, July 11 11:30 am – 12:30 pm Weather and Climate SIG SDCC – Room 17 A

Summary, and come see us!

Additional details on these sessions listed above, and much more, is available in the full agenda. Be sure to download the Esri Events Mobile App, too!   (UC content coming soon)

Please come by the Spatial Analyst island within Spatial Analysis & Data Science at the Product Showcase in the Exhibit Hall during the week (Tuesday and Wednesday from 9:00 – 6:00, and Thursday from 9:00 – 4:00.  We’d love to hear about the work you do, any difficulties you have encountered, and any ideas you might have to make our software suit your needs even better.

***Note: All listings subject to change; not every paper or event may be presented here. For the list of the all sessions visit the full agenda page.

more
0 0 288

Introducing the new resolution preserving cell size projection method

Esri Contributor

ArcGIS Pro 2.3 and ArcMap 10.7 Spatial Analyst tools now support a new environment, the Cell Size Projection Method, to control the calculation of the output raster cell size when datasets are projected during tool execution. To learn more about projections, see Coordinate systems, projections, and transformations.

In previous software versions, the cell size was converted using a linear unit conversion when projecting from a Projected Coordinate System (PCS) to another PCS or an angular unit conversion when going from Geographic Coordinate System (GCS) to another GCS. So, projecting a 30 meter raster dataset from UTM zone 10N to WGS84 Web Mercator would continue to use 30 meter as the output cell size. When projecting from GCS to PCS or vice versa, previous versions took an average cell size based on the ratios of diagonal lengths in source and destination projections. Therefore, this method, currently exposed as the ‘convert units’ method simply copied the cell size from input to output, changing units if necessary.

This approach has some limitations:

• It does not account for distortion when projecting from one PCS to another PCS (1 projected meter in, say, a UTM zone at a given location may not cover the same ground distance as 1 projected meter at the same location in an Albers projection).
• When computing the ratios of diagonal lengths, the behavior of the projection at only the four corner points was used. This may have introduced excessive distortion, depending on the projection and the extent.

The two new methods: ‘preserve resolution’ and ‘center of extent’ both are intended to calculate a projected cell size that doesn’t change the ground distance of the cell, thus avoiding unnecessary resampling during an implicit projection operation happening as a Spatial Analyst tool or a feature to raster conversion tool executes. You should use the ‘center of extent’ method when you know the geographic location where your cell size is most accurate. If you use this method, you can use a raster with a small extent in that same area as the cell size source or create a temp raster ‘around’ that location and use that as the cell size source. If you are unsure, use the ‘preserve resolution’ method. To learn more about the three cell size projection methods see How the Cell Size Projection Method environment setting works.

If you do not know at which specific geographic location your cell size is best but want to improve on the ‘convert units’ method when a projection is involved while working with Spatial Analyst tools, you can use the ‘preserve resolution’ method.

Why use the resolution preserving cell size projection method?

In the ‘preserve resolution’ method the same number of square cells as are in the original extent are preserved in the projected extent. The output cell size is calculated based on the ratios of the areas of the projected extent to the original extent. This approach is based on how Esri software currently chooses a cell size when moving from a camera image coordinate system to a geodetic (GCS or PCS) system. This method calculates the average size of a square cell more accurately for all combinations of GCS and PCS than the default ‘convert units’ method.

If the areas of the original rectangular extent and (shape preserving) projected extent are A0and A1, then the areas of the square cells, respectively, are, ca0 = A0/n and ca1 = A1/n

Since the number of cells remains constant for both cases, the ratio of the area of extent to the area of the square cell are equal, A0/ca0 = A1/ca1

Figure 1: Cell size projection using the new ‘preserve resolution’ method

So, the output cell area is ca1 = (A1/A0) * ca0

and the output cell size is CellSize_projected = √((A1/A0) * ca0)

In this method, the cell size conversion factor is, √(A1/A0)

Let’s look at two examples to see how the ‘preserve resolution’ method picks a better output cell size than ‘convert units’.

Example 1:

Let’s take an elevation raster (R_input), located in Vermont (Figure 2), in a PCS (NAD_1983_StatePlane_Vermont_FIPS_4400) and project it to another PCS (WGS 1984 World Mercator) using the default cell size projection methods, ‘convert units’ (R_out_CU) and ‘preserve resolution’ (R_out_PR). Here both the spatial references are shape preserving, where the input spatial reference is suitable at a state level and the output spatial reference is suitable for the entire world. We will then compare the geodesic distance between the cell centers to determine which method preserves the geodesic distance more accurately.

Figure 2: Location of the input raster (R_Input), its cell boundaries and cell centers.

Before we compare the geodesic distances between the two methods, lets understand what are the different ways a raster is projected in ArcGIS Pro. When you add a raster to the map in ArcGIS Pro, the spatial reference of the map becomes the same as the spatial reference of the raster. For example, if the first layer added has a NAD_1983_StatePlane_Vermont_FIPS_4400 PCS, the map will have the same spatial reference, and all other layers will project on the fly to match this spatial reference. This on-the-fly raster projection is meant for a richer display experience but does not preserve the raster structure (different cells can have on-the-fly-projected different sizes, rotations, and distortions). However, when you project a raster using the Project Raster tool or specifying the Output Coordinate System environment of a geoprocessing tool, the raster is actually  projected into a new raster structure (every cell is an identical rectangle in the output spatial reference, with sides parallel to the coordinate system axes). When measuring geodesic distance for the comparison of the cell size projection methods, it is recommended to use the actual projected raster instead of the raster that is projected on the fly.

In figure 3, the cell size of the input raster, R_input is 30 meters. This is approximately the same as its geodesic ground distance, which you can find out using the Measure tool in ArcGIS Pro.

Figure 3: Cell boundaries of R_input, cell centers of R_input, R_out_CU, R_out_PR and the geodesic distance between the cell centers. To overlay the points, the cell centers have been projected back to the spatial reference of R_input for comparison.

When the raster is projected using the ‘convert units’ methods, the projected cell size of the output raster, R_out_CU remains 30 meters, but 30m in WGS 1984 World Mercator is a much smaller distance on the ground, 21 meters. So, raster projection using the ‘convert units’ method has unnecessarily increased the resolution of the output raster. If we were projecting in the opposite direction (starting from WGS 1984 World Mercator), then we would have lost a significant amount of raster data. When the same raster is projected using the ‘preserve resolution’ method, its cell size becomes 42 meters and the geodesic ground distance remains 30 meters, which is the same as the geodesic distance of the input raster.

Example 2:

In this example, we will project from a UTM zone to its adjacent zone, which can happen when mosaicking together many different DEMs for a larger (state sized) area. In figure 4, let’s take a raster (R_input2) in WGS 1984 UTM Zone 11N and project it to its adjacent zone WGS 1984 UTM Zone 12N, creating output (R_out_CU2) and (R_out_PR2) for ‘convert units’ and ‘preserve resolution’ respectively. From here we will again compare the geodesic distance between the cell centers to determine which method preserve the geodesic distance more accurately.

Figure 4: Cell boundaries of R_input2, cell centers of R_input2, R_out_CU2, R_out_PR2 and the geodesic distance between the cell centers. To overlay the points, the cell centers have been projected back to the spatial reference of R_input2 for comparison.

The cell size for the input raster, R_input2 is 30 meters, which is approximately the same as the geodesic ground distance in this area. When the raster is projected using the ‘convert units’ method, the cell size of the output raster, R_out_CU2 becomes 30 meters and the geodesic ground distance becomes approximately 31 meters. When the same raster is projected using the ‘preserve resolution’ method, its cell size becomes 28.99 meters, and the geodesic ground distance is 30 meters, which is the same as the input raster.

In both the examples, the ‘preserve resolution’ method seems to preserve the geodesic ground distance between cell centers better than the ‘convert units’ method. Other combinations of input and output spatial references may, of course, show difference between the methods, but overall it is safe to say that the ‘preserve resolution’ Method is a better approach for preserving geodesic distance while projecting datasets.

An important property of the ‘preserve resolution’ method is that the output cell size depends on the location of the dataset. The same combination of input and output spatial reference will yield a different output cell size if the input raster dataset is at a different geographic location. If you need to use one cell size for different raster datasets (for example, processing adjacent DEM tiles), specify one raster dataset to use as the cell size source while performing raster analysis for the tiles.

Next time you use a Spatial Analyst geoprocessing tool or python command to:

• create an output with a spatial reference different from that of the input dataset,
• use input datasets with different spatial references,
• specify an analysis cell size using a dataset with a different spatial reference,

pay attention to the output cell size and consider using either the preserve resolution or center of extent methods. By default, the projection method will do the ‘convert units’, which existed in previous versions of ArcGIS, but as we have seen, this method may unnecessarily increase or decrease the resolution of your valuable raster data.

Handling projections during analysis in Spatial Analyst

How the analysis window is determined in Spatial Analyst

This blog post has been prepared by James TenBrink, senior software engineer in the raster analysis group at Esri, and Sarmistha Chatterjee.

Original post: https://www.esri.com/arcgis-blog/products/spatial-analyst/analytics/introducing-the-new-resolution-p...

more
3 5 679

What’s New for Spatial Analyst and Raster Analysis in ArcGIS Pro 2.2 and Desktop 10.6.1.

Esri Contributor

As you will have heard, ArcGIS Pro 2.2, ArcGIS Desktop 10.6.1, and ArcGIS Enterprise 10.6.1are now available for download!  If you use the Spatial Analyst extension, or do Raster Analysis in general, read on for more information about the new capabilities on offer.

Where do I get it, and what’s changed?

You can download and install the software right now from the following links: ArcGIS Pro 2.2, ArcGIS Desktop 10.6.1, ArcGIS Enterprise.  For a quick summary of all the changes that have been made for these releases, have a look at their respective What’s New topics:

- What’s New in ArcGIS Pro

- What’s New in ArcMap 10.6.1

- What’s New in ArcGIS Enterprise Portal

Here are the main categories of the changes.

1. New and updated tools for the Spatial Analyst extension in ArcGIS Pro and ArcMap.

2. Enhanced performance potential for Spatial Analyst tools with Parallel Processing.

3. New Raster Functions are available.

4. New capabilities are available for Raster Analysis through a Portal in ArcGIS Pro.

5. New capabilities are available for Raster Analysis through a Map Viewer with ArcGIS Enterprise.

6. If you perform Raster Analysis through the REST API, a new task was added.

Following that, some background information on how to configure and perform raster analysis with ArcGIS Enterprise is listed.

1. Spatial Analyst extension

For the Spatial Analyst extension product, there some changes to environment behavior, some tools have received new parameters, and more tools have been enabled for parallel processing.

Interactive Python Window

Spatial Analyst tools can use a new ArcPy environment,   arcpy.env.buildStatsAndRATForTempRaster, that can be set to true in the interactive Python window in the application to improve the display of the raster output. This property is accessible and can be set in a standalone python scripts, but will have no effect as the output is not displayed.

It is a Boolean environment that specifies whether or not to calculate statistics and build a raster attribute table for a temporary raster created from an interactive Python session while adding it to the map. It is True by default for all Spatial Analyst tools. When set to False, the statistics will be approximate for the purpose of symbolizing output raster layers, and no raster attribute table will be built.

Distance toolset

The Euclidean distance tools have a new parameter option, distance_method.  With the Geodesic option, the distances are calculated on the ellipsoid, which means that the results of the operation do not change regardless of the input or output projection.  The default Planar option will return the same result as in previous releases. The tools are: Euclidean Allocation (Pro | ArcMap), Euclidean Direction (Pro | ArcMap), and Euclidean Distance (Pro | ArcMap).

An improvement was made to the default behavior for a number of the Cost and Path distance tools.  The processing extent and spatial reference now default to that of the input cost raster.  This will improve the accuracy of the results, and better meets the expected behavior. What’s the reason behind this change?  Previously, the processing extent defaulted to the input source dataset, and since that is often smaller than that of the cost surface extent, the resulting cost paths were limited.  Now, the tools will return the true least-cost path between locations within the full extent of the cost surface.  The full list of the seven affected tools is: Cost Allocation (Pro | ArcMap), Cost Back Link (Pro | ArcMap), Cost Connectivity (Pro | ArcMap), Cost Distance (Pro | ArcMap), Path Distance (Pro | ArcMap), Path Distance Allocation (Pro | ArcMap), Path Distance Back Link (Pro | ArcMap).

Extraction toolset

For the Extract By Mask  (Pro | ArcMap) tool, the input raster will now be used as the default snap raster.  This improves the accuracy of the output by ensuring the correct alignment of the feature mask with the value raster. If both the input raster and the input mask are raster, by default the tool will use the input with the largest cells size as the cell size for the operation.  If the input mask is a feature, by default the input raster will define the cell size and snap raster environment for the operation.

Hydrology toolset

The Flow Accumulation (Pro | ArcMap) and Flow Distance (Pro | ArcMap) tools have a new optional parameter that allows you to specify which option was used to create the particular flow direction raster you are using as input for the tool.  The flow_direction_type raster parameter has the choices of D8 (the default method, which assigns the flow direction to the steepest downslope neighbor), MFD (this method assigns multiple flow directions towards all downslope neighbor), and DINF (this uses the D-Infinity flow method to assign flow direction based on the steepest slope of a triangular facet).

Segmentation and Classification toolset

The Export Training Data For Deep Learning (Pro | ArcMap) tool has a new optional parameter, start_index. This is useful when appending image chips to an existing sequence for training the deep learning classifier.

Surface toolset

The Contour (Pro | ArcMap) tool has two new parameters.  With contour_type, you can now choose to create contour output as polygons instead of just polylines.  There are several types of filled contours to choose from.  When contouring larger or complicated surfaces with polygons, you can use the max_vertices_per_feature parameter to control the maximum number of vertices per polygon feature.

Zonal toolset

For zonal tools that perform a zonal operation on an input value raster, that raster will be used as the default snap raster for the operation.  The benefit here is improved accuracy for the output due to the correct alignment of the feature zone with the value raster. The tools with this new default behavior are Zonal Histogram (Pro | ArcMap), Zonal Statistics (Pro | ArcMap), and Zonal Statistics as Table (Pro | ArcMap).

The Tabulate Area (Pro | ArcMap) tool has a new default behavior as well.  Now, when both the zone input and the class input are raster, the class raster will be used as the snap raster for the operation.  If one of these inputs is a raster and the other a feature, the one that is raster will be used for setting both the processing cell size and the snap raster.

2. Parallel Processing

As promised, the parallel processing capability that was added to a number of tools at ArcGIS 10.6 but not Pro 2.1 is now available for those tools in Pro 2.2!  We also enabled this capability for a few more tools for ArcGIS 10.6.1.

ArcGIS Pro 2.2

Distance toolset: Cost Allocation (Pro), Cost Back Link (Pro), Cost Distance (Pro), Euclidean Allocation (Pro), Euclidean Direction (Pro), Euclidean Distance (Pro), Path Distance (Pro), Path Distance Allocation (Pro), and Path Distance Back Link(Pro).

Generalization toolsetNibble (Pro)

Hydrology toolsetFill (Pro), Flow Accumulation (Pro), Flow Direction (Pro), Flow Distance (Pro), Sink (Pro), Stream Link (Pro), and Watershed (Pro).

ArcMap 10.6.1

Distance toolsetCost Back Link (ArcMap), Euclidean Direction (ArcMap), Path Distance (ArcMap), Path Distance Allocation (ArcMap), and Path Distance Back Link (ArcMap).

3. Raster Functions

Have you  been using Raster Functions to perform select raster operations on pixels displayed in your ArcGIS Pro map window?  There is a large number of operations you can use on your raster or imagery data out-of-the box, with more available when you have a Spatial Analyst or an Image Analyst extension license available.

Three new distance raster functions are available in Pro 2.2, Cost Back Link, Cost Path, and Euclidean Direction.

In addition, the Flow Accumulation and Flow Distance functions have the new flow direction type parameter.

Following is the current list of Raster functions you can use that require either a Spatial Analyst (SA) or an Image Analyst (IA) license in ArcGIS Pro. Which specific license(s) are required are identified.

Analysis Functions: Kernel Density (SA), Weighted Overlay (SA), Weighted Sum (SA, IA)

Classification Functions: Classify (SA, IA), ML Classify (SA, IA), Segment Mean Shift (SA, IA)

Data Management Functions: Nibble (SA)

Distance Functions : Cost Allocation, Cost Back Link, Cost Distance, Cost Path, Euclidean Allocation, Euclidean Direction, Euclidean Distance, Least Cost Path (All require SA)

Hydrology Functions: Fill, Flow Accumulation, Flow Direction, Flow Distance, Stream Link, Watershed. (All require SA)

Math : Abs, Divide, Exp, Exp10,  Exp2, Float, Int, Ln, Log10, Log2, Minus, Mod, Plus, Power, Round Down, Round Up, Square, Square Root, Times (All require either SA or IA)

Math : Conditional : Con (SA, IA), Set Null (SA, IA)

Math : Logical : Bitwise And, Bitwise Left Shift, Bitwise Not, Bitwise Or, Bitwise Right Shift, Bitwise XOr, Boolean And, Boolean Not, Boolean Or, Boolean XOr, Equal To, Greater Than, Greater Than Equal, Is Null, Less Than, Less Than Equal, Not Equal (All require either SA or IA)

Math : Trigonometric: ACos, ACosH, ASin, ASinH, ATan, ATan2, ATanH, Cos, CosH, Sin, SinH, Tan, TanH (All require SA or IA)

Statistical Functions: Cell Statistics (SA or IA), Zonal Statistics (SA or IA).

Surface Functions: Viewshed (SA).  Note, there are other surface functions, such as Aspect, Hillshade or Slope, that you can use without an Spatial Analyst or Image Analyst license.

4. Raster Analysis Tools in ArcGIS Pro

In ArcGIS Pro 2.2, one new Raster Analysis tool has been added, and two other tools have new parameter options.

Hydrology toolset

The Flow Accumulation and Flow Distance tools have a new parameter that allows you to specify the type of the input flow direction raster: D8, Multi Flow Direction (MFD), or D-Infinity (DINF).

Use Proximity toolset

The new Determine Travel Cost Path As Polyline tool is used to calculate the least cost path between sources and destinations.

Follow this link to see the full suite of functionality available in the Raster Analysis toolbox in ArcGIS Pro.

5. Raster Analysis in the Map Viewer

Some new tools are available for performing raster analysis in the Map Viewer when you are connected to an updated version of the Portal for ArcGIS in ArcGIS Enterprise.

There is a new toolset for proximity (distance) analysis, with the following three tools:

• Calculate Distance
• Determine Optimum Travel Cost Network
• Determine Travel Cost Path As Polyline

In the Analyze Terrain toolset, there is a new Watershed tool.

Four new Map Viewer tools for Raster Analysis are available in this release

6. Raster Analysis in the REST API

The Determine Travel Cost Path as Polyline task was added to the ArcGIS REST API for 10.6.1.  This task can be used to calculate the least cost way to travel from the source location to a destination.

See Get started with the Raster Analysis service to learn more about using the REST API to execute other Raster Analysis service tasks, and how to configure your portal to enable the functionality.

Some background on raster analytics in ArcGIS Enterprise

Raster Analytics using ArcGIS Enterprise is a flexible raster processing, storage, and sharing system that employs distributed computing and storage technology based on ArcGIS Image Server.  The source data and processed results are stored, published, and shared across your enterprise as image and feature web layers,  according to your needs and priorities.

This capability can be further extended by leveraging cloud computing capabilities and resources. The net result is that raster processing and analysis on very large datasets, which heretofore were difficult to perform in a timely manner on single machines, can now be executed in short order by harnessing the compute power of multiple servers to simultaneously attack the tasks you give them.

In order to have access to Raster Analytics, you need to have an ArcGIS Enterprise deployment available.  After ArcGIS Server is installed, ArcGIS Image Server will also need to be installed and enabled.   Please see the following resources for more details on configuring and using these services for performing raster analysis.

Server

Portal

Summary

Above we’ve covered the important changes and improvements for raster analysis that have been introduced to the platform in ArcGIS Pro 2.2, ArcGIS Desktop 10.6.1, and ArcGIS Enterprise 10.6.1.  After you have downloaded the new versions, please give the new capabilities a try.  If you encounter any difficulties along the way, be sure to communicate them back to us through the Support channels or GeoNet, so we can work on resolving them.

more
1 2 878

Getting the most out of Zonal Statistics

Esri Contributor

When working with Zonal tools in the Spatial Analyst toolbox, have you occasionally gotten results that that you didn’t quite expect? Here we’ll cover a few scenarios where and why you might have run into some issues, how to work around them, and how things have changed in the latest release to avoid them in the first place. The Zonal Statistics tool calculates statistics on values of a raster within the zones defined by another dataset. To learn more, read.

The zone input can either be a feature or a raster. If the zones are defined by features, an internal feature to raster conversion will occur. The internal conversion for a polygon zone uses the cell center method to rasterize the input. With an analysis extent calculation method of intersection-of or union-of, the origin of the internal raster may be determined entirely by the feature class. Snapping is then performed relative to that origin: the internal raster origins at the lower-left corner of the analysis extent and its upper-right corner is adjusted by the cell size. The cells of the internal zone raster and the value raster may not align, which will trigger a resampling during the zonal operation. Resampling will also occur if the input zone is a raster with different cell size and/or alignment. Often times, we end up getting an unexpected result due to the feature to raster internal conversion or misalignment of the zone and value raster. Let’s look at some of these scenarios in detail:

1. Unexpected statistics values

The common source of unexpected statistics values is the misalignment of the cells of the zone and value rasters.

In this example in figure 1a below, you may expect the Zonal Statistics tool to compute the statistics based on the cells of the value raster whose cell center falls within the feature zone, which are values 79, 81 and 27. However, due to the way the default internal conversion will be performed based on the origin and extent, how the feature zone gets rasterized will create an unexpected output. As you can see in figure 1b the rasterization grid used for the internal conversion does not align with the value raster. Therefore, using the cell center method, the zone raster will end up analyzing a different set of cells from the value raster (see figure 1c) than you originally anticipated from figure 1a.

Figure 1: Internal conversion of feature zone without considering the value raster for cell alignment.

To avoid this issue, there is a step you can take to ensure that the feature zone being converted to a raster is aligned with the value raster. Simply set the snap raster to the value raster from the tool environment. The misalignment can also occur when the input zone is a raster, if the cell size and/or cell alignment does not align with the value raster. You can set the snap raster environment to the value raster to get the expected output.

Let’s look in the example below, in figure 2b, to understand how setting the snap raster to the value raster ensures the rasterization grid used for the internal conversion aligns with the value raster in figure 2b. As a result, the zone raster will analyze those cells from the value raster (see figure 2c) that you originally anticipated from figure 2a.

Figure 2: Internal conversion of feature zone considering the value raster for cell alignment.

The good news is that ArcGIS Pro 2.2 and ArcMap 10.6.1 use the value raster as the snap raster by default for internal conversion of the feature zones. If you are using a version prior to ArcGIS Pro 2.2 or ArcMap 10.6.1, specify the value raster as the snap raster in the environment.

2. Missing zones in the output

The most frequent cause of missing zones in the output occurs when the cell center of the rasterization grid does not fall within the feature zone. This can occur for zones that are smaller than the area of a cell of the internal zone raster or even for larger zones.

In the example below, let’s look at how rasterization occurs for zones of different size and location. Figure 3a has three zones, where, zone1 is larger than a cell, and zone2 and zone3 are smaller than a cell, and the cell center falls outside zone2 and within zone3. During the zone rasterization process in figure 3b, it so happens that no cell centers fall within zone1 and zone2, and only zone3 contains a cell center. Therefore, only zone3 will be rasterized and the other two zones will essentially disappear, as shown in figure 3c.

Figure 3: Internal conversion of feature zone leading to missing zones.

To avoid this, ensure that each of your zones contains one or more cell centers. You can create more cell centers by specifying a smaller cell size in the environment. The default analysis cell size comes from the value raster. Therefore, specifying a cell size that is smaller than that of the value raster will enable more zones to be captured.

For figure 4, let’s repeat the same example from figure 3 to see how changing cell size better captures all the zones. Similar to figure 3a, figure 4a also has three zones, where, zone1 is larger than a cell, and zone2 and zone3 are smaller than a cell but are spatially located differently. Figure 4b shows a finer rasterization grid based on a cell size that is four times smaller than the default cell size. During the zone rasterization process, multiple cell centers now fall within each of the zones. As a result, all the zones get rasterized as shown in figure 4c.

Keep in mind that specifying a smaller cell size, will generate a larger output raster.  In this example, the output raster will be sixteen times larger than the raster with default cell size. Even more important, the higher resolution output may give the wrong perception of a higher quality result than what it actually is, since the additional detail does not actually exist in the input value raster.

Figure 4: Internal conversion of feature zone with a smaller cell size capturing all zones.

Note: If you are using a version prior to ArcGIS Pro 2.2 or ArcMap 10.6.1, specify the value raster as the snap raster in the environment.

3. Other sources of unexpected result

You may also get unexpected results during the rasterization of feature zones if you have:

• coincident points or multiple points on one cell
• coincident polylines or multiple polylines passing through one cell
• coincident polygons or partially overlapping polygons

Be sure to read the Zonal Statistics and How Zonal Statistics works help pages to learn more about how this tool works.

Summary

With the information presented here, you should be able to better understand how this tool operates, and how to guarantee better and more understandable results.  Keep in mind that there is a similar behavior in play for other Zonal tools, such as Tabulate Area, Zonal Histogram and Zonal Statistics as Table so you can employ the lessons learned here to achieve greater success with those tools, too. Keep in mind, that ArcGIS Pro 2.2 and ArcMap 10.6.1 now use the value raster as the snap raster by default for internal conversion of the feature zones. If you are using a version prior to ArcGIS Pro 2.2 or ArcMap 10.6.1, specify the value raster as the snap raster in the environment.

Let us know if you encounter other scenarios where you get an unexpected result while doing your analysis, or if you have any questions or comments. You can comment here or reach me at schatterjee@esri.com.

more
3 3 866
67 Subscribers