1 2 Previous Next

24 posts

# Spatial Analyst UC 2020 Workshops and Demos

Posted by SChatterjee-esristaff Jul 6, 2020

The 2020 Esri User Conference (July 13-16) is almost here, and we are going virtual! So, if you haven’t already, register to join the world’s largest, virtual GIS conference and be able to access the plenary, technical sessions, and many other presentations. We here on the Spatial Analyst team are excited to show you what we’ve been working on since last summer.  If you do any kind of raster or imagery analysis, or work with multidimensional data, look at the guide below to plan out your time with us.

Streaming Live Plenary Session

Join us for three inspiring days of thought-provoking Plenary Session content. Don’t miss out on the brand-new Interactive Suitability Modeler, which will be covered in the Monday, July 13, 10:00 am – 12:00 pm PDT timeslot.

Streaming Live Technical Sessions

Take advantage of this unique learning experiences and check out the following technical workshops with live Q&A:

You can find out more about these and all other sessions from the full schedule. You can also filter the schedule based on your interest or level to find out the sessions you would like to attend. If you are interested in sessions related to Spatial Analysis and Data Science, don’t forget to check out this blog.

On-Demand Sessions

The following on-demand sessions can be accessed starting on July 13th for all registered attendees. You can view the recordings at your convenience.

As you are attending the sessions, make sure to give us your feedback, so that we can improve them for the next round. We would also like to hear from you while you are attending the conference, so visit us virtually at the Esri UC 2020 expo in the Spatial Analysis and Data Science showcase to learn more about the ongoing projects, or if you have questions. You can also follow our Spatial Analyst Resources blog page to get updates throughout the year.

Enjoy the conference!

# Unleash the power of RasterCellIterator to perform custom raster analysis

Posted by h.hu-esristaff Feb 13, 2020

If you’ve read through the Introducing the Raster Cell Iterator blog, you’ll be familiar with the basic concepts of the Raster Cell Iterator (RCI), a new ArcPy class added in ArcGIS Pro 2.5 in the Spatial Analyst module that allows you to iterate over raster cells, query and modify individual cell values.

In this blog, let’s explore how RCI can be used to customize your analysis on problems that cannot be easily solved using the existing out-of-the-box geoprocessing tool. In the first example, we will use RCI to create a new raster with a specific spatial pattern – checkerboard. And in two other examples, we will use RCI to customize two focal operations: 1) count the number of neighboring cells that have a different value from the center cell, and 2) calculate the minimum slope from a DEM.

## Create a new raster with the checkerboard pattern

In atmospheric modeling, a heterogenous surface raster represented by a checkerboard pattern can be used as an initial condition for idealized model simulations. However, there is not an existing geoprocessing tool with which you can create such a raster directly. In earlier ArcGIS releases, you can accomplish this task by using NumPy, but it would require a multi-step workflow to do so.  In ArcGIS Pro 2.5 with RCI, this task has become much easier! You simply need to assign a binary value to each cell based on the current row and column while iterating through a raster.  Following is a simple code sample which does exactly that.

import json    from arcpy.sa import RasterInfo, Raster    # Create an empty RasterInfo object  myRasterInfo = RasterInfo()    # Load raster info from a Python dictionary  myRasterInfoData = {    'bandCount': 1,    'extent': {      'xmin': -107.0,      'ymin': 38.0,      'xmax': -104.0,      'ymax': 40.0,      'spatialReference': {'wkid': 4326},    },    'pixelSizeX': 0.01,    'pixelSizeY': 0.01,    'pixelType': 'U8',  }         # Convert myRasterInfoData to a JSON string and load into myRasterInfo  myRasterInfo.fromJSONString(json.dumps(myRasterInfoData))    # Create a new Raster object based on myRasterInfo  myRaster = Raster(myRasterInfo)    for (r, c) in myRaster:       # Checkerboard with 10 pixels width      if r % 20 < 10 and c % 20 < 10 or r % 20 >= 10 and c % 20 >= 10:          myRaster[r, c] = 1      else:          myRaster[r, c] = 0myRaster.save(r 'C:\output\checkerboard.tif')  

From the code snippet above, lines 8-23 show the steps to construct a RasterInfo object from scratch. Line 26 creates an empty raster based on the raster info specified in the previous step. Lines 28-33 show the main logic of creating a checkerboard pattern raster by assigning binary values to the cell based on the current row and column. An implicit way of using Raster Cell Iterator is demonstrated by calling a simple For loop to go through all the cells. Since there is only one raster involved in this analysis, you don’t need to create a RasterCellIterators object explicitly. It is recommended to follow this pattern when you are working with only one raster. Finally, at line 35, we are saving the output raster. Figure 1 shows the output raster with the checkerboard pattern.

Figure 1. Output raster with the checkerboard pattern

## Custom Focal Operations

Focal operations produce an output raster dataset in which the output value at each cell is a function of the input cell value at that location and its neighboring cell values. With RCI, you can customize these kinds of focal operations with more flexibility.

### Count the number of neighboring cells that have a different value from the center cell

Stathakis & Tsilimigkas (2015) proposed several metrics to calculate the compactness of cities using land use adjacency information. To derive the compactness metric, one of the pre-processing steps is to iterate through the land use data and count the number of neighboring cells that have a value different to the center cell. This focal operation process is illustrated in Figure 2.

Figure 2. Examples of input and output with the operation counting the number of neighboring cells that have a value different to the center cell.

RCI makes it much easier to iterate over all cells and compare each cell value with its neighbors using a simple if logic. Here’s how you can do it yourself:

from math import isnan    from arcpy.sa import Raster, RasterCellIterator        landu = Raster(r"C:\sample_data\landuse.tif")    # Create a temporary output raster based on the raster info of landuse    output = Raster(landu.getRasterInfo())       with RasterCellIterator({'rasters':[landu, output]}) as rci:        for i,j in rci:            count = 0             # Assign NoData value to the output if the input is NoData            if isnan(landu[i,j]):                output[i,j] = math.nan                continue            # Create a moving window            for x in [-1,0,1]:                for y in [-1,0,1]:                    # Count the number of adjacent cells with a different value                      if not isnan(landu[i+x,j+y]) and landu[i+x,j+y] != landu[i,j]:                        count+=1            output[i,j] = count    output.save(r"C:\output\landuse_diff_count.tif")   

In line 6, we are creating a new empty raster using the land use raster as an input template. Line 8 creates a RasterCellIterator object explicitly using input and output. In lines 9-21, we are looping through all cells in the input raster, and for each cell comparing its value to each of its immediate neighbors based on its relative position.

While performing the analysis, it is required to check for the NoData cells in the input, so that an appropriate value can be assigned to the output raster.  In lines 12 and 19, we are using isnan() function from the math module to check if the cell value in the input raster is NoData.  Line 13 uses nan from the same math module to represent NoData. In this example, NoData values are not considered while iterating through the cells and they are excluded from the neighbor cells when counting. The input land use data and the final output raster used in the sample code are shown in Figure 3.

Figure 3. The figure on the left shows the input land use raster dataset. The figure on the right shows the output raster with values ranging from 0 to 8, which counts the number of neighboring cells that have a different value from the center cell.

### Calculate minimum slope from a DEM

For the Slope geoprocessing tool, the slope value of each cell is determined by the rates of change of the elevation in the horizontal and vertical directions from the center cell (How slope works). In some cases, you might be interested in calculating the slope using a different equation. For an example, when building a fish trawlability model, an analyst might want to use a slope of the seabed that is based on the minimum change of elevation across each cell. By creating a custom focal operation with RCI, the following example shows how to calculate the minimum slope for your bathymetric DEM:

from arcpy.sa import Raster, RasterCellIteratordem = Raster(r'C:\sample_data\demtif.tif')cell_x = dem.meanCellHeightcell_y = dem.meanCellWidthraster_info = dem.getRasterInfo()# Set the output pixel type to be float 32 raster_info.setPixelType('F32')min_slope = Raster(raster_info)with RasterCellIterator({'rasters':[dem, min_slope]}) as rci:    for r,c in rci:        slopes = []        # Iterate through 8 neighbors         for x,y in [(-1,-1),(-1,0),(1,0),(0,-1),(0,1),(1,1),(-1,1),(1,-1)]:            # Calculate the slope from center cell to each neighbor            slope = abs(dem[r,c]-dem[r+x,c+y])/math.sqrt(cell_x**2+cell_y**2)            # Add all the slope values to a list            slopes.append(slope)        # Assign the minimum slope to the output cell value         min_slope[r,c] = min(slopes)min_slope.save(r'C:\output\min_slope.tif')

In the code sample, lines 13-21 implement the main logic for calculating the minimum slope. For each cell from DEM, we iterate through its eight neighbors and calculate the rate of change. Then we find the minimum slope and assign that value to the output cell. Note that in line 9, the pixel type is set as ‘F32’, which means that the output is expected to be a 32-bit floating raster. In this example, the raster info of the slope output is inherited from the input DEM data. If the pixel type of your DEM data is integer, you need this step to make sure that the slope value of each cell is calculated as a floating-point value instead of an integer.

## Summary

In this blog, we went through three different examples of using Raster Cell Iterator in ArcPy, each of them showing how easy it is to perform custom raster analysis. We hope you start using this simple but powerful capability to extend your own raster analysis capabilities.  Please let us know if you have any questions or want to see other examples and use cases.  You can also pass along any cool applications you come up with, and we can share your solutions with the community at large!

## Reference:

Stathakis, D., & Tsilimigkas, G. (2015). Measuring the compactness of European medium-sized cities by spatial metrics based on fused data sets. International Journal of Image and Data Fusion6(1), 42-64.

The original blog post Unleash the power of RasterCellIterator to perform custom raster analysis has been written by Hao Hu and Yu Wang.

# What’s New for Spatial Analyst and Raster Analysis in ArcGIS Pro 2.5

Posted by jlaguna-esristaff Feb 11, 2020

Do you use Spatial Analyst extension for performing raster analysis?  With the release of ArcGIS Pro 2.5, we have made significant changes in the areas of distance analysis, multidimensional raster analysis, and zonal analysis. There is also whole host of improvements across many other areas.  If you are comfortable in ArcPy, there is an exciting new way to perform custom raster analysis.  Also in ArcPy, many new functions are available for analyzing, visualizing, and managing raster data using Python.

We will have many more blog posts coming that go into much more detail on some of these changes, and they will be linked to below.

## Where do I get it?

ArcGIS Pro 2.5 was released of in February 6 of this year.

For a quick summary of all the changes that have been made for these releases, have a look at the What's New topic (and video!) here:

ArcMap Desktop 10.8 and ArcGIS Enterprise 10.8 will be released very soon.  Watch this space, or your usual channels, for more details once they become available.

## What's changed for Spatial Analyst?

Here are the main categories of the changes.

1. Distortion free distance analysis
2. Multidimensional raster analysis
3. Multiband raster analysis
4. Zonal analysis
5. General tool and function enhancements
6. Iterating over raster cells using ArcPy
7. Modifying raster cell values using ArcPy
8. Enhanced multidimensional raster processing with ArcPy
9. Additional raster functions available in ArcPy

### 1.  Distortion Free Distance Analysis

With ArcGIS Pro 2.5, Spatial Analyst sees a significant change in distance analysis capabilities.  First and foremost, a new algorithm for cost-based distance analysis provides more accurate and precise results than before.  By avoiding the distortion in output that was caused by the previously-used network model of cell connectivity, several benefits are achieved:

• Cost accumulation is measured the same way in all directions. An important special case is that cost distance with a constant cost surface now produces the same output as Euclidean distance mapping.
• Surface distance over a digital elevation model is computed much more accurately.
• Paths around barriers are followed more accurately.
• Cost distance accumulation and allocation analysis have the option to perform the analysis using either a planar or a geodesic method.

As part of making this improved capability available, the distance toolset has been reorganized into fewer but more capable tools.  The Corridor tool is carried over as-is. The new tools are:

Rest assured, all the tools that existed in the previous release are still available; they have simply been moved into a new sub-toolset.  We have a help topic that well covers the migration from legacy to distortion free distance tools.

To complement the new GP tools, two new raster functions are available in the Distance group:

• The Least Cost Path function has been updated to use the Distance Accumulation tool to perform distortion-free distance analysis.
• The Cost Path function has been updated with a new parameter that allows you to determine a hydrological flow path.

Related blog:

In ArcMap 10.8, the Distance toolset remains basically the same as before.  The only exception is that the Cost Path and Cost Path As Polyline tools have been updated with a new parameter which provides the option to make those tools treat the input backlink raster as a flow direction raster instead.

### 2.  Multidimensional Raster Analysis

As part of the increased focus on multidimensional analysis in the ArcGIS platform, a number of tools in various toolsets have been updated to directly support multidimensional raster data. There is also a new pathway for accessing multidimensional analytical tools directly from the ArcGIS Pro interface.

In the Multidimensional Analysis toolset, the following tools have improvements:

In the Extraction toolset, the Sample tool has several new parameters to process multidimensional data.  When processing this type of data, the tool has options to specify the following:

• The output table layout, either as rows or columns
• The time, depth, or other acquisition information associated with the location features to extract information for a given time or calculate statistics within a period of the observation time
• Lines and polygons as the input location, in addition to points and rasters
• The Parallel Processing Factor environment

In the Zonal toolset, the Zonal Statistics tool, the Zonal Statistics as Table tool, and the Zonal Statistics raster function now have the Process as multidimensional parameter available, which allows you to calculate various statistics on each slice of a multidimensional value raster.

In ArcGIS Pro, for the Raster Layer tab, the new Multidimensional tab is enabled when a multidimensional raster layer or a multidimensional mosaic layer is selected in the Contents pane. It provides tools and functionality for working with multidimensional raster data.

### 3.  Multiband Raster Analysis

Not to be confused with multidimensional rasters, multiband rasters are raster datasets that are composed of two or more raster bands, or layers, where each band is a measure of a single characteristic. The following tools have changes pertaining to how multiband raster data is handled.

• The Aggregate Multidimensional Raster tool can now do band-wise aggregation for multiband multidimensional input.
• The Pick and Cell Statistics tools have a new parameter that allows you more control over how multiband input rasters are processed.
• The Cell Statistics raster function can calculate single-band or multiband output based on the multiband processing type.

### 4.  Zonal Analysis

In response to an often-requested capability, the Zonal Statistics as Table tool now processes overlapping polygon zones. The statistics for each zone are calculates separately.

### 5.  General tool and function enhancements

In addition to the changes described above, some new other new tools and tool enhancements are available.

• In the Raster segmentation and classification toolset, a new tool, Linear Spectral Unmixing, has been added to perform subpixel classification and calculate the fractional abundance of different land cover types for individual cells. The Export Training Data For Deep Learning tool has four new parameters.
• For generalizing rasters, the Aggregate tool now supports the Parallel Processing Factor environment. The Median option for the Aggregation technique now only produces a float output raster, regardless of the input type. The new Aggregate raster function dynamically generalizes a raster in the display at a reduced resolution. When the output from the function is persisted, it will be generated at the full resolution.
• A new Random raster function is available for creating a dynamic raster on the fly with random cell values.

### 6.  Iterating over raster cells using ArcPy

With ArcGIS Pro 2.5, a powerful new capability to perform custom raster analysis is available to you in ArcPy.  Two classes are introduced that allow you iterate over rasters programmatically.

The RasterCellIterator class in the Spatial Analyst module allows you to visit each cell location in a raster. The iterator makes it easy to inspect cell values at each location, as well as neighboring locations. While iterating over a raster, you can read and write cell values.

The RasterInfo class in the ArcPy classes describes a set of raster properties that facilitate the creation of a new raster dataset using the Raster object. This class has several methods and properties available.

Related blogs:

### 7. Modifying raster cell values using ArcPy

The Raster object now allows you to read cell values using a [row, column] index notation (also known as neighborhood notation).  In addition to reading values, you can also assign a new value to a particular raster cell, thereby modifying the Raster object. Once the values of raster cells have been modified through an assignment, you can persist these changes by calling the save() method on the Raster object.

### 8. Enhanced multidimensional raster processing using ArcPy

The Raster object has more capabilities for processing multidimensional rasters.  The additional properties and methods are:

• Properties—bandNames, blockSize, noDataValues, properties, and readOnly
• Methods—addDimension, exportImage, getProperty, getRasterBands, getRasterInfo, read, removeVariables, renameVariable, setProperty, and write

### 9. Additional raster functions available in ArcPy

In the Spatial Analyst module, many new ArcPy functions are available for analyzing, visualizing, and managing raster data using Python.  To keep things organized, they are grouped into functional categories, which you can see from Overview page.

### New resources

Do you have our Spatial Analyst resources blog post bookmarked?

Not only does it include a list of links to informative content, we also work to keep it updated as new resources become available.  For example, here is some of the material that was recently added:

### Summary

That about covers what's changed since the last release.  Once you've downloaded and installed the latest versions of the software, give it a try.  We hope you will enjoy the improvements and enhancements.

# Introducing the Raster Cell Iterator

Posted by NRajasekar-esristaff Feb 6, 2020

Have you ever wished you could easily interact with raster datasets in ArcGIS Pro at an individual cell level? Be able to read and write cell values directly, and create your own custom analysis routines? Look no further, as the Raster Cell Iterator (RCI) in ArcGIS Pro 2.5, lets you do just that. RCI is available through the Spatial Analyst module, an extension of the ArcPy Python site package. With RCI, visit each cell location in a Raster object, all in a Python environment. The iterator makes it easy to query and modify cell values at each cell location and its neighboring cell locations. Iterable access to raster cells enables you to write custom raster analysis scripts and combine them with the existing suite of Spatial Analysis geoprocessing tools, expanding your analytical capabilities immensely!

Here is a short video that helps you get started with RCI and shows how to use it for custom raster analysis.

As you can see in this video, using RCI is very simple and requires few lines of coding. You can invoke RCI on a single raster object or multiple raster objects to iterate through their row and column indices. Using these indices, you can query a cell value at a given cell location. You can also query neighborhood cell values by using relative indexing. Additionally, you can use index notation to write cell values to an output raster.

Read A quick tour of using Raster Cell Iterator to get started with RCI in ArcGIS Pro.

Read the blog Unleash the power of RasterCellIterator to perform custom raster analysis, which illustrates how to apply RCI to solve a read-world problem.

# What's New in the Spatial Analyst Distance Toolset in Pro 2.5

Posted by SChatterjee-esristaff Dec 12, 2019

The Spatial Analyst team has made some important improvements in the Distance toolset for ArcGIS Pro 2.5.
We have significantly improved the precision of the geoprocessing tools in the Distance toolset and the corresponding raster functions. We have also provided additional capabilities which let you answer some new “how much/how many/how far” questions.

These improvements allows us to remove the strong distinction between the “Euclidean” and “Cost” Distance tools. The number of tools has been reduced from 17 to 6. The new user experience is now simplified to:

1. Enter Sources

2. Do you have barriers?

3. Do you have a surface?

4. Do you have a cost surface?

5. Do you need to consider direction of motion and other source characteristics?

The new toolset organization is shown in Figure 1 below. The new tools are:

Figure 1 - Organization of the Distance toolset at ArcGIS Pro 2.5

The original Distance tools that you are familiar with are available in the  Legacy sub-toolset.
The Spatial Analyst team recommends that you move your Euclidean and Cost Distance analysis work to the new tools.

## Why should I use the new tools?

At ArcGIS Pro 2.5, we’ve uniformly adopted a new algorithm [4] for all distance mapping tools. This algorithm removes the distortion in outputs caused by using a network model of cell connectivity, as shown in Figure 2 on the left below. This has the following benefits:

• Cost distance is now measured the same way in all directions. An important special case is cost distance with a constant cost surface now produces the same output as the Euclidean distance tools.
• Surface distance over a digital elevation model can now be precisely computed.
• Paths around barriers are now followed precisely, as shown in Figure 3.
 Figure 2- Output of legacy Cost Distance tool using a constant cost input with barriers encoded as ndoata, along with output of legacy Cost Path As Polyline tool. Figure 3 – Output of new Distance Accumulation tool with barrier inputs, along with output of new Optimal Path As Line tool.

At ArcGIS Pro 2.4, we introduced this improvement in a limited context: Euclidean distance with barriers. ArcGIS Pro 2.5 generalizes and extends this work.

## What else is new?

Along with this fundamental change to the behavior of the cost distance algorithm, several additional capabilities have been added.

Figure 4- Geodesic flight paths from some Northern European airports to LAX, but avoiding paths over southern Greenland.

• As a side effect of the improved algorithm, you can now create buffer regions whose distance is measured over the surface of a digital elevation model. This isn’t really a separate capability, but it’s important enough to mention explicitly. In the example below, the study area is Lake Wakatipu near Queenstown, New Zealand – a high relief area. Notice how, in such areas, the output of the Distance Accumulation tool (gray gradient) with a maximum accumulation cutoff of 1500m “pulls away” from the output of the vector Buffer tool that used a buffer distance of 1500m. This surface-aware distance computation can be done in either planar or geodesic mode.

Figure 5 – Comparison of planar (blue) and surface distance (gray gradient) buffers around Lake Wakatipu in the Queenstown area of New Zealand. Both buffers extend to a distance of 1500m, but the gray gradient output measures distance over the surface of a DEM instead of distance in the plane of the data’s projected coordinate system.

• Using the new Optimal Path as Raster tool, you can count how many least cost paths pass through a cell en route from some destinations to a source.

Figure 6 – Optimal Path As Raster output from ridgeline hiking trail in SW to access road in NE. Green is less intense usage, red is more intense.

• Using the new source_location output from Distance Accumulation and Distance Allocation, you can do intensity mapping of the boundary of sources, showing where most cost paths in the study area would end up, without plotting all cost paths. The RasterCellIterator python object, also new at ArcGIS Pro 2.5, makes this type of intensity calculation very easy to perform. See A quick tour of using Raster Cell Iterator to know more about it.
• Figure 7 – Lake Wakatipu, New Zealand. Hot spots (red and blue fuzzy dots), computed from Distance Accumulation output, show locations of most intense drainage into lake from surrounding watershed.

## Which geoprocessing tools and raster functions should I start using?

For each legacy distance tool that you are currently using, table 1 shows you which new tool to switch to.

Table 1 – Upgrade guide for distance mapping tools
If you usedThen switch to
Cost DistanceDistance Accumulation
Cost BacklinkDistance Accumulation with back direction output
Cost AllocationDistance Allocation
Euclidean DistanceDistance Accumulation
Euclidean AllocationDistance Allocation
Euclidean DirectionDistance Accumulation with source direction output
Path DistanceDistance Accumulation
Path BacklinkDistance Accumulation with back direction output
Path AllocationDistance Allocation
Cost PathOptimal Path As Raster
Cost Path As PolylineOptimal Path As Line
Cost ConnectivityOptimal Region Connections

For new workflows, the back direction raster replaces the back link raster. The Optimal Path as Raster/Line tools no longer accept a backlink raster as input (they will still accept a flow direction raster). If you were using the backlink raster in a workflow that involves something other than the Cost Path or Cost Path As Polyline tools, then the Spatial Analyst team would like to hear from you. The Euclidean distance blog mentioned above discusses the backdirection raster in more detail.If you used the maximum_distance parameter in the legacy cost tools, you should now use the Maximum Accumulation parameter in the “Characteristics of the sources” parameter group.

These are the new raster functions.

The functions have the same parameters as the tools, with an important difference: the functions optionally emit multi-band outputs. This avoids having to run these (possibly expensive) global functions twice to get closely related outputs (like accumulated cost surface and back direction raster).

Also, the Least Cost Path raster function has been updated to use the Distance Accumulation tool behind the scenes.

Finally, although not directly related to these enhancements, your advanced use of the Distance toolset needs to consider if direction of motion between sources and destinations is important. If so, please check out this blog on Creating accumulative cost surfaces using directional data.

### References

1) Goodchild, M. F. (1977). An Evaluation of Lattice Solutions to the Problem of Corridor Location. Environment and Planning A. 9. 727-738. 10.1068/a090727.
2) Sethian, J. A. (1997). Tracking Interfaces with Level Sets: An “act of violence” helps solve evolving interface problems in geometry, fluid mechanics, robotic navigation and materials sciences. American Scientist Vol. 85, No. 3 (MAY-JUNE 1997), pp. 254-263
3) Sethian, J. A. (1999). Level set methods and fast marching methods: evolving interfaces in computational geometry, fluid mechanics, computer vision, and materials science (2nd edition). Cambridge University Press.
4) Zhao, H. (2005). A fast sweeping method for eikonal equations. Mathematics of computation, 74(250), 603-627.

The original blog post has been written by James TenBrink, and is available here, What’s New in the Spatial Analyst Distance Toolset in Pro 2.5James TenBrink is a senior software engineer in the raster analysis group at Esri. He has a Bachelor of Science degree in Computer Science from Hope College in Holland, Michigan, and a Master of Science degree in Computer Science from Rensselaer Polytechnic Institute in Troy, New York. He has been with Esri 28 years and has had an office in almost every building, on almost every floor, of the Esri campus.

# Creating accumulative cost surfaces using directional data

Posted by SChatterjee-esristaff Aug 26, 2019

The following question was recently posted to GeoNet:

“I want to add wind direction and strength to a cost distance surface. How do I create a raster surface which incorporates these two factors…?“

In this blog post, the spatial analyst team responds to this user’s question.

The Path Distance tool in the Spatial Analyst toolbox can do part of this. It can be used to create an accumulative cost surface from costs that are determined by the direction of the wind. Unfortunately, Esri doesn’t yet have a way to make cost of motion vary based on both the direction and speed of the wind.

Also, when using Path Distance, it is extremely important to know in advance whether you intend to move from source cells to other cells, or vice versa. You will see that changing this intent changes the result dramatically.

If you decide to use Path Distance to incorporate only wind direction into your accumulative cost surface, here is what we recommend:

You can encode wind direction azimuths (clockwise 0-360 from north, both 0 and 360 are treated as “north”) in the input horizontal raster. In figure 1 below, a 10 x 10 horizontal raster is shown. It has a constant cell value of 90 and represents wind blowing to the east with some uniform speed over the study area.

You should not use wind speed as the friction cost surface, because that cost is paid regardless of which direction you travel through a cell. In fact, Path Distance doesn’t need a cost friction surface at all (nor do the other Path Distance tools: allocation, backlink).

Figure 1-A 10 x 10 horizontal direction raster used as input to Path Distance. Each cell has a value of 90 (wind blowing from west to east). Cell centers are shown in yellow, along with a vector field renderer (blue arrows).

## Case 1: Motion from source

Let’s first assume that:

• You are sitting on the single source point (the selected blue point in the center of the raster in figure 2).
• That its easiest to move with the wind and impossible to move either against it or at right angles to it.
• That you want to calculate accumulated cost from your source cell to every other cell in the study area.

For this case, you could use a horizontal_factor parameter value of Horizontal Forward Function AND (this is important) a source_direction parameter value of “FROM_SOURCE”. The result is shown in figure 2. You can see that your accumulated cost surface is only defined in areas that can be reached travelling with (or somewhat with) the wind.

Figure 2- The results of using Path Distance with the horizontal raster from figure 1 and the horizontal forward function. The single source is the selected point in the center and we want to know how expensive it is to reach other cells in the study area. The accumulated cost surface shows that we are only allowed to move with the wind, and its easiest to move directly east. For this result, we use the horizontal forward function AND a source_direction parameter value of motion from source.

Lets go into a bit of detail on how Path Distance turns wind direction into a travel cost (also called a weight). We zoom in on the single source cell and its 8 neighbors:

Figure 3 - Detail of source cell (marked with an S) and its 8 neighbors. Each cell has a wind direction.

From that source, we can move in 8 directions to reach a neighbor cell and we need to figure out what the step cost for that move will be.

Figure 4 - You can move 8 ways from the source cell to its neighbors. Each choice makes a different angle relative to each cell's wind direction.

In our case, the only thing that affects the step cost is wind direction. Each of those 8 step directions makes some angle with the preferred moving direction, which in this case is the wind direction.

HRMA calculation

We call this angle the horizontal relative moving angle (HRMA) because it is your direction of motion relative to an azimuth value stored in your horizontal raster. Each horizontal factor function turns this HRMA into a weight. In this case, we’ve used the Horizontal Forward Function. See this help topic for more details on HRMA.

The HRMA and horizontal factor function are evaluated twice for each of the 8 possible steps: once for the wind direction at the source cell and once for the direction at the neighbor cell. The two results are then averaged to get the horizontal weight to be used when calculating the cost of a step into the neighbor cell.

## Case 2: Motion To Source

Now let’s assume that the single source point represents a location that you want to reach from other points in the study area. In this case, all inputs to Path Distance remain the same, except you now choose a source _direction parameter value of “TO_SOURCE”. Figure 5 shows the result. As you can see, the results are very different.

Figure 5 -The result of using Path Distance to find accumulated cost of travelling to the single source point from other cells in the study area: source_direction is set to “ motion to source”. All other inputs are the same as for figure 2.

Why are these results so different? In both cases, the accumulative cost surface is grown outward starting from the source and visiting all other cells; but in the TO_SOURCE case, the calculation of the HRMA is different, so the output of the horizontal factor function applied to that HRMA will also be different. In the TO_SOURCE case the HRMA is the geometric supplement of the HRMA calculated in the FROM_SOURCE case, as shown in figure 6.

HRMA value changes based on direction of motion

## To summarize:

Path distance can adjust the cost of motion through a cell based on input data that describes, for each cell, the easiest direction to move through the cell. This calculation can produce very different results, based on whether you want to move from a source to other cells, or from other cells back to the source.

We haven’t explicitly talked about the vertical raster and vertical factor functions, but the reasoning process is basically the same as described above. However, instead of accounting for horizontal influences, the vertical factor determines the cost to overcome the changes in elevations (the slopes) between the cells.

The original blog content has been written by James TenBrink and Kevin Johnston and is available here: https://www.esri.com/arcgis-blog/products/arcgis-pro/analytics/creating-accumulative-cost-surfaces-using-directional-data/

James TenBrink is a senior software engineer in the raster analysis group at esri. He has a Bachelor of Science degree in Computer Science from Hope College in Holland, Michigan, and a Master of Science degree in Computer Science from Rensselaer Polytechnic Institute in Troy, New York. He has been with esri 28 years and has had an office in almost every building, on almost every floor, of the esri campus.

Kevin Johnston has been a Product Engineer on the Spatial Analyst development team for over 29 years. He has degrees in Landscape Architect from Harvard and in environmental modeling from Yale. At Esri, Kevin’s current focus is developing suitability and connectivity tools. He hopes the tools that he works on can help users make more informed decisions.

# Doing more with Euclidean Distance: Barriers and Paths

Posted by SChatterjee-esristaff Jul 3, 2019

Currently, Euclidean Distance Mapping geoprocessing tools can be used to assign distance properties to raster cells. Example applications include distance from runways used as part of an airport noise model, or distance from streams used as a criterion layer in a habitat suitability model. From ArcGIS Pro 2.4 and ArcMap 10.7.1, you can now  use the Euclidean distance tool to consider an important use case: generating precise shortest paths around barriers. You can also use barrier-aware distance and allocation maps  to replace their corresponding cost distance versions as input layers in suitability models.

Consider these related use cases posted to geonet:

• “Aim: To determine the shortest path from a nest to marine foraging locations for an individual bird. The bird is restricted to traveling along water only (i.e., freshwater and ocean).” [1]
• “I have been running some least cost path analyses on manatee GPS telemetry data using ArcGIS 9.3.1.  The goal of this analysis is to create the shortest path between points without the path crossing land. [2]

Prior to ArcGIS Pro 2.4/ArcMap 10.7.1, the only way to plot shortest paths around barriers was to use the Cost Distance (CD) tool using a constant cost surface with a small, positive, non-zero cost for the areas where movement was allowed and either a high cost or nodata where movement was prohibited. As the geonet posters point out, the cost paths obtained from this approach are unsatisfactory. To drive this point home, lets recreate the seabird flight paths (using completely fake nesting and feeding site data) using the CD + constant cost surface approach. The study area is Vancouver, British Columbia.

Figure 1: Sea bird over-water flight paths created using the CostDistance + constant cost surface approach. The Cost Distance sources are the feeding sites in the northwest corner and the Cost Path As Polyline destinations are the nesting locations in the southeast corner.

Goodchild [3] discusses why these paths deviate. The underlying cost distance algorithm is based on an 8-way connected graph, which restricts the directions that can be considered, and the absence of meaningful variation in the cost surface (other than barriers) means that the CostPath/CostPath As Polyline operation doesn’t know which of many equal length routes to choose from.

At 2.4/10.7.1, we can do better, as shown below in figure 2 in green, using enhancements to the Euclidean Distance (ED) and Cost Path As Polyline GP tools.

Figure 2: Sea bird over-water flight paths (in green) created using barrier-aware Euclidean Distance and Cost Path As Polyline tools.

Figure 3: Detail showing Euclidean distance (in green) and Cost Distance (in blue) flight paths.

The CD paths in this example are only about 1% longer than the ED paths, but their deviations from the geometric shortest paths are significant.

There is also a geodesic option for ED, but we haven’t used it in this example because we wanted to compare results with CD.

## Creating Shortest Paths With Barriers

To generate these improved paths, use the following workflow:

1. Create a barrier dataset. It can be any kind of feature class or a raster. In the example above, the barrier dataset is a polyline feature class representing coastlines. If it’s a raster, DATA cells act as barriers.

2. Use the Euclidean Distance tool instead of the Cost Distance tool. Specify values for the new optional input parameter “Input raster or feature barrier data” and the new optional output parameter “Out back direction raster”, as shown below:

Figure 4: Euclidean distance tool user interface

3. Use the Cost Path As Polyline tool to create the paths from the destinations (Nesting sites) back to the sources (Feeding Sites). This tool has been enhanced to accept the new back direction output from Euclidean distance:

Figure 5: Cost Path As Polyline tool user interface

Simple!

## The Back Direction Raster

The new back direction raster, created by the Euclidean toolset, encodes a floating point azimuth value in the range (0.0, 360.0] degrees, with zero itself reserved for source cell locations. This azimuth points in the direction of the shortest path as it passes through the cell. This is different than the “Output direction raster” parameter, which uses the 1-360 azimuth value (rounded to an integer) to point directly at the closest source cell (the cell where the shortest path will eventually end).

You can always ask for the back direction raster to be generated whether or not barriers are specified. This provides a convenient way to use Cost Path As Polyline tool to identify which source cell is closest to a given destination cell (it’s the cell on which the polyline ends) even if you’re not interested in barriers or specific pathways. Locating this cell was difficult in earlier versions of the Euclidean toolset: For planar Euclidean distance, you could have done some math with the distance and direction outputs, but that wouldn’t work for geodesic output.

## Barrier-aware Distance and Allocation Outputs

So far, we’ve emphasized the use of the new functionality in generating better shortest paths. For this application, the most important output was the new back direction raster, which gave Cost Path as Polyline the information it needed to plot those paths step-by-step.

The distance and allocation maps may also be directly useful. As an example, compare the euclidean and cost distance versions of the barrier-aware distance map in figures 6 and 7 below. The euclidean version avoids the distortions present in the cost distance version.

Figure 6: Barrier-aware Euclidean distance map used to compute the sea bird over-water flight paths.

Figure 7: Cost distance map produced from encoding coastline as nodata into a constant cost surface

## Use of Distance and Allocation Outputs in Suitability Models

Let’s repeat that the difference between CostDistance and Euclidean Distance with barriers is that the former performs its computation using a network distance metric and the latter uses a euclidean distance metric. This is something to be aware of when using the outputs of cost distance as layers in a suitability model. It is probably the case that some other layers in that model assume a Euclidean distance metric, so the mixture of distance metrics in a single suitability model will introduce some amount of error into the model results. For example, if both the Cost allocation and Euclidean allocation datasets shown in the figure below were used in the same model, then the classification of cells near an allocation boundary becomes more uncertain, as shown below.

Figure 8: Comparison between cost allocation and barrier-aware euclidean allocation zones. Note the zig-zag distortion in the cost allocation boundary.

The only reason to mix distance metrics like this is because, up until now, there hasn’t been a good alternative. Also, the distance distortion itself is bounded by a factor of about 1.1 ([3, Table 2], so that may be good enough for many applications. However, now we have a better alternative for the special case of barriers on a constant cost surface. For a distortion free solution for more general cost distance problems, we’ll have to wait for an upcoming ArcGIS platform release.

## Summary

This blog has shown you how to generate shortest paths around barriers, using the versions of the Euclidean Distance and Cost Path as Polyline tools available in ArcGIS Pro 2.4 and ArcMap 10.7.1. Also, if you are using cost distance tools with a constant cost raster (containing some nodata cells) to generate inputs for a suitability model, you should consider using the barrier-aware euclidean tools instead.

## References

[1] Sea bird geonet posting

[2] ManateeGISDB geonet posting

[3] Description of length and direction distortions in current cost distance implementation

Goodchild, M. F. (1977). An Evaluation of Lattice Solutions to the Problem of Corridor Location. Environment and Planning A. 9. 727-738. 10.1068/a090727.

[4] Readable introduction to barrier-aware euclidean distance method

Sethian, J. A. (1997). Tracking Interfaces with Level Sets: An “act of violence” helps solve evolving interface problems in geometry, fluid mechanics, robotic navigation and materials sciences. American Scientist Vol. 85, No. 3 (MAY-JUNE 1997), pp. 254-263

The original blog is available at https://www.esri.com/arcgis-blog/products/spatial-analyst/analytics/doing-more-with-euclidean-distance-barriers-and-paths/

The content of the blog has been prepared by James TenBrink, a senior software engineer in the raster analysis group at esri. He has a Bachelor of Science degree in Computer Science from Hope College in Holland, Michigan, and a Master of Science degree in Computer Science from Rensselaer Polytechnic Institute in Troy, New York. He has been with Esri 28 years and has had an office in almost every building, on almost every floor, of the Esri campus.

# Raster Analysis and Spatial Analyst session and events at UC 2019

Posted by SChatterjee-esristaff Jul 1, 2019

The 2019 Esri User Conference (July 8-12) is almost here. We here on the Raster Analysis team are busy practicing and polishing our presentations and are looking forward to showing you what we’ve been working on since last summer.  If you do any kind of raster or multidimensional analysis, please look at the guide below to help you plan out your time with us.

Expo hall

Visit the UC Expo to meet with us and with several amazing exhibitors. The UC Expo Hall has extended hours on Thursday this year, meaning more time is available for you to come by and get your questions answered.

Tuesday, July 9th         9:00 AM – 6:00 PM

Wednesday, July 10th  9:00 AM – 6:00 PM

Thursday, July 11th      9:00 AM – 4:00 PM

Technical Workshops and Demos

To help target your areas of interest, we’ve categorized workshops and demos relevant to raster analysis with Spatial Analyst into different groups.  In each group, you will find the general purpose of the sessions within the group, followed by the specific session titles in chronological order. Repeat presentations are also listed, in case you are not able to attend the first offering. Finally, remember to check out the Road Ahead sessions for a peek into the future.

### 1.     Raster Analysis

The ArcGIS platform offers many capabilities for raster analysis, data management, and spatial modeling.  Attend the introductory sessions to get a broad overview of the capabilities for raster analysis.  If your interest is more for specific types of analysis, such as suitability modeling, cost distance modeling, or hydrological modeling, we offer many sessions more specifically focused on those areas.

 Raster Analysis Tuesday, July 9 10:00 AM Spatial Analyst: An Introduction SDCC – Room 30 B 10:00 AM Hydrologic Analysis and Applications in ArcGIS SDCC – Room 33 C 11:15 AM Working With Living Atlas Data: Habitat Modeling SDCC – Expo Demo Theater 11 1:00 PM Interpolating Surfaces in ArcGIS SDCC – Room 33 C 1:15 PM Map Algebra: Getting Started SDCC – Expo Demo Theater 10 1:15 PM Working With Living Atlas Data: Elevation Analysis SDCC – Expo Demo Theater 11 4:00 PM Finding the Best Locations Using Suitability Modeling SDCC – Room 33 C Wednesday, July 10 8:30 AM Cost Distance Analysis: Identifying the Best Path SDCC – Room 30 C 11:15 AM Surface Interpolation in ArcGIS SDCC – Expo Demo Theater 10 1:15 PM Bobcat Suitability Modeling SDCC – Expo Demo Theater 10 2:30 PM Spatial Analyst: An Introduction SDCC – Room 32 A/B Thursday, July 11 2:30 PM Finding the Best Locations Using Suitability Modeling SDCC – Room 15 A 4:00 PM Cost Distance Analysis: Identifying the Best Path SDCC – Room 17 A 4:00PM Interpolating Surfaces in ArcGIS SDCC – Room 15 A 4:00 PM Hydrologic Analysis and Applications in ArcGIS SDCC – Room 14 A
 Raster Data Management Tuesday, July 9 10:00 AM Basics of creating a Mosaic Dataset SDCC Demo Theater 2 Thursday, July 11 11:15 AM Managing and Serving Elevation and Lidar Data SDCC Demo Theater 2

### 2. Raster Analysis using Python

The integration of Map Algebra with Python via the Spatial Analyst ArcPy module offers a broad range of capabilities for raster analysis, data management and automation workflows. With the arcgis.raster module in the ArcGIS Python API, you can perform raster analysis on very large raster datasets in the server environment. Various Python packages also let you extend your analytical power though custom tools, as well as machine and deep learning.

 Tuesday, July 9 1:15 PM ArcGIS API for Python: Integrating Machine Learning and Deep Learning SDCC – Expo Demo Theater 08 2:30 PM Python: Raster Analysis SDCC – Room 30 A Wednesday, July 10 10:00 AM ArcGIS API for Python: Imagery Analysis SDCC – Room 16 A 4:00 PM Python: Raster Analysis SDCC – Room 30 A

### 3.     Raster Analytics using ArcGIS Image Server

The Raster Analytics capabilities offered by ArcGIS Image Server can be used to enable cloud computing, which applies distributed computing and distributed storage technology to analyze and process large raster data collections.  The following sessions will introduce the capabilities of this product, describe the architecture, detail the deployment options, and identify the performance benefits offered by distributed processing.

 Tuesday, July 9 8:30 AM ArcGIS Enterprise: Raster Analytics in Image Server SDCC – Room 33 A/B 1:00 PM ArcGIS Enterprise: Managing and Serving Imagery in the Cloud using ArcGIS Image Server SDCC – Room 15 B Wednesday, July 10 10:00 AM ArcGIS Enterprise: Sharing Imagery SDCC Demo Theater 2 12:30 PM Working with OGC WCS Services SDCC Standards & Interoperability Spotlight Theater 4:00 PM ArcGIS Enterprise: Managing and Serving Imagery in the Cloud using ArcGIS Image Server SDCC – Room 05 B Thursday, July 11 8:30 AM ArcGIS Enterprise: Deploying Distributed Raster Analytics SDCC – Room 05 A 2:30 PM ArcGIS Enterprise: Raster Analytics in Image Server SDCC – Room 08

### 4.     Scientific Multidimensional Data Analysis

The ArcGIS platform has great capabilities for the ingesting, management, visualization, analysis, and sharing of scientific multidimensional data.  If you would like to use netCDF, HDF or GRIB data for doing spatial and temporal analysis, the following sessions will be of interest to you.

 Wednesday, July 10 2:30 PM Analyzing Multidimensional Scientific Data in ArcGIS SDCC – Room 33 C Thursday, July 11 10:00 AM Analyzing Multidimensional Scientific Data in ArcGIS SDCC – Room 10 10:00 AM Scientific and Multidimensional Raster Support in ArcGIS SDCC – Room 05 A 10:30 AM Working with Scientific Data in ArcGIS Platform SDCC – Expo – Open Platform: Standards & Interoperability Spotlight Theater

Come join us for the Road Ahead sessions to find out what enhancements have been made in spatial analytics and what we are planning for the upcoming year, especially in the areas of machine learning, big data and real time analysis.

Special Interest Group Meetings

Special Interest Groups – or SIGs—are communities within multiple organizations with a shared interest in advancing a specific area of knowledge, learning, or technology. Every year, the Esri UC supports dozens of SIG meetups, both technical and non-technical. SIGs are a great way to network with your peers and learn something new. We encourage you to check out and participate in any of several SIG meetups highlighted below.

 Sunday, July 07 7:30 am – 5:00 pm Imagery Summit Marriott – Marina Foyer 8:30 am – 5:00 pm GeoSpatial Deep Learning SDCC – Room 32 A/B 9:00 am – 4:30 pm Esri GIS Hydro Meeting Marriott – Santa Rosa Tuesday, July 9 11:30 am – 12:30 pm Ocean SIG SDCC – Room 26 A Thursday, July 11 11:30 am – 12:30 pm Weather and Climate SIG SDCC – Room 17 A

Summary, and come see us!

Additional details on these sessions listed above, and much more, is available in the full agenda. Be sure to download the Esri Events Mobile App, too!   (UC content coming soon)

Please come by the Spatial Analyst island within Spatial Analysis & Data Science at the Product Showcase in the Exhibit Hall during the week (Tuesday and Wednesday from 9:00 – 6:00, and Thursday from 9:00 – 4:00.  We’d love to hear about the work you do, any difficulties you have encountered, and any ideas you might have to make our software suit your needs even better.

***Note: All listings subject to change; not every paper or event may be presented here. For the list of the all sessions visit the full agenda page.

# Introducing the new resolution preserving cell size projection method

Posted by SChatterjee-esristaff May 6, 2019

ArcGIS Pro 2.3 and ArcMap 10.7 Spatial Analyst tools now support a new environment, the Cell Size Projection Method, to control the calculation of the output raster cell size when datasets are projected during tool execution. To learn more about projections, see Coordinate systems, projections, and transformations.

In previous software versions, the cell size was converted using a linear unit conversion when projecting from a Projected Coordinate System (PCS) to another PCS or an angular unit conversion when going from Geographic Coordinate System (GCS) to another GCS. So, projecting a 30 meter raster dataset from UTM zone 10N to WGS84 Web Mercator would continue to use 30 meter as the output cell size. When projecting from GCS to PCS or vice versa, previous versions took an average cell size based on the ratios of diagonal lengths in source and destination projections. Therefore, this method, currently exposed as the ‘convert units’ method simply copied the cell size from input to output, changing units if necessary.

This approach has some limitations:

• It does not account for distortion when projecting from one PCS to another PCS (1 projected meter in, say, a UTM zone at a given location may not cover the same ground distance as 1 projected meter at the same location in an Albers projection).
• When computing the ratios of diagonal lengths, the behavior of the projection at only the four corner points was used. This may have introduced excessive distortion, depending on the projection and the extent.

The two new methods: ‘preserve resolution’ and ‘center of extent’ both are intended to calculate a projected cell size that doesn’t change the ground distance of the cell, thus avoiding unnecessary resampling during an implicit projection operation happening as a Spatial Analyst tool or a feature to raster conversion tool executes. You should use the ‘center of extent’ method when you know the geographic location where your cell size is most accurate. If you use this method, you can use a raster with a small extent in that same area as the cell size source or create a temp raster ‘around’ that location and use that as the cell size source. If you are unsure, use the ‘preserve resolution’ method. To learn more about the three cell size projection methods see How the Cell Size Projection Method environment setting works.

If you do not know at which specific geographic location your cell size is best but want to improve on the ‘convert units’ method when a projection is involved while working with Spatial Analyst tools, you can use the ‘preserve resolution’ method.

## Why use the resolution preserving cell size projection method?

In the ‘preserve resolution’ method the same number of square cells as are in the original extent are preserved in the projected extent. The output cell size is calculated based on the ratios of the areas of the projected extent to the original extent. This approach is based on how Esri software currently chooses a cell size when moving from a camera image coordinate system to a geodetic (GCS or PCS) system. This method calculates the average size of a square cell more accurately for all combinations of GCS and PCS than the default ‘convert units’ method.

If the areas of the original rectangular extent and (shape preserving) projected extent are A0and A1, then the areas of the square cells, respectively, are, ca0 = A0/n and ca1 = A1/n

Since the number of cells remains constant for both cases, the ratio of the area of extent to the area of the square cell are equal, A0/ca0 = A1/ca1

Figure 1: Cell size projection using the new ‘preserve resolution’ method

So, the output cell area is ca1 = (A1/A0) * ca0

and the output cell size is CellSize_projected = √((A1/A0) * ca0)

In this method, the cell size conversion factor is, √(A1/A0)

Let’s look at two examples to see how the ‘preserve resolution’ method picks a better output cell size than ‘convert units’.

### Example 1:

Let’s take an elevation raster (R_input), located in Vermont (Figure 2), in a PCS (NAD_1983_StatePlane_Vermont_FIPS_4400) and project it to another PCS (WGS 1984 World Mercator) using the default cell size projection methods, ‘convert units’ (R_out_CU) and ‘preserve resolution’ (R_out_PR). Here both the spatial references are shape preserving, where the input spatial reference is suitable at a state level and the output spatial reference is suitable for the entire world. We will then compare the geodesic distance between the cell centers to determine which method preserves the geodesic distance more accurately.

Figure 2: Location of the input raster (R_Input), its cell boundaries and cell centers.

Before we compare the geodesic distances between the two methods, lets understand what are the different ways a raster is projected in ArcGIS Pro. When you add a raster to the map in ArcGIS Pro, the spatial reference of the map becomes the same as the spatial reference of the raster. For example, if the first layer added has a NAD_1983_StatePlane_Vermont_FIPS_4400 PCS, the map will have the same spatial reference, and all other layers will project on the fly to match this spatial reference. This on-the-fly raster projection is meant for a richer display experience but does not preserve the raster structure (different cells can have on-the-fly-projected different sizes, rotations, and distortions). However, when you project a raster using the Project Raster tool or specifying the Output Coordinate System environment of a geoprocessing tool, the raster is actually  projected into a new raster structure (every cell is an identical rectangle in the output spatial reference, with sides parallel to the coordinate system axes). When measuring geodesic distance for the comparison of the cell size projection methods, it is recommended to use the actual projected raster instead of the raster that is projected on the fly.

In figure 3, the cell size of the input raster, R_input is 30 meters. This is approximately the same as its geodesic ground distance, which you can find out using the Measure tool in ArcGIS Pro.

Figure 3: Cell boundaries of R_input, cell centers of R_input, R_out_CU, R_out_PR and the geodesic distance between the cell centers. To overlay the points, the cell centers have been projected back to the spatial reference of R_input for comparison.

When the raster is projected using the ‘convert units’ methods, the projected cell size of the output raster, R_out_CU remains 30 meters, but 30m in WGS 1984 World Mercator is a much smaller distance on the ground, 21 meters. So, raster projection using the ‘convert units’ method has unnecessarily increased the resolution of the output raster. If we were projecting in the opposite direction (starting from WGS 1984 World Mercator), then we would have lost a significant amount of raster data. When the same raster is projected using the ‘preserve resolution’ method, its cell size becomes 42 meters and the geodesic ground distance remains 30 meters, which is the same as the geodesic distance of the input raster.

### Example 2:

In this example, we will project from a UTM zone to its adjacent zone, which can happen when mosaicking together many different DEMs for a larger (state sized) area. In figure 4, let’s take a raster (R_input2) in WGS 1984 UTM Zone 11N and project it to its adjacent zone WGS 1984 UTM Zone 12N, creating output (R_out_CU2) and (R_out_PR2) for ‘convert units’ and ‘preserve resolution’ respectively. From here we will again compare the geodesic distance between the cell centers to determine which method preserve the geodesic distance more accurately.

Figure 4: Cell boundaries of R_input2, cell centers of R_input2, R_out_CU2, R_out_PR2 and the geodesic distance between the cell centers. To overlay the points, the cell centers have been projected back to the spatial reference of R_input2 for comparison.

The cell size for the input raster, R_input2 is 30 meters, which is approximately the same as the geodesic ground distance in this area. When the raster is projected using the ‘convert units’ method, the cell size of the output raster, R_out_CU2 becomes 30 meters and the geodesic ground distance becomes approximately 31 meters. When the same raster is projected using the ‘preserve resolution’ method, its cell size becomes 28.99 meters, and the geodesic ground distance is 30 meters, which is the same as the input raster.

In both the examples, the ‘preserve resolution’ method seems to preserve the geodesic ground distance between cell centers better than the ‘convert units’ method. Other combinations of input and output spatial references may, of course, show difference between the methods, but overall it is safe to say that the ‘preserve resolution’ Method is a better approach for preserving geodesic distance while projecting datasets.

An important property of the ‘preserve resolution’ method is that the output cell size depends on the location of the dataset. The same combination of input and output spatial reference will yield a different output cell size if the input raster dataset is at a different geographic location. If you need to use one cell size for different raster datasets (for example, processing adjacent DEM tiles), specify one raster dataset to use as the cell size source while performing raster analysis for the tiles.

Next time you use a Spatial Analyst geoprocessing tool or python command to:

• create an output with a spatial reference different from that of the input dataset,
• use input datasets with different spatial references,
• specify an analysis cell size using a dataset with a different spatial reference,

pay attention to the output cell size and consider using either the preserve resolution or center of extent methods. By default, the projection method will do the ‘convert units’, which existed in previous versions of ArcGIS, but as we have seen, this method may unnecessarily increase or decrease the resolution of your valuable raster data.

Handling projections during analysis in Spatial Analyst

How the analysis window is determined in Spatial Analyst

This blog post has been prepared by James TenBrink, senior software engineer in the raster analysis group at Esri, and Sarmistha Chatterjee.

Original post: https://www.esri.com/arcgis-blog/products/spatial-analyst/analytics/introducing-the-new-resolution-preserving-cell-size-projection-method/

# What’s New for Spatial Analyst and Raster Analysis in ArcGIS Pro 2.2 and Desktop 10.6.1.

Posted by SChatterjee-esristaff Jul 31, 2018

As you will have heard, ArcGIS Pro 2.2, ArcGIS Desktop 10.6.1, and ArcGIS Enterprise 10.6.1are now available for download!  If you use the Spatial Analyst extension, or do Raster Analysis in general, read on for more information about the new capabilities on offer.

### Where do I get it, and what’s changed?

You can download and install the software right now from the following links: ArcGIS Pro 2.2, ArcGIS Desktop 10.6.1, ArcGIS Enterprise.  For a quick summary of all the changes that have been made for these releases, have a look at their respective What’s New topics:

- What’s New in ArcGIS Pro

- What’s New in ArcMap 10.6.1

- What’s New in ArcGIS Enterprise Portal

Here are the main categories of the changes.

1. New and updated tools for the Spatial Analyst extension in ArcGIS Pro and ArcMap.

2. Enhanced performance potential for Spatial Analyst tools with Parallel Processing.

3. New Raster Functions are available.

4. New capabilities are available for Raster Analysis through a Portal in ArcGIS Pro.

5. New capabilities are available for Raster Analysis through a Map Viewer with ArcGIS Enterprise.

6. If you perform Raster Analysis through the REST API, a new task was added.

Following that, some background information on how to configure and perform raster analysis with ArcGIS Enterprise is listed.

### 1. Spatial Analyst extension

For the Spatial Analyst extension product, there some changes to environment behavior, some tools have received new parameters, and more tools have been enabled for parallel processing.

Interactive Python Window

Spatial Analyst tools can use a new ArcPy environment,   arcpy.env.buildStatsAndRATForTempRaster, that can be set to true in the interactive Python window in the application to improve the display of the raster output. This property is accessible and can be set in a standalone python scripts, but will have no effect as the output is not displayed.

It is a Boolean environment that specifies whether or not to calculate statistics and build a raster attribute table for a temporary raster created from an interactive Python session while adding it to the map. It is True by default for all Spatial Analyst tools. When set to False, the statistics will be approximate for the purpose of symbolizing output raster layers, and no raster attribute table will be built.

Distance toolset

The Euclidean distance tools have a new parameter option, distance_method.  With the Geodesic option, the distances are calculated on the ellipsoid, which means that the results of the operation do not change regardless of the input or output projection.  The default Planar option will return the same result as in previous releases. The tools are: Euclidean Allocation (Pro | ArcMap), Euclidean Direction (Pro | ArcMap), and Euclidean Distance (Pro | ArcMap).

An improvement was made to the default behavior for a number of the Cost and Path distance tools.  The processing extent and spatial reference now default to that of the input cost raster.  This will improve the accuracy of the results, and better meets the expected behavior. What’s the reason behind this change?  Previously, the processing extent defaulted to the input source dataset, and since that is often smaller than that of the cost surface extent, the resulting cost paths were limited.  Now, the tools will return the true least-cost path between locations within the full extent of the cost surface.  The full list of the seven affected tools is: Cost Allocation (Pro | ArcMap), Cost Back Link (Pro | ArcMap), Cost Connectivity (Pro | ArcMap), Cost Distance (Pro | ArcMap), Path Distance (Pro | ArcMap), Path Distance Allocation (Pro | ArcMap), Path Distance Back Link (Pro | ArcMap).

Extraction toolset

For the Extract By Mask  (Pro | ArcMap) tool, the input raster will now be used as the default snap raster.  This improves the accuracy of the output by ensuring the correct alignment of the feature mask with the value raster. If both the input raster and the input mask are raster, by default the tool will use the input with the largest cells size as the cell size for the operation.  If the input mask is a feature, by default the input raster will define the cell size and snap raster environment for the operation.

Hydrology toolset

The Flow Accumulation (Pro | ArcMap) and Flow Distance (Pro | ArcMap) tools have a new optional parameter that allows you to specify which option was used to create the particular flow direction raster you are using as input for the tool.  The flow_direction_type raster parameter has the choices of D8 (the default method, which assigns the flow direction to the steepest downslope neighbor), MFD (this method assigns multiple flow directions towards all downslope neighbor), and DINF (this uses the D-Infinity flow method to assign flow direction based on the steepest slope of a triangular facet).

Segmentation and Classification toolset

The Export Training Data For Deep Learning (Pro | ArcMap) tool has a new optional parameter, start_index. This is useful when appending image chips to an existing sequence for training the deep learning classifier.

Surface toolset

The Contour (Pro | ArcMap) tool has two new parameters.  With contour_type, you can now choose to create contour output as polygons instead of just polylines.  There are several types of filled contours to choose from.  When contouring larger or complicated surfaces with polygons, you can use the max_vertices_per_feature parameter to control the maximum number of vertices per polygon feature.

Zonal toolset

For zonal tools that perform a zonal operation on an input value raster, that raster will be used as the default snap raster for the operation.  The benefit here is improved accuracy for the output due to the correct alignment of the feature zone with the value raster. The tools with this new default behavior are Zonal Histogram (Pro | ArcMap), Zonal Statistics (Pro | ArcMap), and Zonal Statistics as Table (Pro | ArcMap).

The Tabulate Area (Pro | ArcMap) tool has a new default behavior as well.  Now, when both the zone input and the class input are raster, the class raster will be used as the snap raster for the operation.  If one of these inputs is a raster and the other a feature, the one that is raster will be used for setting both the processing cell size and the snap raster.

### 2. Parallel Processing

As promised, the parallel processing capability that was added to a number of tools at ArcGIS 10.6 but not Pro 2.1 is now available for those tools in Pro 2.2!  We also enabled this capability for a few more tools for ArcGIS 10.6.1.

ArcGIS Pro 2.2

Distance toolset: Cost Allocation (Pro), Cost Back Link (Pro), Cost Distance (Pro), Euclidean Allocation (Pro), Euclidean Direction (Pro), Euclidean Distance (Pro), Path Distance (Pro), Path Distance Allocation (Pro), and Path Distance Back Link(Pro).

Generalization toolsetNibble (Pro)

Hydrology toolsetFill (Pro), Flow Accumulation (Pro), Flow Direction (Pro), Flow Distance (Pro), Sink (Pro), Stream Link (Pro), and Watershed (Pro).

ArcMap 10.6.1

Distance toolsetCost Back Link (ArcMap), Euclidean Direction (ArcMap), Path Distance (ArcMap), Path Distance Allocation (ArcMap), and Path Distance Back Link (ArcMap).

### 3. Raster Functions

Have you  been using Raster Functions to perform select raster operations on pixels displayed in your ArcGIS Pro map window?  There is a large number of operations you can use on your raster or imagery data out-of-the box, with more available when you have a Spatial Analyst or an Image Analyst extension license available.

Three new distance raster functions are available in Pro 2.2, Cost Back Link, Cost Path, and Euclidean Direction.

In addition, the Flow Accumulation and Flow Distance functions have the new flow direction type parameter.

Following is the current list of Raster functions you can use that require either a Spatial Analyst (SA) or an Image Analyst (IA) license in ArcGIS Pro. Which specific license(s) are required are identified.

Analysis Functions: Kernel Density (SA), Weighted Overlay (SA), Weighted Sum (SA, IA)

Classification Functions: Classify (SA, IA), ML Classify (SA, IA), Segment Mean Shift (SA, IA)

Data Management Functions: Nibble (SA)

Distance Functions : Cost Allocation, Cost Back Link, Cost Distance, Cost Path, Euclidean Allocation, Euclidean Direction, Euclidean Distance, Least Cost Path (All require SA)

Hydrology Functions: Fill, Flow Accumulation, Flow Direction, Flow Distance, Stream Link, Watershed. (All require SA)

Math : Abs, Divide, Exp, Exp10,  Exp2, Float, Int, Ln, Log10, Log2, Minus, Mod, Plus, Power, Round Down, Round Up, Square, Square Root, Times (All require either SA or IA)

Math : Conditional : Con (SA, IA), Set Null (SA, IA)

Math : Logical : Bitwise And, Bitwise Left Shift, Bitwise Not, Bitwise Or, Bitwise Right Shift, Bitwise XOr, Boolean And, Boolean Not, Boolean Or, Boolean XOr, Equal To, Greater Than, Greater Than Equal, Is Null, Less Than, Less Than Equal, Not Equal (All require either SA or IA)

Math : Trigonometric: ACos, ACosH, ASin, ASinH, ATan, ATan2, ATanH, Cos, CosH, Sin, SinH, Tan, TanH (All require SA or IA)

Statistical Functions: Cell Statistics (SA or IA), Zonal Statistics (SA or IA).

Surface Functions: Viewshed (SA).  Note, there are other surface functions, such as Aspect, Hillshade or Slope, that you can use without an Spatial Analyst or Image Analyst license.

### 4. Raster Analysis Tools in ArcGIS Pro

In ArcGIS Pro 2.2, one new Raster Analysis tool has been added, and two other tools have new parameter options.

Hydrology toolset

The Flow Accumulation and Flow Distance tools have a new parameter that allows you to specify the type of the input flow direction raster: D8, Multi Flow Direction (MFD), or D-Infinity (DINF).

Use Proximity toolset

The new Determine Travel Cost Path As Polyline tool is used to calculate the least cost path between sources and destinations.

Follow this link to see the full suite of functionality available in the Raster Analysis toolbox in ArcGIS Pro.

### 5. Raster Analysis in the Map Viewer

Some new tools are available for performing raster analysis in the Map Viewer when you are connected to an updated version of the Portal for ArcGIS in ArcGIS Enterprise.

There is a new toolset for proximity (distance) analysis, with the following three tools:

• Calculate Distance
• Determine Optimum Travel Cost Network
• Determine Travel Cost Path As Polyline

In the Analyze Terrain toolset, there is a new Watershed tool.

Four new Map Viewer tools for Raster Analysis are available in this release

### 6. Raster Analysis in the REST API

The Determine Travel Cost Path as Polyline task was added to the ArcGIS REST API for 10.6.1.  This task can be used to calculate the least cost way to travel from the source location to a destination.

See Get started with the Raster Analysis service to learn more about using the REST API to execute other Raster Analysis service tasks, and how to configure your portal to enable the functionality.

### Some background on raster analytics in ArcGIS Enterprise

Raster Analytics using ArcGIS Enterprise is a flexible raster processing, storage, and sharing system that employs distributed computing and storage technology based on ArcGIS Image Server.  The source data and processed results are stored, published, and shared across your enterprise as image and feature web layers,  according to your needs and priorities.

This capability can be further extended by leveraging cloud computing capabilities and resources. The net result is that raster processing and analysis on very large datasets, which heretofore were difficult to perform in a timely manner on single machines, can now be executed in short order by harnessing the compute power of multiple servers to simultaneously attack the tasks you give them.

In order to have access to Raster Analytics, you need to have an ArcGIS Enterprise deployment available.  After ArcGIS Server is installed, ArcGIS Image Server will also need to be installed and enabled.   Please see the following resources for more details on configuring and using these services for performing raster analysis.

Server

Portal

### Summary

Above we’ve covered the important changes and improvements for raster analysis that have been introduced to the platform in ArcGIS Pro 2.2, ArcGIS Desktop 10.6.1, and ArcGIS Enterprise 10.6.1.  After you have downloaded the new versions, please give the new capabilities a try.  If you encounter any difficulties along the way, be sure to communicate them back to us through the Support channels or GeoNet, so we can work on resolving them.

# Getting the most out of Zonal Statistics

Posted by SChatterjee-esristaff Jul 5, 2018

When working with Zonal tools in the Spatial Analyst toolbox, have you occasionally gotten results that that you didn’t quite expect? Here we’ll cover a few scenarios where and why you might have run into some issues, how to work around them, and how things have changed in the latest release to avoid them in the first place. The Zonal Statistics tool calculates statistics on values of a raster within the zones defined by another dataset. To learn more, read How Zonal Statistics works.

The zone input can either be a feature or a raster. If the zones are defined by features, an internal feature to raster conversion will occur. The internal conversion for a polygon zone uses the cell center method to rasterize the input. With an analysis extent calculation method of intersection-of or union-of, the origin of the internal raster may be determined entirely by the feature class. Snapping is then performed relative to that origin: the internal raster origins at the lower-left corner of the analysis extent and its upper-right corner is adjusted by the cell size. The cells of the internal zone raster and the value raster may not align, which will trigger a resampling during the zonal operation. Resampling will also occur if the input zone is a raster with different cell size and/or alignment. Often times, we end up getting an unexpected result due to the feature to raster internal conversion or misalignment of the zone and value raster. Let’s look at some of these scenarios in detail:

1. Unexpected statistics values

The common source of unexpected statistics values is the misalignment of the cells of the zone and value rasters.

In this example in figure 1a below, you may expect the Zonal Statistics tool to compute the statistics based on the cells of the value raster whose cell center falls within the feature zone, which are values 79, 81 and 27. However, due to the way the default internal conversion will be performed based on the origin and extent, how the feature zone gets rasterized will create an unexpected output. As you can see in figure 1b the rasterization grid used for the internal conversion does not align with the value raster. Therefore, using the cell center method, the zone raster will end up analyzing a different set of cells from the value raster (see figure 1c) than you originally anticipated from figure 1a.

Figure 1: Internal conversion of feature zone without considering the value raster for cell alignment.

To avoid this issue, there is a step you can take to ensure that the feature zone being converted to a raster is aligned with the value raster. Simply set the snap raster to the value raster from the tool environment. The misalignment can also occur when the input zone is a raster, if the cell size and/or cell alignment does not align with the value raster. You can set the snap raster environment to the value raster to get the expected output.

Let’s look in the example below, in figure 2b, to understand how setting the snap raster to the value raster ensures the rasterization grid used for the internal conversion aligns with the value raster in figure 2b. As a result, the zone raster will analyze those cells from the value raster (see figure 2c) that you originally anticipated from figure 2a.

Figure 2: Internal conversion of feature zone considering the value raster for cell alignment.

The good news is that ArcGIS Pro 2.2 and ArcMap 10.6.1 use the value raster as the snap raster by default for internal conversion of the feature zones. If you are using a version prior to ArcGIS Pro 2.2 or ArcMap 10.6.1, specify the value raster as the snap raster in the environment.

2. Missing zones in the output

The most frequent cause of missing zones in the output occurs when the cell center of the rasterization grid does not fall within the feature zone. This can occur for zones that are smaller than the area of a cell of the internal zone raster or even for larger zones.

In the example below, let’s look at how rasterization occurs for zones of different size and location. Figure 3a has three zones, where, zone1 is larger than a cell, and zone2 and zone3 are smaller than a cell, and the cell center falls outside zone2 and within zone3. During the zone rasterization process in figure 3b, it so happens that no cell centers fall within zone1 and zone2, and only zone3 contains a cell center. Therefore, only zone3 will be rasterized and the other two zones will essentially disappear, as shown in figure 3c.

Figure 3: Internal conversion of feature zone leading to missing zones.

To avoid this, ensure that each of your zones contains one or more cell centers. You can create more cell centers by specifying a smaller cell size in the environment. The default analysis cell size comes from the value raster. Therefore, specifying a cell size that is smaller than that of the value raster will enable more zones to be captured.

For figure 4, let’s repeat the same example from figure 3 to see how changing cell size better captures all the zones. Similar to figure 3a, figure 4a also has three zones, where, zone1 is larger than a cell, and zone2 and zone3 are smaller than a cell but are spatially located differently. Figure 4b shows a finer rasterization grid based on a cell size that is four times smaller than the default cell size. During the zone rasterization process, multiple cell centers now fall within each of the zones. As a result, all the zones get rasterized as shown in figure 4c.

Keep in mind that specifying a smaller cell size, will generate a larger output raster.  In this example, the output raster will be sixteen times larger than the raster with default cell size. Even more important, the higher resolution output may give the wrong perception of a higher quality result than what it actually is, since the additional detail does not actually exist in the input value raster.

Figure 4: Internal conversion of feature zone with a smaller cell size capturing all zones.

Note: If you are using a version prior to ArcGIS Pro 2.2 or ArcMap 10.6.1, specify the value raster as the snap raster in the environment.

3. Other sources of unexpected result

You may also get unexpected results during the rasterization of feature zones if you have:

• coincident points or multiple points on one cell
• coincident polylines or multiple polylines passing through one cell
• coincident polygons or partially overlapping polygons

Summary

With the information presented here, you should be able to better understand how this tool operates, and how to guarantee better and more understandable results.  Keep in mind that there is a similar behavior in play for other Zonal tools, such as Tabulate Area, Zonal Histogram and Zonal Statistics as Table so you can employ the lessons learned here to achieve greater success with those tools, too. Keep in mind, that ArcGIS Pro 2.2 and ArcMap 10.6.1 now use the value raster as the snap raster by default for internal conversion of the feature zones. If you are using a version prior to ArcGIS Pro 2.2 or ArcMap 10.6.1, specify the value raster as the snap raster in the environment.

Let us know if you encounter other scenarios where you get an unexpected result while doing your analysis, or if you have any questions or comments. You can comment here or reach me at schatterjee@esri.com.

# Executing Spatial Analyst Tools Using ArcGIS Pro SDK

Posted by xuguang_wang-esristaff Jun 29, 2018

ArcGIS Pro SDK for .Net allows you to extend the ArcGIS Pro user interface by creating custom add-ins leveraging different GIS functionalities. In this blog, we will take a look at how to execute Spatial Analyst tools in your add-in using the ArcGIS Pro SDK.

How to execute a geoprocessing tool

The ArcGIS.Desktop.Core.Geoprocessing namespace in the SDK provides a Geoprocessing class to help execute a geoprocessing tool. The Geoprocessing class within the namespace offers necessary methods to specify the input parameters, set up environment settings, and execute the geoprocessing tool. The static method, ExecuteToolAsync is used to execute the tool.
The first argument for ExecuteToolAsync is the geoprocessing tool name as a string. To specify a tool uniquely, the tool name must be appended with its toolbox alias. For example, the Copy Features tool is in the data management toolbox, and the alias of the data management toolbox is management. Therefore, to execute the Copy Features tool, the tool name should be specified as CopyFeatures_management. Without the toolbox alias, or with a misspelled toolbox alias, an error will be returned and the tool will not be executed because the geoprocessing framework is not able to find the tool.
The second argument for ExecuteToolAsync is the tool parameter values array, which holds all the input and output parameter values for the geoprocessing tool. Do not create the array yourself. Simply call the method MakeValueArray, to create it, passing in all input and output parameter values as arguments.
Additionally, you may want to specify an environment setting for the tool execution. The third argument for ExecuteToolAsync allows you to do that. Like the input/output parameters, the environment settings also need to be specified in an array. Again, no need to create the array by yourself, simply call the method MakeEnvironmentArray to make one.
The rest of the arguments for ExecuteToolAsync are optional. As an asynchronous method, ExecuteToolAsync will start the geoprocessing tool execution in a different thread other than the user interface (UI) thread, which allows the ArcGIS Pro UI to remain active while the tool executes.

How to execute a Spatial Analyst tool

Executing a Spatial Analyst tool is no different than any other geoprocessing tool. All we need to do is supply a tool name with an appropriate toolbox alias, which is sa. The following example will walk you through a C# example on how to execute the Zonal Statistics tool to calculate average elevation for each zone, using SDK.

Let’s start by specifying the tool input and output parameters:

// declaring tool parameter variables
string inZoneRas = @"c:\data\county.tif" ; // input zone raster
string zoneField = "Value"; // zone field
string inValueRas = @"c:\data\elevation.tif" ; // input valueraster
string outRaster = @"c:\output\outzs01.tif" ; // output raster
string statType = "MEAN" ; // statistics type
bool ignoreNoData = true ; // ignore NoData option

Now let’s create the first array with the above parameter values using the Geoprocessing.MakeValueArray method:

// create an array for the input parameter values
var valueArray = Geoprocessing.MakeValueArray(inZoneRas, zoneField, inValueRas, outRaster, statType, ignoreNoData);

Next, create another array for environment settings using Geoprocessing.MakeEnvironmentArray to set the cell size for analysis and to overwrite output if it exists:

// create an array for the environment settings
var envArray = Geoprocessing.MakeEnvironmentArray(overwriteoutput: true, cellSize: 30);

Finally, let’s execute the tool by calling the ExecuteToolAsync method, passing in the tool name, the tool parameter values array, and the environment settings array that we created above. Notice that the tool name ‘ZonalStatistics’ is appended with its toolbox alias ‘sa’, which refers to the Spatial Analyst toolbox:

// run ZonalStatistics
var gpresult1 = await Geoprocessing.ExecuteToolAsync("ZonalStatistics_sa", valueArray, envArray);

Because the Spatial Analyst is an extension product, a license is required in order to use its capabilities. However, we did not enable the license in the preceding code snippet as in our case, it is enabled through the ArcGIS Pro application. If the Spatial Analyst license is not already enabled, the tool execution will fail with a licensing error. To enable the Spatial Analyst license, configure your licensing options from the ArcGIS Pro application.

How to execute a map algebra expression

Now that we have learned how to execute a Spatial Analyst tool using the ArcGIS Pro SDK, let’s look at another example. In raster modeling, we often need to execute map algebra expression to solve a complex problem. In the Spatial Analyst toolbox, the tool that allows us to execute map algebra expression is called Raster Calculator.
In your add-in, you may take advantage of the Raster Calculator tool to execute single-line map algebra expressions. The Raster Calculator tool has two parameters only. The first parameter is the map algebra expression itself. The second parameter is the output raster, which saves the result of the calculation.
In the example below, let’s build a map algebra expression to find the land uses for areas with slope greater than 30 degrees and aspect less than 120 degrees, and use the Raster Calculator tool to execute it.

First, let’s specify the input raster datasets that will be used in our map algebra expression:

// input raster datasets for the map algebra expression
string inRas1 = @"c:\data\in_slope.tif" ;
string inRas2 = @"c:\data\in_aspect.tif" ;
string inRas3 = @"c:\data\in_landuse.tif" ;

Now, let’s build the map algebra expression. Notice that the input raster datasets are embedded in the expression itself, enclosed by single quotation marks:

// define the map algebra expression
string maExpression = String.Format("Con((‘{0}’ > 30) & (‘{1}’ < 120), ‘{2}’)",inRas1, inRas2, inRas3);

We need another variable to specify the output raster:

// define the output raster
string outRaster = @"c:\output\outrascal01.tif" ;

Now we can create an array using maExpression and outRaster:

// make the input parameter values array
var valueArray = Geoprocessing.MakeValueArray(maExpression, outRaster);

Finally, we kick off the Raster Calculator tool, which will execute the map algebra expression and save the result in outRaster:

// execute the Raster calculator tool to process the map algebra expression
var gpresult1 = await Geoprocessing.ExecuteToolAsync("RasterCalculator_sa", valueArray);

Summary

Now that you are comfortable executing Spatial Analyst tools and map algebra expressions, you can perform raster modeling in your add-ins. If you need more information, check out our help pages on ArcGIS Pro SDK and ArcGIS Pro Spatial Analyst.

# Raster Analysis and Spatial Analyst Sessions at UC2018

Posted by SChatterjee-esristaff Jun 27, 2018

The 2018 Esri User Conference (July 9-13) is almost here – are you ready?  We here on the Spatial Analyst team are busy practicing and polishing our presentations, and are looking forward to showing you what we’ve been working on since last summer.  If you do any kind of raster or imagery analysis, or do multidimensional analysis, please look at the guide below to help you plan out your time with us.

To help target your areas of interest, we’ve categorized things into several groupings. Each one will list out a general purpose of the grouping, followed by the specific session titles in chronological order.  Repeat presentations are also listed, in case you are not able to attend the first showing.

1. Introduction to Raster Analysis
2. Applied Raster Analysis
3. Imagery
4. Raster Analytics
5. Python
6. Multidimensional and Scientific data

See the full agenda for more details on these sessions, and more. Be sure to download the Esri Events Mobile App too!   (UC content coming soon)

Please come by the Island in the Showcase during the week (Tuesday and Wednesday from 9 – 6, and Thursday from  9 – 1:30).  We’d love to hear about the work you do, any difficulties you encounter, and any ideas you might have to make our software suit your needs even better.

Finally, remember to check out the Spatial Analysis : The Road Ahead session for a peek into the future.

### 1. Introduction to Raster Analysis with Spatial Analyst

The ArcGIS Spatial Analyst extension offers a large number of spatial modeling and analysis tools.  The following sessions cover many aspects of this product.  The introductory sessions demonstrate the wide range of capabilities on offer.  The Map Algebra session introduces the simple and powerful language that Spatial Analyst uses for performing a wide range of raster and feature geographic analysis. Following that are more focused sessions on specific aspects of the capabilities, such as suitability modeling, cost distance modeling, and interpolating surfaces from vector data.

Spatial Analyst: An Introduction

 Tue, July 10 8:30 am – 9:30 am Tech Workshop SDCC – Room 29 D Tue, July 10 4:00 pm – 5:00 pm Tech Workshop SDCC – Room 29 D

Map Algebra: Getting Started

 Tue, July 10 10:00 am – 10:45 am Demo Demo Theater 10

Finding the Best Locations Using Suitability Modeling

 Tue, July 10 2:30 pm – 3:30 pm Tech Workshop SDCC – Room 30 D Wed, July 11 1:00 pm – 2:00 pm Tech Workshop SDCC – Room 29 D

Cost Distance Analysis: Identifying the Best Path

 Wed, July 11 8:30 am – 9:30 am Tech Workshop SDCC – Room 29 C Thu, July 12 4:00 pm –   5:00 pm Tech Workshop SDCC – Room 29 C

Interpolating Surfaces in ArcGIS

 Wed, July 11 10:00 am – 11:00 am Tech Workshop SDCC – Room 30 C Thu, July 12 4:00 pm – 5:00 pm Tech Workshop SDCC – Room 31 A

Surface Interpolation in ArcGIS

 Wed, July 11 11:15 am – 12:00 pm Demo Demo Theater 10

### 2. Applied Analysis

If you are interested in learning more about applied raster analysis capabilities offered by ArcGIS, the following sessions will be of interest to you.  Hydrological modelling is one of the specific applications for which raster-based processing is most highly suited.  Attend these sessions to learn more about creating hydro-conditioned surfaces, watershed delineation, stream flow calculation, flood modeling, and flood impact assessment.  Also included in this group is a session on applied suitability modelling, in particular, the workflow used to identify the best habitat patches for bobcat, and how to connect them with the optimum network of wildlife corridors

HydroGIS 2018 – Forecasting and Response for Flows and Floods

 Sun, July 8 8:30 am – 5:00 pm Free preconference Workshop SDCC – Room 29 D
 Wed, July 11 8:30 am – 9:30 am Tech Workshop SDCC – Room 30 A Wed, July 11 4:00 pm – 5:00 pm Tech Workshop SDCC – Room 30 A

Hydrologic and Hydraulic Modeling in ArcGIS

 Wed, July 11 10:00 am – 11:00 am Tech Workshop SDCC – Room 29 D Thu, July 12 4:00 pm – 5:00 pm Tech Workshop SDCC – Room 30 A

GIS Techniques for Floodplain Delineation

 Wed, July 11 1:15 pm – 02:00 pm Demo Demo Theater 10
 Thu, July 12 12:15 pm – 1:00 pm Demo Demo Theater 10

### 3. Raster Analytics with Image Server

The ArcGIS Enterprise Image Server can be used for quickly processing very large collections of raster and imagery data. The following sessions will introduce the capabilities of this product, describe the architecture, detail the deployment options, and identify the performance benefits offered by distributed processing.

ArcGIS Enterprise: Raster Analytics in Image Server

 Wed, July 11 10:00 am – 11:00 am Tech Workshop SDCC – Room 01 A/B Thu, July 12 10:00 am – 11:00 am Tech Workshop SDCC – Ballroom 06 F

ArcGIS Enterprise: Building Raster Analytics Workflows

 Wed, July 11 1:00 pm – 2:00 pm Tech Workshop SDCC – Room 31 A

ArcGIS Enterprise: Deploying Distributed Raster Analytics

 Thu, July 12 1:00 pm – 2:00 pm Tech Workshop SDCC – Room 30 E

### 4. Imagery and Image Analyst

Whether your data comes from satellite imagery, aerial imagery, or drones, ArcGIS provides extensive capabilities for imagery management, visualization and analysis.  See the following list of sessions if you want to find out more about imagery collection, processing, display, classification, interpretation and information extraction.

 Sun, July 8 2:15 pm – 3:00 pm Imagery Summit Marriott – Grand Ballroom 5 Sun, July 8 4:15 am – 5:00 pm Imagery Summit Marriott – Grand Ballroom 5

Imagery in ArcGIS: What’s New

 Tue, July 10 8:30 am – 9:30 am Tech Workshop SDCC – Rm 07 A/B Thu, July 12 2:30 pm – 3:20 pm Tech Workshop SDCC – Rm 32 A/B
 Tue, July 10 10:00 am – 11:00 am Tech Workshop SDCC – Room 03 Tue, July 10 4:00 pm – 5:00 pm Tech Workshop SDCC – Ballroom 06 F

ArcGIS Pro: Image Segmentation, Classification, and Machine Learning

 Tue, July 10 1:00 pm – 2:00 pm Tech Workshop SDCC – Room 17 B Thu, July 12 1:00 pm – 2:00 pm Tech Workshop SDCC – Room 15 A

Image Analysis and Visualization using the Web Map Viewer

 Tue, July 10 1:15 pm – 2:00 pm Demo Demo Theater 01

ArcGIS Enterprise: Managing and Serving Imagery in the Cloud Using ArcGIS Image Server

 Tue, July 10 2:30 pm – 3:30 pm Tech Workshop SDCC – Room 04 Thu, July 12 10:00 am – 11:00 am Tech Workshop SDCC – Room 02

How Raster Functions Revolutionized the World Population Estimate

 Wed, July 11 10:00 am – 10:45 am Demo Demo Theater 11

Image Processing using Raster Functions    (Intermediate)

 Wed, July 11 2:30 pm – 3:15 pm Demo Demo Theater 01

### 5. Raster and Image Analysis with Python

Python offers a powerful way to perform raster and image analysis in ArcGIS.  The integration of Map Algebra with Python via the Spatial Analyst ArcPy module offers a broad range of capabilities for raster analysis and automation of geoprocessing workflows. If you like NumPy, we’ve got that covered too!

With the raster module of the ArcGIS API for Python, you can derive information from raster imagery with the large number of available raster functions. As the datasets continue to get larger and larger, learn about how parallel processing, distributed processing and the ArcGIS Image Server can be used to optimize performance.

Python: Raster Analysis

 Tue, July 10 1:00 pm – 2:00 pm Tech Workshop SDCC – Room 31 C Thu, July 12 2:30 pm – 3:30 pm Tech Workshop SDCC – Room 03

ArcGIS API for Python: Image Analysis

 Thu, July 12 10:00am – 11:00 am Tech Workshop SDCC – Room 16 A

### 6. Scientific and Multidimensional data

Scientific and multidimensional data is a burgeoning area of growth in GIS.  The ArcGIS platform has great capabilities for supporting the import, management, visualization, analysis and sharing of this type of information.  If you would like to use netCDF, HDF or GRID data for global scale spatial and temporal analysis, the following sessions will be of interest to you.

Scientific Tools for Atmospheric and Climate Analysis

 Sun, July 08 8:30 am – 5:00 pm Preconference SDCC – Room 30 D

Working with Scientific Data in ArcGIS

 Tue, July 10 1:00 pm – 1:20 pm Showcase SDCC – Interoperability and Standards Spotlight Theater

Scientific and Multidimensional Raster Support in ArcGIS

 Wed, July 11 8:30am – 9:30am Tech Workshop SDCC – Room 31 B

Analyzing Multidimensional Scientific Data in ArcGIS

 Wed, July 11 1:00 pm – 2:00 pm Tech Workshop SDCC – Room 30 E

Come join us for this Road Ahead session to find out what enhancements have been made in ArcGIS Spatial Analysis and what we are planning for the upcoming year, especially in the areas of machine learning, big data, real time analysis, and integrating spatial analysis with the scientific Python ecosystem.

 Wed, July 11 1:00 pm – 2:00 pm Tech Workshop SDCC – Ballroom 06 B Fri, July 13 9:00 am –  10:00 am Tech Workshop SDCC – Room 05 A

# Multiprocessing with ArcGIS - Raster Analysis

Posted by NRajasekar-esristaff Mar 29, 2018

With growing raster dataset sizes, the processing times for analysis are becoming longer and longer. By employing concepts from parallel and distributed computing, we can improve performance and scalability of workflows with Spatial Analyst tools. For some tools, Spatial Analyst offers enhanced performance with built-in parallel processing capability. These tools use the multi-core processors on modern computing hardware to complete processing tasks more quickly. Additionally, the ArcGIS Image Server provides capabilities for scalable distributed raster analysis and distributed storage of large image and raster collections. However, built-in capabilities are not yet available across all Spatial Analyst tools. In this blog, we will identify scenarios where the multiprocessing Python module (Python.org) can be used to optimize performance of raster analysis workflows by dividing up the work between multiple processes on a given machine. We will identify candidate raster operations that benefit most from parallel computation and learn how to develop efficient parallel systems for raster geoprocessing within the robust Python environment.

Candidate raster operations for parallelization can be broadly classified into three geoprocessing scenarios. Parallelism is invoked using a different mechanism for each scenario.
Processing a large raster dataset with a single analysis tool
Batch processing a collection of raster datasets by running a single analysis tool multiple times
• Raster geoprocessing workflow by running suitable analysis tools chained together

From our in-house testing, we evaluated the performance of a local operation executed parallelly on a large sample dataset. The graph below summarizes the performance improvements that were observed.

The advantages of optimizing raster analysis tasks with Python multiprocessing are evident from this graph. We can see significant improvements while adding more processes to help with the analysis, especially going from one to four processes. Beyond that, gains were negligible due to the overhead of spawning multiple processes.

## Processing a large raster dataset

The increase in resolution of raster datasets has led to larger and larger data sizes. Currently, datasets are on the order of gigabytes and increasing, with billions of raster cells. While computing power of the processors and size of the memory in computers have increased appreciably, legacy equipment and algorithms suited to manipulating small rasters with coarser resolution make processing these improved data sources costly. [1]

Data decomposition, also known as divide and conquer, is a popular strategy used in parallel computing that we will take advantage of to parallelly process a large raster dataset. The algorithms used in raster analysis tools can be broadly classified into four categories – local, focal, zonal and global operations. For a deeper dive into the types of cell-based raster operations, read this article. Local, focal and zonal raster operations are simple to program with when it comes to data decomposition. Once the data is decomposed appropriately, each data ‘chunk’ can be operated on independently on by a process without the need to communicate with other processes. However, global raster operations are tougher to integrate with data decomposition and require communication between processes. Let us look at an example of processing a large raster dataset using a local math raster operation, Square Root. The Local, or per-cell operations, are the simplest to parallelize using a ‘divide and conquer’ strategy, since the resulting value at each cell only depends on the input value at that cell location. For each cell, the Square Root tool calculates the square root of the value from that cell. Using the tool serially would mean simply running the Square Root tool on the entire large dataset, but, this process can be time consuming.

Default, non-optimized serial approach to processing a large raster

Instead, through data decomposition, we can redesign the analysis task to utilize multiple worker processes simultaneously, thus improving the performance of the overall analysis. The graphic below depicts splitting the domain of a large raster into several smaller chunks, and using multiple worker processes to simultaneously perform analysis on each of the sub-sections. The results are then stitched back together for the final output.

Multiprocessing large rasters parallelly

A sample script is shared on GitHub here that goes in depth on how this problem can be solved programmatically using ArcGIS Desktop and the Python multiprocessing module.

## Batch processing a collection of raster datasets

When it comes to batch processing many datasets through a raster analysis tool, the common approach would be to use an iterator in ModelBuilder or write a Python script with a loop to iterate over each dataset in the batch and process them serially. This method can be time consuming when working with a large collection of rasters, particularly when running analysis tasks that have significant compute times.

Default sequential approach to processing a multiple rasters

Parallelism is a straightforward mechanism for batch processing when the analysis of each dataset in the batch can be performed independently from the rest. From the graphic below we can see how multiple processes working simultaneously enable faster processing of the batch queue.

Using multiprocessing to efficiently process multiple rasters in a parallel fashion

A sample script is shared on GitHub here that describes in detail how batch processing can be solved programmatically using ArcGIS Desktop and the Python multiprocessing module.

## Raster geoprocessing workflows

A geoprocessing workflow is a multiple-step procedure that combines geoprocessing tools and geographic data to produce a meaningful result. Within ArcGIS, raster geoprocessing workflows can be written as models using ModelBuilder or as Python scripts. The standard approach in either cases is to chain together the tools needed to perform your analysis and execute them serially. Let us look at an example workflow that assesses a suitability raster by running the Slope, Aspect, Reclassify and Weighted Sum tools. These tools are available as part of the Spatial Analyst Extension in ArcGIS Desktop.

Default approach to performing a workflow

In the above example, as tools are executed in a serial fashion, only one worker process can be utilized at a time to complete the workflow. To parallelize any geoprocessing workflow, the first task is to decompose the problem and identify parts of it that may be handled independently. This workflow is a good candidate for parallel processing as the execution of Slope, Aspect and Reclassify tools are independent tasks. These tasks have no dependencies on the execution of other tools in the workflow. However, the Weighted Sum tool is a dependent task, having dependencies on the outputs from the Slope, Aspect and Reclassify tools. Until the Slope, Aspect and Reclassify tools have finished executing, the Weighted Sum tool cannot begin its analysis. Having identified the independent and dependent tasks within this workflow, we can redesign this problem to parallelize it.

Incorporating parallel processing into model workflows

In this redesigned workflow, we are making use of multiple worker processes to execute independent tasks simultaneously, rather than depend on a single worker to process each task serially.

## Best practices and considerations

There are many factors to consider while combining the capabilities of the arcpy and multiprocessing modules to improve performance of raster geoprocessing tasks.

• There are always overheads associated with spawning multiple processes that must be accounted for when considering a parallelized approach. In situations where the processing task is complex, and the processing data is large or the batch size is huge, parallelism can provide performance benefits that outweigh the overhead. In other situations, such as when the processing task is very simple, or if the processing datasets were not particularly large, it may be that utilizing multiprocessing does not provide any significant performance benefit and, in some cases, slows down overall processing speed. As parallelizing your code can be tedious, you should consider the end goal and opportunity cost of investing time in parallelization. For a task that is run repetitively, say on a scheduled basis, and in big data processing, parallelization can be advantageous.

• Avoid writing output rasters from multiple processes to a common File Geodatabase (FGDB) or to multiple Esri Grid rasters within a common folder workspace. These output formats often experience schema locks or synchronization issues when accessed by multiple simultaneous processes.

• You are encouraged to use ArcGIS Pro, ArcGIS Server or ArcMap with ArcGIS for Desktop Background Geoprocessing (64-bit) for parallelized raster analysis. Using 64-bit processing to perform analysis on systems with large amounts of RAM may help with processing large data efficiently.

The Spatial Analyst development team continues to work on parallelizing and distributing additional tools to further improve performance and scalability. Be on the lookout for these enhancements with future releases of ArcGIS Desktop, ArcGIS Pro and ArcGIS Image Server!

## References

[1] Barnes, Lehman, Mulla. “Priority-Flood: An Optimal Depression-Filling and Watershed-Labeling Algorithm for Digital Elevation Models”. Computers & Geosciences. Vol 62, Jan 2014, pp 117-127, doi: ”10.1016/j.cageo.2013.04.024”.

# Interpolation in the presence of barriers

Posted by slynch-esristaff Jan 25, 2018

For the Spline with Barriers interpolation tool, the enhancements that were made when the tool was implemented in ArcGIS Pro 2.0 are now also available in ArcGIS 10.6. Let’s have a look at the history of this tool, and some example applications.

Tool History

The Spline with Barriers tool was first implemented in ArcGIS more than ten years ago.  It was included in the Spatial Analyst and 3D Analyst toolboxes initially as a script tool which called a Java library that performed the interpolation.  The tool was not included in the initial release of ArcGIS Pro.  However, for ArcGIS Pro 2.0 the tool was rewritten in C++ with some optimizations and bug fixes. This improved C++ implementation is now also available in ArcGIS 10.6, and has replaced the older Java implementation.

Applications

Often we have a variable that is spatially not continuous and we want to use interpolation to produce an output raster that has values everywhere. For example, soil characteristics, or water quality in a braided meandering stream. In these cases you can use Spline with Barriers and specify either a polygon or polyline feature class to define the boundary of the discontinuity.

Methods and comparisons

Spatial Analyst and 3D Analyst have two interpolation methods that have an optional barrier input, those being IDW and Spline with Barriers.

The main difference between IDW and Spline with Barriers, other than the algorithm, is the way in which the points that are to be used in the interpolation are defined. IDW uses a “line of sight” approach, which means that if a point can’t be “seen” because of a barrier, then it is not used to make the prediction at that particular location. Spline with Barriers, on the other hand, weights the points based on the shortest distance around a barrier. In the illustration below, IDW (on the left) will only use 3 points to make a prediction when a barrier intervenes. In contrast, Spline with Barriers will utilize all 4 points to make the prediction.

Figure 1: Searching neighborhood for IDW (left) and Spline with Barriers (right) when using a barrier.

Case Study

As an example, we’ll use the same case study described in Creating Faulted Geologic Surfaces with ArcGIS by Mike Price in the ArcUser Summer 2014 issue. In this article, the Spline with Barriers tool (the original Java version) is used.

We’ll use the point feature class, which contains the depth to the top of the Mississippian Leadville Limestone, and a polyline feature class defining the subsurface faults, to create an output depth raster with a cell size of 5 meters.

Figure 2: The geoprocessing tool dialog is on the left. To the right is the output depth raster, with the input points and barrier layers displayed on top.

Figure 3: The output depth raster displayed in a 3D perspective view. Note the steps created by the barriers.

We often rely on contour lines to aid our interpretation of an interpolated surface. However, in the vicinity of fault lines there may be abrupt changes in the depth values between some adjacent raster cells.  As a result, we cannot use the standard Contour tool as it will incorrectly produce many contours in these areas. The Contour with Barrierstool is well suited for this type of data as illustrated in the images above.

Conclusion

This exercise shows how to model the elevation of the top of the Mississippian limestone in Lisbon Valley, Utah, using only formation top elevations and fault barriers to generate a reliable predictive elevation surface.

By date: By tag: