Applications Prototype Lab Blog - Page 5

Solar Eclipse Finder is a JavaScript-based web application that displays past and future solar eclipses that pass through a user defined location.  The source data is published as an AGOL hosted service with the paths of 905 solar eclipses from 1601 to 2200.  The eclipse paths were prepared by Michael Zeiler from data courtesy of Xavier Jubier.

The live application is available here.

Source code is available on agol and github.

Originally developed 2½ years ago as a Silverlight-based web application (see blog posting here), we wanted to confirm that the same performance and advanced symbology is achievable today with HTML5/JavaScript in modern browsers.

jQuery & Bootstrap

jQuery is a JavaScript framework for DOM manipulating.  It is important to note that jQuery is not a prerequisite for mapping apps using Esri's ArcGIS API for JavaScipt. It is however a prerequisite of many third party JavaScript libraries like Bootstrap, a popular user interface framework.  This application uses Bootstap's popover tooltips in the fly-out attribute window and its modal dialog during start-up.

D3

The tapered symbol used by eclipse shadow paths is achieved using a linear gradient fill.  Linear gradient fills are not supported by ArcGIS API for JavaScript.  However linear gradient fills are supported by SVG, the underlying technology used by Esri's JavaScript API for renderering vectors.  We used Mike Bostock's D3.js JavaScript library to insert and apply linear gradient fills directly to the map's embedded SVG node.

Conclusion

Updating this application was a two step process.  First the eclipse dataset was republished as an AGOL hosted feature service and, second, the app was rewritten in HTML/JS.  Both tasks were relatively effortless and only took a couple of days in total.

GeoJigsaw is a community driven geographic jigsaw puzzle.  If you are feeling creative you can create a puzzle and share it with the puzzle community.  If you are feeling competitive, try beating the high score on someone else's puzzle.

Click here to view the live application.(works best in Microsoft Internet Explorer and Mozilla Firefox)

Click here to download the source code from ArcGIS Online.  A simpler version of this application is available on github here.

GeoJigsaw is a JavaScript-based web application inspired by a Silverlight application developed about two years ago called Puzzle Map.  Unlike a recently published geogame that uses Facebook, in this application we wanted to explore anonymous collaboration.  That is, anyone can anonymously create, share and play puzzles.

Voronois

In the app developed two years ago, the puzzle design was static.  In this application we wanted to offer puzzles of varying difficulty and size so we needed to implement a technique of dynamic puzzle creation.  After a little research we discovered this example of a voronoi tessellation using D3.  D3's voronoi implementation and associated SVG-based visualization library are the basis of this game.

Unlike the D3 sample, our app did not use completely randomized points.  If a user created or selected an "impossible" puzzle then a 10 by 10 grid of points is created and nudged slightly before being turned into 100 piece voronoi diagram using D3.  This was only part of the puzzle (excuse the pun), each piece needed the addition of one or more tabs.  Tab addition is essential to give the game its recognizable jigsaw look. Through a process of iteration, tabs are appended to any side of sufficient length and reversed if an opposing tab exists.SVG Filters

The finishing touch to give the puzzle a realistic look is the application of an inner bevel using a SVG filter.  SVG filters are hardware accelerated in Internet Explorer and Mozilla Firefox but not in Google Chrome.  Unfortunately the Chrome's software rendering of SVG filters makes complex puzzles almost unplayable.  This may change in future releases of Chrome.

Security

Puzzles designs, ratings and scores are stored in ArcGIS Online (AGOL) hosted services.  We intended the application and associated services to be accessed by non-AGOL users.  This meant that AGOL user credentials could not be used to restrict access to prevent unanticipated malicious activity.  As such, we used the security model discussed in the previous post, that is, app registration and an intermediate web proxy.

Libraries Used

• Sylvester by James CoglanVector and Matrix mathematics modules for JavaScript.

Conclusion

This project demonstrations that modern browsers are more than capable of impressive visualizations without the need of plugins such as Silverlight or Flex.  We also wanted to experiment with anonymous game play, time will tell if the lack of user identification is an incentive or disincentive to play.  Good luck!

Map Quiz is fun JavaScript-based geo-game developed by the Applications Prototype Lab. The game tests your geographic knowledge with six randomly selected satellite images. Prove that you are a geo-genius at home, school or in your workplace!

Click here to access the live application.

Click here to download the source code.

Introduction

This project is a port of an application initially developed as a Windows Store application.  The purpose of the application is to present a fun geography-focused game (or "geo-game") based on Esri technology.  The application is primarily based on Esri's ArcGIS API for JavaScript and is powered by ArcGIS Online (AGOL) hosted services for scoring and questions.  Originally the Windows Store based application used AGOL-based authentically, in the JavaScript edition we decided to use Facebook authentication in an effort to appeal to a large audience.

Why Facebook?

One of the motivations of gaming is the thrill of competition, whether personally or with others.  In order to identify users so that they could monitor their scores, or those of others, there needed to be some sort of authentication.  The obviously choice is AGOL.  AGOL is the perfect choice when collaboration is needed within the GIS community but a large percentage of the target audience of this application may not have AGOL credentials. Arguably, Facebook may not be completely ubiquitous but certainly common.  As such, we decided to use Facebook and the Facebook API to authenticate users.  To ease privacy concerns this application requests and displays only a small subset of profile information, specifically, a person's profile picture, last name and initial of first name.

Controlling Access to Hosted Services

The game's questions, answers and scores are stored in two ArcGIS Online hosted feature services.  Hosted services are easy to create and allow for powerful spatial queries.  However access to hosted service is either unrestricted, or confined to an organization or to users that belong to specific groups.  Because this game is intended for non-AGOL users, we needed a way of restricting access to the hosted services to just the Map Quiz web application.

This was achieved by registering the app on AGOL.  The resulting app id and secret were then used in a web proxy that granted exclusive access to the hosted services to only the Map Quiz web application.  The proxy and instructions how to implement it are here.

Spinning

The spinning map on the landing page or the gradual zooming of each question is achieved using CSS3 animation and the animo JavaScript library (see above).  On modern browsers the animation effects are smooth and consistent.  With respect to the spinning map, map edges needed to be expanded outwards to avoid white patches appearing in the corners.  To avoid this we applied negative margins using a Pythagorean computation.  One disadvantage of the spinning map is that image seams are occasionally observed.

Silverlight vs. JavaScript

As a reformed Silverlight developer I have been pleasantly surprised with the performance and capabilities of JavaScript-based web applications.  To date, I have yet to encounter any Silverlight capability that could not be achieved with HTML5/CSS3.  The biggest issue has been the paradigm shift from Silverlight's large well-documented framework to the necessity of working with a half dozen lightly documented open source libraries.

Libraries Used

As mentioned above, this project is based on a few open source libraries, these libraries are listed and described below.

• jQuery by jQuery Foundation IncA JavaScript framework for DOM manipulation and a foundation for many other frameworks.

Conclusion

This  project was a fun and enjoyable exercise and we hope the reader is equally entertained by this application.  We showed that geo-games can be easily created with Esri's JavaScript framework and cloud services together with common libraries like jQuery, bootstrap and Facebook.

Map Lens is a sample JavaScript-based web application that demonstrates lensing or perhaps a better description is "draggable map insets".  Lensing offers many benefits over traditional swiping which is commonly used to horizontally transition between two web maps, for example, before and after a hurricane imagery.  In comparison, lensing offers the user an unlimited number of rectangular map overlays, for example, one lens per Esri basemap as shown above.

Click here for the live application.

The source code is available on agol and github.

Lensing is achieved with the ArcGIS API for JavaScript and leverages jQuery and jQueryUI, specifically the draggable and resizeable methods.  The biggest challenge developing this sample was the handling (or suppressing) of the various mouse and navigation events.

Map lensing is not without its disadvantages.  Maps embedded in each lens cannot contain dynamic content, maps ideally should only contain tiled map or image service.  This is a performance consideration due to the number of navigation events invoked during lens interaction.

Image services are not only for serving imagery; they can also perform dynamic pixel-level analysis on multiple overlapping raster datasets using chained raster functions.  Image services deliver blazing fast performance with pre-processed source imagery, especially when it is served from a tile cache.  Dynamic image services, on the other hand, may not respond as quickly because of the additional processing demands they place on the server.  High performance is important for dynamic image services because the data is re-processed automatically each time the user pans or zooms the map.  Dynamic image services typically produce sub-second responses for simple types of analysis.  But more complex analysis may take much longer depending on the complexity of the processing chain and the number of input datasets involved.  How can we get better performance in those situations?  To answer that question I ran a series of tests to see how the following factors affect the performance of dynamic image services:

• Map scale and source data resolution

• Resampling method

• Source data format and compression

• Project on-the-fly
• Request size

In this article I present the results of those tests along with some suggestions for maximizing performance.  My testing machine is a desktop computer running Windows 7 SP1 and ArcGIS 10.2.1 with 18GB of RAM and a quad-core Intel Xeon W3550 processor running at 3 GHZ.  The test data was stored on an otherwise empty 2 TB SATA hard drive that I defragmented and consolidated prior to testing.   The tests were configured to determine the average response times of services under various conditions.  By “response time” I mean the time it takes a service to retrieve the source data, process it, and transmit an output image.  Transmission time was minimized by running the testing application directly on the server machine.

This information is written with the intermediate to advanced GIS user in mind.  I assume the reader has a general understanding of image services, raster data and analysis, raster functions, geoprocessing, mosaic datasets, map projections, and map service caching.

Map Scale and Source Data Resolution

The pixels that are processed for analysis by dynamic image services are generally not identical to the pixels stored in the source datasets.  Instead, the source data pixels are first resampled on-the-fly to a new size based on the current scale of the map.  This formula shows the relationship between map scale and resampling size when the map units of the data are meters:

Resampled pixel size = map scale * 0.0254/96

The resampled pixel size is analogous to the “Analysis Cell Size” parameter in the Geoprocessing Framework and is sometimes referred to as the “pixel size of the request”.  As you zoom out to smaller map scales, the resampled pixel size increases until eventually the service resamples from the pixels in the pyramids.  Resampling from pyramids helps to keep the performance of the service relatively consistent over a range of map scales.

Performance still varies depending on map scale and typically looks similar to chart 1.    I generated these results using an application configured to simulate a single user panning the map 100 times in succession at specific map scales.  The chart shows the average time the service took to process and transmit the output images for different map scales.  This particular service was configured with a raster function template to perform a binary overlay analysis on eleven overlapping rasters in a mosaic dataset.  The pixel sizes of the source datasets ranged from 91.67 to 100 meters.  The raster function template was configured to return a binary result, where each output pixel is classified as either “suitable” or “unsuitable” based on the analysis parameters.

Take a look at the three points along the horizontal axis where the response time drops abruptly.  At those map scales the resampled pixel size is the same as the pixel sizes of the pyramids in the source data.  The processing time for resampling is the lowest at those scales because there is nearly a 1:1 match between source data pixels and resampled pixels.  Client applications which use this particular service will see dramatically faster response times if they are limited somehow to only those scales.  One way to do this is to use a tiled basemap layer.  Web mapping applications which use tiled basemaps are generally limited to only those map scales.  The most commonly used tiling scheme is the ArcGIS Online/Bing Maps/Google Maps tiling scheme (referred to hereafter as the “AGOL tiling scheme” for brevity).  The red-dotted vertical lines in the chart indicate the map scales for levels 7 – 12 of this tiling scheme.  Unfortunately those scales are not very close to the scales where this service performs it’s best.  There are two options for aligning source data pixels and tiling scheme scales:

1. Build a custom basemap with a custom tiling scheme that matches the pixels sizes of the data.
2. Sample or resample the data to a pixel size that matches the tiling scheme of the basemap.

The horizontal axis in chart 2 represents the "pixel size of the request" rather than map scale as in chart 1.  The orange graph shows the response times of another service configured identically to the first one in blue, except it uses source datasets that were up-sampled to 38 meter pixels using the Resample geoprocessing tool.  Up-sampling to 38 meters aligned the service’s fastest response times with the AGOL tiling scheme scales, which resulted in a significant decrease in processing time at those scales from approximately 1.5 seconds to about 0.5 seconds.  Furthermore, notice that performance is improved at nearly all scales except for the very largest.  This is most likely due to having all the source data at the same resolution (38m) instead of three (91.67m, 92.5m, 100m), and/or because the source data pixels are also aligned between datasets (accomplished by defining a common origin point for each resampled raster using the “Snap Raster” environment setting).

Admittedly, using the Resample tool to prepare data for analysis is not ideal because it results in second-generation data that is less accurate than the original.  This may be perfectly acceptable for applications intended to provide an initial survey-level analysis;however, it’s best to generate new first-generation data at the desired pixel size whenever possible.  For example, if you have access to land-class polygons, you could use them to generate a new first-generation raster dataset at the desired pixel size using the Polygon to Raster tool, rather than resampling an existing land-class raster dataset.

To determine by how much performance improved with 38 meter pixels, I calculated the percentage change in average response times for each scale and averaged the values over multiple scales.

Up-sampling the source data to 38 meter pixels reduced response times by  63.8% at the poorest-performing -target map scales!  38 meters was not my only option in this example.  I could have chosen a size that corresponded to one of the other tiling scheme scales.  The following table lists all the map scales of the AGOL tiling scheme, and the corresponding pixel sizes in units of meters, feet and Decimal Degrees.   The three columns on the right provide suggested values for sampling raster data.  These suggestions are not set in stone.  It’s not necessary to sample your data to exactly these recommended sizes.  The key is to choose a size that is slightly smaller than one of the sizes of the target tiling scheme scales.

Map Scales and Pixel Sizes for the ArcGIS Online/Bing Maps/Google Maps tiling scheme

By the way, matching the pixel sizes of your data with a basemap tiling scheme is also useful for workflows that involve static imagery overlaid onto a tiled basemap.  For those cases, you can build mosaic dataset overviews for viewing at smaller scales instead of raster pyramids.  One of the great things about mosaic dataset overviews is that you can define the base pixel size of overviews as well as the scale factor to match your target tiling scheme.  This way you don't have resample the source data to a new base pixel size in order to cater to any particular tiling scheme.

Resampling Method

The resampling method specified for an image service request also has an impact on performance.  The choice of which one to use should be based primarily on the type of data used in the analysis.  Chart 3 shows the performance of the binary overlay analysis service (with 38 meter data) with different resampling methods.

Bilinear resampling is the default method.  Here is how the response times for the other methods compared to bilinear averaged over the five map scales tested:

Raster Format

The storage format of the data can have a huge impact on performance.  For example, the response time of the binary overlay analysis service averaged over all map scales was 36% lower when the data was stored in the GeoTIFF format versus file geodatabase managed raster.  The Data Sources and Formats section of the  Image Management guide book recommends leaving the data in its original format unless it is in one of the slower-performing formats such as ASCII.  GeoTIFF with internal tiles is the recommended choice for reformatting because it provides fast access to the pixels for rectangular areas that cover only a subset of the entire file.Pixel Type and Compression

The pixel type determines the precision of the values stored in the data and can have a huge impact on performance.  In general, integer types are faster than floating-point types, and lower-precision types are faster than higher-precision types.  Compression of imagery can potentially increase or reduce performance depending on the situation.   For more information about the affect of compression on file size refer to the Image Management guide book section on Data Sources and Formats.  To assess the impact of pixel type and compression on the performance of data stored on a local hard drive, I tested a group of image services configured to perform an extremely intensive overlay analysis on 15 raster datasets.  The services were configured identically except for the pixel and compression types of the analysis data.  The tests were run at the map scale corresponding to the pixel size of the data.

The following table shows the percentage change in response times with the reformatted datasets versus the original double-precision floating-point dataset.

On-the-fly Projection

On-the-fly projection is a very important feature of the ArcGIS platform.  It has saved GIS users like me countless hours of work by eliminating the need to ensure that every dataset in a map is stored in same coordinate system.  However, in some cases this flexibility and convenience may be costly when ultra-fast performance is required.   The following chart shows one of those cases.

Chart 8 shows the performance of a service which performs a weighted overlay analysis  on six datasets in an Albers projection.  The upper graph shows the performance when the output of the service is set to Web Mercator (Auxiliary Sphere).  The lower graph shows the performance when the output of the service is the same coordinate system as the data.  Performance without reprojection to Web Mercator improved by an average of 45% over all map scales.  This is a fairly extreme example.  The performance cost of reprojection is related to the mathematical complexity of the input and output projections.  Equal-area projections such as Albers are mathematically complex compared to cylindrical projections such as Mercator.  I have not run tests to prove this, but I expect that the performance cost of reprojection between two cylindrical projections such as UTM and Web Mercator would be less costly than seen in this example, and that a simple projection from geographic coordinates to Web Mercator would be even less costly.

To avoid on-the-fly projection you must ensure that all of your data is in the same coordinate system, including the basemap.  Most of the basemap services currently available from Esri on ArcGIS Online are in Web Mercator (auxiliary sphere).  So if you are going to use one of those basemaps, you would have to convert your data to the same coordinate system.  This can be an acceptable solution for some situations, but keep in mind that it results in second-generation data with less positional accuracy than the original source data.  Alternatively, you can create your own basemap in the same coordinate system as your data, and either publish it to an ArcGIS Server site or upload it to ArcGIS Online as a hosted map service.  If you take this approach, I recommend caching the basemap using a custom tiling scheme with scale levels that match the pixel sizes of your data.

Request Size

Request size is directly related to the size of the map window in the application and is specified in the REST API as the number of rows and columns of pixels in the output image.  To measure its impact on performance, I ran a series of tests at different request sizes on the weighted overlay analysis service that I used for the reprojection-on-the-fly tests.  I measured the average response times for request sizes ranging from 400x400 to 2200x2200, increasing at 100 pixel increments (e.g. 500x500, 600x600, etc…).  All of the tests were run at the map scale of 1:113386, which corresponds to the 30 meter pixel size of the source raster datasets.

Chart 9 shows that the throughput for this service levels off at a request size of approximately 1000x1000 pixels to about 1.5 – 1.6 MP/s.  Chart 10 shows that request size has a linear impact on performance.  This service is capable of providing sub-second response times for requests up to about 1,440,000 pixels, or a request size of 1200x1200.

Summary

Raster analysis can involve many stages of data processing and analysis.  Complex on-the-fly processing chains can place heavy processing loads on a server and contribute to sluggish performance.  Huge performance improvements can be achieved in some cases by pre-processing the data into a more efficient format for resampling and on-the-fly processing.

For applications which use tiled basemap layers, the greatest performance improvements are likely to be achieved by aligning the pixel sizes of the data with the scales of the basemap tiling scheme.  The section “Map Scales and Source Data Resolution” describes the theory behind this approach and provides a table with recommended pixel sizes for applications which use basemaps with the ArcGIS Online/Bing Maps/Google Maps tiling scheme.  Alternatively, developers can build basemaps with custom tiling schemes to align with the existing pixel sizes of the analysis data.

Another way to significantly reduce the processing load on a server in some cases is to avoid on-the-fly projection of the analysis data.  This is accomplished by ensuring that the basemap and the analysis data are in the same coordinate system.  The performance impact of on-the-fly projection varies depending on the input and output coordinate systems and is discussed in the section titled “On-the-fly Projection”.

The file format, pixel type, and compression type of the analysis data can also have a huge impact on performance.  GeoTIFF with internal tiles is recommended for situations where it’s necessary to re-format the data from a slower format.  Lower-precision pixel types give better performance than higher-precision types.  Pixel compression has the potential to either increase or decrease performance depending on the how the data is stored and accessed by the server.  These topics are discussed in the sections titled “Raster Format” and “Pixel Type and Compression”.

Client applications can also play a role in dynamic image service performance.  Service response times are the lowest when applications specify nearest neighbor resampling, followed by bilinear resampling.  And there is a direct relationship between service performance and the size of the map window in an application.  These topics are discussed in the sections titled “Resampling Method” and “Request Size”.

The source code is available here.

The landsat program recently celebrated its 40th birthday. Since the launch of the first satellite in 1972 the program has amassed more than 3,000,000 images.

The USGS has published this archive as a single ArcGIS Image Service called LandsatLook. The prototype described in this posting uses this service in an HTML5 web mapping application.

While the map view is perfect to identify an area of interest, it is not so useful for sorting through hundreds or thousands of overlapping images. This prototypes uses a control developed by LobsterPot called the PivotViewer to present a sortable collection of imagery. Using the map and pivotviewer together, the presenter in the video above was able to quickly find and download a recent cloud-free image of London.Contributed by Richie C.

This post discusses the release of a sample utility called Geometric Network Configuration Manager.  In some instances when it is necessary to temporarily remove a geometric network, this tool can recreate the geometric network from a backed definition file.

The add-in can be downloaded from here.

The source code is available here or here on github.

Configuration Manager has a very long lineage.  More than a decade ago the Prototype Lab published Geodatabase Designer for documenting geodatabases and exchanging schema.  Designer is now obsolete but fortunately much of its capabilities are now incorporated into ArcMap or other tools such as ArcGIS Diagrammer and XRay for ArcCatalog.

However the one feature from Designer that has yet to be replicated is the ability to save and restore geometric networks.  This useful if loading large amounts data or performing a schema change like switching a feature class from a simple edge to a complex edge.Prerequisites

• Esri ArcGIS Desktop 10.2


• Microsoft .NET Framework 4.5

Libraries Used

• AvalonDock


• Microsoft Ribbon for WPF October 2010

The following section will walkthrough the steps required to backup geometric network, remove it and then restore it.  The use case for this workflow could be for bulk data loading or transferring a geometric network from a test server to a production server.Walkthrough

Following the successful installation of the add-in.  Display the Geometric Network Tools toolbar and click the first button to launch the main dialog.  Drag and drop a geometric network into the configuration manager window.

The complete definition of the geometric network will be loaded into the dialog.  The four tabs below the ribbon make it is possible to review and, in some cases, modify classes, weights and connectivity rules.

The geometric network definition is current stored in memory and should really be saved to a file so that it can backed-up or restored at a later time.  Click the save or Save As button to export the definition to a file with a esriGeoNet extension.

Recreating a geometric network is just a matter of loading an esriGeoNet file and clicking Export.  The application will prompt the user for the name and location of the exported geometric network.  If the geometric network already exists, it will be overwritten.Known Issues:

• The dialog that appears when the Export button is click may be hidden by the Configuration Manager window.  Either minimize or move the window to the side to continue with the export operation.


• Add and removing of network classes and weights is currently not supported.

NOTE March, 2019:  An even newer and easier to use version of the this tool is now available for ArcGIS Pro and an updated blog post about using the tool has also been posted.

NOTE:  The Distributive Flow Lines tool was updated  March 1, 2017.  Please download the latest version.  The interface has changed a little since the original post so some of the description below will be a little off but the concepts are still very close so I have left the graphics and captions alone for now.

Based on comments and feedback from the first version of the Flow Map tool and blog post we gave it a second look and have released an update.  As you can see in the screen capture above, the output is similar to the original tool but we have switched to an all raster approach which provides more control over the lines and reduces processing time.

I have pasted the introductory text from Brad Simantel’s original post below to provide a concise introduction to the history and use of Flow Maps.

“Flow maps are used to show the movement of goods or people from one place to another. These maps use lines to symbolize the movement, often varied in width to represent the quantity of the flow, and fall into one of three categories: radial, network, and distributive. Radial flow maps are used to show relationships between one source and many destinations. If there are more than a handful of destinations and you still want to show the quantity of flow, however, the lines overlap too much to discern individual values. Network flow maps are used to show the quantity of flow over some existing network — transportation and communication networks being the most common. Distributive flow maps are similar to radial flow maps, but rather than having individual lines from the source to each destination, lines are joined together, only forking once they get close to their destinations”.

If you would like to try it out as you read this post, you can access the new  Distributive Flow Lines tool (DFLT) on ArcGIS Online. If you do not have data handy, the English Coal export data, used to create the first map in this post, is also available for download on ArcGIS Online.Tip: The DFLT requires the Distributed Quantity Field to be of type Integer.  If you use the British Coal sample data a new integer field will need to be added and calculated based on the existing CoalTonnage field.  The current field represents thousands of tons so multiplying by 10 and rounding to the nearest integer would represent hundreds of tons rather than thousands.

Like the first Flow Map tool, the Spatial Analyst extension is required to use the new DFLT.   In fact, the new tool is completely raster based until the end when the flow line feature class is created as output.

The DFLT will accept polygon or point features as input for the Source and Destinations, however, point features are recommended.   If polygons feature classes are used as input for either Source or Destinations they will be converted to points for internal use but these intermediate point datasets are not saved as part of the tool output.  After the Destination feature class is selected the user will need to select an integer field from this dataset representing the quantity that will flow to the destinations from the source.

The DFLT provides an option for the user to specify a feature class to use as impassable features.  These features will be buffered by 1.4 times the Cell Size parameter.  The area inside this buffer will be NoData and will be used as a processing mask within the tool.  A second, slightly larger buffer, 3 times the Cell Size parameter, is also created. The area between the inside buffer and this larger buffer is given a very high cost so Flow Direction will not calculate routes that go into the NoData area (most of the time).

The second optional parameter is a Impedance feature class.  These polygons represent  features in your map that you would like flow lines to avoid as much as possible.  How much these features are avoided is controlled by the Impedance Weight slider.  When impedance features are specified, they are given a high cost; but may still be crossed in extreme cases where there is no alternative or where they can be crossed in a narrow location as shown in the example below.Tip: One common effect of the DFLT is that it will often skirt Impedance features too close.  This can be avoided by buffering the Foreground features and then using this new feature class as the Foreground features in the tool.

Since the primary use case for the DFLT is to create pleasing flow lines, the user will most likely run it several times using different values for the weight sliders to achieve a good starting point for the effect they are looking for.  Using an all raster approach makes this process a bit easier and faster.  In raster processing, cell size has a major effect on processing time. The DFLT calculates a default cell that size works well for the final product.  In most cases it will not be necessary to use a smaller cell size than the default. We recommend using a much larger cell size during initial iterations to reduce processing time until you start getting results close to what you want.Tip: It is a good idea to make a note of the default cell size for final processing.  For initial runs of the tool,  multiplying the default cell size by 5 or 10  works well to get a feel for how the various weights and optional parameters effect the output flow lines.  More drastic changes in cell size may have a significant effect on how the lines are routed around Impassable and Foreground features.Tip: If you use the post 10.1 sp1 version of the tool you will notice the tool dialog has been simplified.  Source Weight and Destination Weight sliders have been replaced by a single slider to control how close to the Source or Destinations the flow lines branch.  Also processing extent options have been reduced to "Same as Display" and "Same as Destinations plus Source."  This was changed to prevent most of the cases where no solution is possible. 

Cell size is so important in the DFLT because the tool calculates Euclidean distances from source and from all of the destination features. These two distance rasters are then Sliced into an equal number of discrete cost classes.  The number of classes is determined by dividing the maximum processing extent dimension by the cell size.  This acts to normalize the effect of the Source and Destination cost rasters and provide a basis for the cost of the Impedance raster.  By normalizing these costs the tool provides the user more control over the shape of the final flow lines through the Source and Destination Weight sliders.

To understand how cell size affects the total cost surface it helps to see a few examples.  In the following ArcScene screenshots we are using a Source location in the center of the US and the destination features are the centroids of a several countries in Africa. The CostDistance surface is shown floating over the “flat Earth” to provide a means of comparing the effects of the weight parameters and cell size. In each example the perspective and Z exaggeration are the same.

In this last example, it was necessary to reduce the elevation exaggeration and to zoom out so the cost surface would display in a reasonable way.  The only difference between the two results is that the cell size has been reduced from 100,000 meters to just 30,000 meters.  The effect is that the number of “cost slices” is larger and so the CostDistance “elevation” is increased.

As previously stated, the new DFLT processes the Impassable features separate from Impedance features.  Impedance features are treated much the same as Impassable features in the first tool.  They are features the user would like the flow lines to avoid if possible but the flow lines will cross them if necessary to reach a destination location.  Impassable features represent a hard barrier in the cost surface and can be used to control the shape of the flow lines as in the example below.

As with the previous Flow Map tool, the output of this new tool will usually be the starting point to produce a final flow map with smoother more organic looking flow lines.  Once the lines are roughly where you want them the next step is to use Graduated Symbols to display the Distributed quantity.  The “GRID_CODE” field will contain values based on the input Distributed quantity field.  The values in this field will be equal to the Distributed quantity field for the smallest lines nearest to the destinations.  Each time a tributary flow line meets another line the combined line closer to the Source feature will have a GRID_CODE value equal to the sum of previous two tributaries. Temporarily placing labels on the lines and destination points makes this clear. In the examples above the lines are represented using 10 classes of line weights ranging from 1 – 10. Using rounded joints and ends on the lines also improves the output. After setting up graduated symbols many users may also prefer to run the Smooth Line tool on the flow lines. This will also add some curve to the flow lines which is often desirable. A good starting point is to use a tolerance value approximately four times the Cell Size used to create the flow lines. It is also recommended to select the PEAK algorithm and FIXED_CLOSED_ENDPOINT options. Even after these steps it may be necessary to make manual adjustments to the shape of the line.Installation instructions:

After downloading and unzipping the Distributive Flow Lines tool you should see the following in the ArcMap catalog window.

Expanding python toolbox will reveal one script tool as shown below:

By default, python toolboxes do not display the “pyt” file extension. To display this and other known extensions in ArcCatalog (or the catalog window in ArcMap) check the following:

This dialog is accessible in ArcCatalog by clicking Customize > ArcCatalog Options.  Unchecking the highlighted option will show the following:

We hope you find the new tool useful and the tips above help get you started experimenting with the tool.  We look forward to you comments.

NOTE:  The Distributive Flow Lines tool was updated  Oct 22, 2013 to address a backwards compatibility issue.  If you are using ArcGIS 10.1 please download the latest version  here.

NOTE:  The Distributive Flow Lines tool was updated  Jan 10, 2014.  If you are using ArcGIS 10.1sp1 or 10.2 please download the latest version.Contributed by Bob Gerlt.

After helping with the Urban Observatory exhibit at the User Conference, 2013 we were inspired to bring the same concept to mobile devices. This would provide for the same visualization and comparison of different themes across different cities around the world.

The Urban Observatory works in a grid format. All the cities to be compared would be in the bottom most row. Each had the same theme such as housing density, traffic etc,. and the map scale in each device was synchronized. As devices are stacked horizontally they would load a user defined city with the same theme and the same map scale as adjoining iPads.  Likewise, vertically stacked devices would display the same city but with a different theme as defined in that row.

We replicated this concept by using multiple iPads. As soon as we put down a new iPad they would realize their position with respect to the previous iPad and would pull in the correct data. All the devices synchronized the map scale and only the same city iPads synchronized the map extent. You can change the city on one device and all the other ones would update accordingly to show a different city. We also brought in the concept of using the iPhone as a remote control to manage the themes and the cities on all the iPads. The top row of iPads could also serve additional information about the cities. We had them show the fly by video of each city shown in the iPad below.

Technical Details :

This application was written using ArcGIS Runtime SDK for iOS.

All the transactions happened using the Gamekit framework for iOS. This helped us automatically setup a sessionID to which other iPads would connect to. Using CoreLocation, we also knew their relative positions and we could connect a lot of these devices together and they automatically knew what data to pull from ArcGIS Online. They established a peer to peer networking using the bluetooth or the internet when available.

The original content for the Urban Observatory, its goals, objectives, sponsors and contributors, could be accessed at http://www.urbanobservatory.org.

The Lab is pleased to announce the release of Raster Shader, a new addin for ArcMap 10.1!  Raster Shader is a tool for creating full-color, relief-shaded visualizations of single-band raster datasets.  Similar to the Image Analysis Window, Raster Shader leverages the power of raster functions to process the data dynamically as it is displayed in the map.  Since the processing occurs entirely in memory, it is extremely fast and does not affect the source datasets.

The idea for this tool came from projects we've done over the last couple of years involving large collections of single-band raster datasets containing climate data such as temperature averages and precipitation totals over time.  Our standard approach for visualizing these datasets is to use chained raster functions to apply a combination stretch/colormap renderer.  However, building and configuring raster function chains which produce an attractive and meaningful visual result can be a challenge if you are not familiar with the concepts and techniques involved.  This is where Raster Shader can help.  Not only does it build the raster function chain for you.  It also provides a simple interactive UI for adjusting key rendering parameters so you can see the results immediately in the map layer.  An interesting feature of Raster Shader is a histogram which shows how the colors in the colormap relate to the distribution of values in the data.  This relationship is based on stretch parameters which you can adjust with slider bars and/or text boxes.  Optionally, if the map contains a layer based on a DEM, you can add topographic context to the data with a hillshade effect and easily adjust parameters such as sun azimuth and sun angle.

Click here for the addin.