Applications Prototype Lab Blog - Page 4

What is it?

This is an add-in for ArcGIS Pro 1.4. It's a dockable pane that shows basic statistics and a histogram for any numeric attribute in your dataset.  It also offers a slider control for selecting and filtering map features on a range of those numeric values.

Why build this?

Quite some time ago, I built a healthcare-oriented Flex app. It let you choose two to four attributes and set a numeric filtering criterion on each, using the drop-down control.

Then the Summary Map button would produce a filtered map with the unioned results of those filters—like a logical "AND" operation. This facilitates exploration of your data set, helping you find, for example, places where diabetes rates are above average and also where uninsured rates are below average.

Now that Flex has come and gone, I wanted to take those capabilities—filtering with a specified range, combining filters together, viewing the data's histogram—to a new platform. Sporting an SDK for extensibility, ArcGIS Pro is Esri's premier desktop application, so I settled on an ArcGIS Pro add-in.

Some searching indicated that all this desired functionality is already available in different places in ArcGIS Pro. A Histogram chart shows the mean and median and a histogram of how the data is distributed throughout its range. There are some pretty neat features here that mirror chart selections on the map and vice-versa. If you create ranges for one or more attributes, you can use the resulting range slider bars to define, apply, and combine filter criteria.

But I wanted all this functionality all together in one place, available with a minimal number of mouse-clicks and dialogs; so I built an add-in to offer it in an ArcGIS Pro DockPane.

To use it

If you want to try it out, here's how:

1. Make sure you have ArcGIS Pro 1.4 or 1.4.1 installed.
2. Download the add-in file here.
3. Double-click the add-in file to install it.
4. Start ArcGIS Pro and open a project that has a map and a layer with numeric attributes. Or you can download a sample project here.
5. Click the "Query Helper" button in the add-ins tab.
6. Select a map, layer, and field; use the track bar control to choose a range of values in that attribute.
7. Click "Add Clause" and then "Apply Query" to filter the features on the map.
8. You can add more fields and ranges to the SQL and apply them for an additive effect.
9. You can also edit the SQL where clause in the text box if you want to explore your data more directly; if you enter an invalid SQL clause, the map result will be blank--just click Clear SQL and start again.

Challenges and lessons learned while building it

To get started, I tried using the ArcGIS Pro DockPane template for Visual Studio 2015. That created an empty DockPane and button, but didn't give me the guidance I needed to work with the MVVM pattern built into the template. (You may also want to see this Microsoft document on the MVVM pattern.) So I ended up extending an existing Esri community sample that loads maps and bookmarks into a DockPane.

I ran into a few instructive challenges in the process, mostly involving helper libraries and controls, but also matters of display and output.

Math.net

There are lots of statistics libraries available for .NET. I was looking for something simple, reliable, and free. Some research led me to the Math.net package, which I ended up using for statistical computation and creating histogram ranks.

Chart control

Since there's no chart control bundled into .NET, I had to look for one—again, it needed to be freely available. Though documentation is minimal, I went with the WPF Extended Toolkit (available through Visual Studio via the NuGet package system). Discussion of the challenges of using this charting system could fill another blog post or several. I'll just say that Google is your friend here.

Decimal point precision

This add-in needs to work with all sorts of numbers, including floating point values with too many decimal places to display in the available on-screen space. I opted to use StringFormats for limiting displayed decimal places to three (although computations are still done on the original, unrounded values).

XAML design mode

I also needed a way to hide the statistics field and histogram when an attribute field wasn't yet chosen and there was no data to display. I did this with a ValueConverter. The suggested way to declare ValueConverters is as static resources in the XAML file; unfortunately, this led to another problem I still don't have a proper fix for. The resource declarations seem to confuse the visual editor into displaying only an error message. My workaround was to remove the static resource declarations for those rare occasions I wanted to use the design view; then pasting those declarations back in before building the project.

Next steps

There's still more to be done. Now that I have a basic level of functionality, I want to add the different range-specification options available in the original Health Indicators Flex app: things like top/bottom X% and choosing a given range of percentages.

Resources

Add-in
http://apl.maps.arcgis.com/home/item.html?id=3f7b7d203b164304989adb88ebf88ed0

Sample data
http://apl.maps.arcgis.com/home/item.html?id=8c14e8d211ee4601a2f085aa1d8094de

Source (GitHub)
https://github.com/markdeaton/pro-addin-field-stats-helper-cs

 

Welcome to the new Applications Prototype Lab GeoNet group!  Our blog, discussions and other updates are now hosted on the GeoNet Community for better integration with industry, product and developer forums.  For your convenience in the future many of our more popular historic blog postings will be ported to, and redirected to, our new GeoNet home.

A little bit about us.  We are small group of twelve geospatial professionals engaged in pre-sales activities, corporate assignments and applied research and development.  We are based at Esri's headquarters in Redlands, California.

Left to right, front to back: Lenny K., John Grayson, Bob Gerlt, David Johnson, Carol Sousa, Richie Carmichael, Al Pascual, Mark Smith (Manager), Witold Fraczek, Mark Deaton, Thomas Emge and Hugh Keegan.

Please be sure to click "follow" in the top right corner of the overview page to be alerted to new announcements, apps, snippets or postings.

If you're new to GeoNet, we also encourage you to check out the GeoNet Help group for tips and FAQs on how to get started and get the most out of your community experience.

Thanks for joining us and we look forward to seeing your contributions!

In my previous post about the Combined Field Statistics addin for ArcMap I mentioned that it was inspired from another addin toolkit I am currently developing which analyzes and displays rates of disease occurrence in a map.  In this post I want to share with you another addin project which was borne out of the disease mapping project.  This one is called Diverging Color Ramp.  This addin enables you to intelligently apply a dichromatic color ramp to feature layers containing points, lines, or polygons in ArcMap 10.4 and above. It is somewhat similar in capability to the smart mapping tools in ArcGIS Online and is useful if you need to display data with values that diverge from a specific threshold value which you define.  The screenshot at the top of this post shows a map layer of population density where the colors diverge from an arbitrarily defined threshold value of 40,000/square mile.   Other examples of threshold values might include 1.0 for ratios, 0.0 for values which represent gain or loss from an average value, or any value which represents a significant threshold of interest in your data.

This tool gives you a great deal of control over how a feature layer is rendered.  It provides a pair of default color ramps, one for for values above the threshold and another for values below the threshold, as well as a separate threshold color symbol.  If you don't like the colors you can change them to anything you like by accessing them from the Style Manager in ArcMap.  You can also choose the classification method and number of classes, and even choose the number classes which are generated for values above and below the threshold.  This addin also has a "secret" trick up it's sleeve.  When it generates the symbols for each class in the map layer's renderer, it actually makes a clone of the symbol for the first class in the current renderer to use as a template for the rest of the symbols.  After that is simply sets the background color of each symbols  to colors pulled from the color ramps.  This makes it easy to quickly apply global changes to other properties in the symbols, such as outline color or fill pattern, since you only have to make the change in the first symbol and then apply it as a template to each of the symbol classes with a single button click.  This is a fun tool to use since it makes it easy to try out different ways to visualize your data.  I hope you enjoy using it.

Download links:Diverging Color Ramp (addin + documentation)Diverging Color Ramp (source code + documentation)

I'm currently working on a new add-in toolkit for ArcMap 10.4 to analyze and display rates of disease occurrence in a map.  This is a work-in-progress and I will provide more details about it in the coming months.  But in the meantime I want to share with you the result of a smaller project that was borne out of the disease mapping project.

One of the requirements for the disease mapping project is the ability to calculate the sums of values from age group columns in a feature layer.  Generally, the tool of choice for a task like this is the Summary Statistics geoprocessing tool in ArcToolBox.  However, I also needed the ability to calculate sums from the combined values in multiple age group columns.  For example, to calculate the total number of individuals in columns for ages 0 -4 and 5 - 9, to obtain an aggregate total for ages 0 - 9.

It occurred to me that this capability could have broader applications if the tool could also calculate the full set of statistics provided by the Summary Statistics tool.  So I decided to build a more generic stand-alone addin tool in parallel with the disease mapping toolkit.  The result of this effort is the Combined Field Statistics Addin.  This tool is very simple to use and includes a detailed user guide to get you started.  It also generates an output log so you can keep track of how the tool was configured to generate a particular output.  If this capability sounds useful to you give it a try and let me know what you think in the comments!

Download links:Combined Field Statistics (addin + documentation)Combined Field Statistics (source code + documentation)

The source code for Dimension Explorer, an add-in for exploring time-aware and multidimensional data, is now available for download.  Version 1.11 is a minor update from version 1.1 and is built specifically for ArcMap 10.4.  This version features tighter integration with the ArcMap Table of Contents window - as layers are added, removed or changed, the  contents of the layer selection list in the Settings window are updated accordingly.

Update - I've had reports that the layer selection list is not getting updated consistently as layers are added, removed, or had their properties updated in the map document.  Until this issue is resolved, I recommend users of ArcMap 10.3.1 and 10.4 use version 1.1 of Dimension Explorer instead.  I apologize for any inconvenience this may have caused.

Download links:

Dimension Explorer 1.11 (addin and documentation)

Dimension Explorer 1.11 (source code and documentation)

Dimension Explorer 1.1 (original blog post)

Note:  The 3D Fences Toolbox was updated Feb 28, 2017.  

This article introduces and discusses a geoprocessing toolbox that can perform geostatistics on vertical slices of three dimensional point clouds.Click here to download the toolbox.

For years the users of ArcGIS and its Geostatistical Analyst extension have been able to perform sophisticated geostatistical interpolation of the data samples in two dimensions. With that, the creation of continuous maps of any phenomenon that was measured only at selected locations was feasible. With the growing popularity of analysis in 3D and the availability of 3D data, the necessity of an option to perform geostatistics on 3D data is in demand. Studies of atmospheric cross-sections, geologic profiles, and bathymetric transects became an integral part of GIS. In each one of these three classical elements (air, earth and water), we can measure natural phenomena, such as the gradient of air temperature, the content of an ore at various depth of the geologic strata, or the salinity of the ocean along adjacent vertical transect lines.

In GIS we analyze not only the natural phenomena, but also those that are man-made. Human impacts on air, earth, and water are also measured. Among those human contributions to the environment that could be analyzed in 3D are plumes of air pollution and oil spills. The later may migrate down in the ground and be moved by the ground water drift, or migrate from the bottom of an ocean up to its surface while sometimes being dragged by ocean currents. Having an insight into such occurrences can greatly increase our understanding of the phenomena. This was the motivation for creating the 3DFences Toolbox.

A slice is a vertical subset of the 3D data analogous to a slice of bread from a loaf. While typically narrow in one dimension it still is a 3D object. A fence is a 2D representation of a slice of 3D data. All points which belong to a slice are projected, or pressed onto a 2D plane. The term of a fence diagram, or a fence, is used in geology to illustrate a cross section of geologic strata generated from an interpolation of the data coming from a linear array of vertical drillings. The equivalent of a fence in the atmospheric sciences is usually referred to as a curtain.

To be explicit, the tools in the 3D Fences toolbox do not perform 3D interpolation. The approach offered by the 3DFences Toolbox does not implement geostatistical analysis directly on the vertical slices of the 3D data. Instead, the tools transform a slice of the 3D data, with its X, Y, Z, and the measure of a phenomenon component by rotating it by 90° to a horizontal 2D plane. The geostatistical interpolation method of Empirical Bayesian Kriging (EBK) is performed on these points producing either the geostatistical surface of Prediction, which is a continuous map of the concentration or intensity of something, or a map of the Prediction Standard Error, which can be explained as a map of a degree of confidence in the Prediction map at each location of the map. The resulting output is converted to a point dataset where the points represent the interpolated value at the center of the raster cells. The points are then placed back into the original coordinate space as a regular matrix of points resembling a fence. The fence is positioned in the center of the selected points of the initial slice, when displayed in ArcScene or ArcGIS Pro. The raster is converted to a point dataset because ArcGIS does not currently support display of raster data as a vertical plane and point symbology options provide added flexibility in displaying results.

Any of the ArcGIS standard or geostatistical point interpolation tools could have been implemented in the tools. For the prototype, we have chosen the Empirical Bayesian Kriging (EBK) geostatistical method known for its best fitting default parameters and accurate predictions.

The 3DFences toolbox consists of three separate tools to support different methods of generating fences. The Parallel Fences tool can generate sets of parallel fences in the directions that are related to either longitudes, latitudes or depths.  In other words, the output sets of parallel fences stretch from N to S, W to E, or through the Z dimension.  The number of these fences in each set is determined by the user. All of the tools support selection sets to create fences from a subset of sample point features.

The Interactive Fences tool can generate fences based on lines digitized on the map. The user sets the buffer distance from the digitized line. All points that are located within the buffer will be used for the geostatistical analysis. The user may digitize multiple lines and even self-intersecting lines with many vertices. The third tool, called Feature based Fences, creates fences based on existing features in a polyline feature. In this case the fence shape is determined by the existing feature(s) and extends through the Z dimension of the selected sample points. As an example, it might be applied to investigate for oil leaks above an oil pipeline placed on the sea floor.

All of the tools contain options enabling the user to determine the minimum number of sample points and fence size required to generate a reasonable geostatistical surface. The tools are also time aware. If the sample data contains a date-time field and the option is enabled, a fence will be generated for each time interval if the samples for that interval and location meet the minimum requirements set by the user. The resulting fences representing consecutive time windows are positioned at the same locations. Thus, to enable better visual analysis, these should be displayed as time animations.Contributed by Bob G. and Witold F.

A new version of Dimension Explorer is now available for download.  Dimension Explorer 1.1 is an addin tool for exploring time-aware and multidimensional map data in ArcMap 10.3 and above.

Here is what's new in version 1.1:

• map layers created with the Make NetCDF Raster Layer and Make NetCDF Feature Layer geoprocessing tools are now supported.

• map layers with vertical dimension values defined as ranges (e.g. 0-10 meters, 5-50 meters, etc) are now supported.

• export a map animation in ArcMap to a series of still images to create video animations with applications such as Windows Movie Maker and ffmpeg.

• various bug fixes and optimizations.

Here is a video animation of the minimum Arctic sea ice extent for the years 1979 - 2014.  I created it with Windows Movie Maker using still images exported via Dimension Explorer 1.1.  The map includes a time-aware layer created with the Make NetCDF Feature Layer geoprocessing tool with data from the NOAA.

Dimension Explorer 1.1 can be downloaded here

If you are looking for data to get started with Dimension Explorer,  the NOAA Earth System Research Laboratory, Physical Sciences Division, (NOAA/ESRL PSD) has many large collections of spatial scientific data for climate and oceans in NetCDF format.  I recommend starting with their gridded climate datasets.   You can add most of their datasets to ArcMap using the Make NetCDF Raster Layer geoprocessing tool.  If you get the error "One or both dimensions have variable spacing in their coordinates", use the Make NetCDF Feature Layer geoprocessing tool instead.  If the datasets are stored as multiple files representing the data at different times, use the Mosaic Dataset to temporally aggregate the files using the NetCDF Raster Type.  Finally, if you are working with temporal data of any type, be sure to time-enable the layer in ArcMap.

Ürümqi in China is the remotest location on Earth, geographically speaking of course. This discovery and the analysis behind it are discussed in a recently published story map entitled Poles of Inaccessibility.

Vilhjalmur Stefansson, an Icelandic explorer, introduced the world to the concept of inaccessibility with his 1920's computation of the Arctic's pole of inaccessible. Story map authors Dr Witold Frączek and Mr Lenny Kneller recomputed this Arctic location and inaccessible locations in six continents. Frączek and Kneller computed and compared remote locations using geodesic and planar computations. Differences were small with the exception of the Eurasian continent which has a variation of approximately 11 kilometers.

The authors discovered that South America was essentially bi-polar with respect to inaccessibility. While both locations are located in Brazil, and separated by 1,400 km, the difference in inaccessibility was less than a kilometer. It is conceivable that the order of remoteness may change with the interpolation and generalization of the coastline.

Lastly, I would encourage you to take a moment to read and view Frączek and Kneller's Poles of Inaccessibility.

Update - a more recent version of Dimension Explorer is now available.  Click here for more information.

Dimension Explorer, an addin tool for ArcMap, has just been released by the Esri Applications Prototype Lab!

Dimension Explorer 1.0 makes it easier to work with  time-aware and multidimensional data in ArcMap 10.3 by providing slider controls for navigation.  It works by retrieving dimensional information from a map layer to build an interactive dimensional model that can be edited and saved in the map document.  Dimension Explorer is the successor to the Timeliner addin for ArcMap, which also works in ArcMap 10.3 and can be downloaded here.

Click here to download Dimension Explorer

With the 10.3 release of ArcGIS, the mosaic dataset now supports multidimensional data in NetCDF, GRIB, and HDF format.  Dimension Explorer supports map layers based on mosaic datasets and image services which are time-aware or multidimensional, and time-aware feature classes.

Download Dimension Explorer and let us know what you think of it in the comments!

Hydro Hierarchy is an experimental web application for visualizing the US river network.

Click here to view the live application.

Source code is available on agol and github.

There are approximately a quarter of a million rivers in the United States, but only 2,500 are displayed in this application.  This subset represents streams with a Strahler stream order classification of four or greater.  The stream data used by this application is derived from the USGS's National Hydrographic Dataset and has undergone significant spatial editing.  Streams geometries have been adjusted to ensure connectivity, generalized for small scale mapping and re-oriented in the direction of flow.

River flow data was acquired from the USGS's WaterWatch website.  Each river segment is attributed with the average flow for each month in 2014 and the ten year monthly average.  Computed values, in cubic feet per second, represent the flow at the downstream end of each river.  Flow data is displayed as a column chart on the left hand side of the browser window whenever the user's mouse passes over a stream.

The preview animated image at the beginning of this post may look sped up.  It is not.  Upstream and downstream rivers are highlighted in real time as the user moves his or her cursor over the hydrologic network.  This performance is achieved using connectivity information loaded when the application first starts from this file.  The file was creating in ArcMap from a network dataset that included the river feature class.  Using this script, connectivity information for each network node was extracted and arranged into a hierarchical data structure.

The radial and column charts on the left hand side of the application are generated using the D3 graphics library.  The column chart displays 2014 flow data for any river segment that is directly below the user's mouse.  The horizontal red line represents the ten year mean monthly flow.  Note that for most river segments, only one or two months ever exceeded the ten year average.  This is indicative of 2014's drought, at least with respect to river flows over the past decade.Contributed by Richie C. and Witold F.