Have you or your organization developed customizations for ArcGIS Desktop with the ArcObjects SDK, or developed add-ins for ArcGIS Pro with the Pro SDK? We welcome your feedback in the new ArcGIS Desktop Development Survey, which is now available online.
There is good documentation in the Esri Resources on how to customise your ArcGIS Pro ribbon so a user can add the tools they use most into the Pro ribbons for easy access, and remove ones they don't use.
Where I have found it harder to interpret the existing help documentation, is identifying what ribbon configuration is included as part of a shared Project Package.
A project package is a file that contains all maps and the data referenced by its layers, as well as folder connections, toolboxes, geoprocessing history, and attachments. Project packages can be used for sharing projects between colleagues in a work group, across departments in an organization, or with any other ArcGIS users through ArcGIS Online or ArcGIS Enterprise. Share a project package—ArcGIS Pro | Documentation
I did some testing to identify what customisation of the ribbon i could do, that would be shared as part of a project package. The answer is that most of the ribbon customisation is cached locally, and not included in a project package. The purpose is for a user to be able to customise their setup to more greatly enable themselves to work.
This does present as a limitation if an organisation wants to influence editing workflows by making tested GP tools available in the ribbon, and removing tools that they dont wish users to use (this could be due to customisation or other factors).
One configuration of the ribbon that did successfully get included in a published project package, was the configuration i did in the Customize Analysis Gallery. This is accessible in the Analysis tab, in the Tools group and displays as a window of tools you can scroll through.
I haven't tested all options, and Id be interested in seeing what other ribbon configurations people have played around with that publish as part of a project package.
But in summary, for those skipping through the content - use the Analysis Gallery to set tools you want to be saved as part of a project package#
To add a new Voxel Layer Map tab-> Add Data -> Add Multidimensional Voxel Layer
Files that work for voxel layer – netCDF will dimensions 3 dimensions and time or netCDF with 2 dimensions and time
Auxilliary variables do not work for voxel layer (variables where the dimensions of the voxel is defined using a proxy dimension such as trajectory instead of directly referencing realworld dimensions)
To change variables, there are two ways:
Go to Appearance tab which only appears when you select Voxel Layer -> go to Variable group
Right Click on Layer -> Select Symbology -> Use the Variable dropdown
To create isosurfaces and sections, select Section radio button :
For isosurfaces -> Right click on Isosurfaces -> Create Isosurface (upto 4).
To Edit Isosurface, right click on isosurface -> Edit -> Voxel Exploration pane should open that you can use to edit the value and color of the isosurface
For Sections -> Right Click on Sections -> Create Section -> Use the toolbar to create horizontal section with one click and vertical sections with 2 clicks
To Edit Section, right click on section -> Edit -> Voxel Exploration pane and the toolbar should open that you can use to edit the position of the section.
To lock the section, right click on section -> Lock section. You cannot move or change the variable of a locked section
If your variable is time enabled, a time slider will show. Use the time slider to animate over different timestamps
Use the popup for more information on specific voxels
Sentinel-5P is a satellite that is operating within ESA's Copernicus program since 2017. The goal of the mission is a very dense scheduled operational monitoring of the atmosphere. Using the TROPOMI instrument on board, various parameters are measured, such as tropospheric pollutants, greenhouse gases, aerosols, and cloud cover. NO2 concentrations are also provided as a separate product, which can be downloaded as NetCDF. More information about this product can be found in the ESA user manual.
Figure 1: Data structure of the Sentinel 5P products
How can NO2 concentrations of the atmosphere be displayed in ArcGIS Pro and how can a time series be created?
This article answers these and other questions about the download and use of remote sensing data in R and ArcGIS Pro.
The analysis consists of two parts: The first part contains the data acquisition and data processing in R, the second the representation and visualization within ModelBuilder (Python) in ArcGIS Pro. Thereby the focus is on automation.
Figure 2: Workflow to create an automated time series analysis based on Sentinel-5P data
The figure above shows the schematic workflow where an R script is used to download Sentinel-5P data from the Copernicus-Hub, extract the corresponding NO2 product variable and export the scenes (filtered and clipped) as GeoTiff.
Afterwards a mosaic dataset is created in ArcGIS Pro, which contains the resulting NO2 GeoTiff files. The Mosaic can be published as Image Service.
Figure 3: Workflow R script
With the help of the R package GetSpatialData various satellite data available on the Copernicus Open Access Hub can be downloaded automatically. Conditions like recording period, satellite, product type and area of interest can be defined. NO2 data is stored in the Sentinel-5P product type L2__NO2___, therefore only this data is of interest and should be downloaded for the desired period.
The subsequently downloaded NetCDF files contain several variables, such as longitude and latitude, different viewing angels and various nitrogen dioxide related variables. Product variable 6 (nitrogendioxide_tropospheric_column) displays the NO2 concentration in the troposphere, which is used for the following analysis. Using the S5Processor package, this variable can be queried and saved as GeoTiff. In addition, cloud masking is applied to prevent outliers. Product variable qa_value indicates the error probability of the measured values (e.g. due to cloud cover or snow). By blending the two variables and the resulting elimination of errors (qa_value < 0.5 = NoData) a corrected nitrogen dioxide layer is obtained (unit molecules/cm²).
Why does it make sense to calculate average values per pixel over five or more days?
The averaging of NO2 concentrations over a 5-day period eliminates different effects:
NoData values due to clouds (masked by qa-variable)
NoData values due to incorrect overlap areas of the satellite scenes (masked by qa variable)
Wind influencing NO2 drifts in or from neighboring areas
When exporting the NO2 values as GeoTiff it is important to save the date of the scene or rolling window as part of the name.
Figure 4: Workflow ModelBuilder (export as Python script)
After all data has been transformed and edited, the workflow is continued in ArcGIS Pro.
ArcGIS Pro makes it easy to create an automated workflow within the ModelBuilder that performs spatial analysis using the appropriate geoprocessing tools.
In this case a mosaic dataset is created and enriched daily with the newly exported NO2 GeoTiff files. The fact that the raster data is dated means that it can automatically be arranged chronologically. With latest addition to ArcGIS Pro (2.5 and later) this Mosaic can be declared a Multidimensional dataset and thus allow the Multidimensional tools (aggregation, change detection, trending, etc.) to be applied.
The mosaic dataset can also be published as an image service using ArcGIS Server.
Once the scripts are set up, they can be run automatically every day, for example, using the Windows Task Scheduler.
But how are the result layers supposed to be interpreted?
This article focuses on the technology used and not on the scientific interpretation of the results. Nevertheless:
Values provide information about NO2 concentrations in the troposphere (up to 15 km above the earth's surface), but not at a specific location near the ground (e.g. a busy road)
NO2 concentrations are based on different influencing factors: Winds influence the airflow of NO2 from or in neighboring countries; cloud cover falsifies measured values and must be masked; low temperatures prevent the air masses from rising and thus also the rise of NO2; thunderstorms: lightning as a source of nitrogen oxides
Averaging over a certain period bypasses NoData values and eliminates meteorological effects that occur selectively
A detailed StoryMap of Esri Germany with illustrations and the Sentinel-5P image service can be found here (only in German).
The R script can be downloaded on GitHub. Günter Dörffel of Esri Global Inc. has integrated the steps of this workflow in a toolbox using R Bridge and native arcpy. The toolbox does not do the data aggregation but relies on the ArcGIS Pro capabilities of flexible aggregation (for days, weeks, months, …) through the Multidimensional Toolset in ArcGIS Pro. The toolbox and its scripts are also available via the link.
To make sure both the R script and the toolbox run without problems, the following packages must be installed previously in R: devtools, getSpatialData, raster, sf, sp, dismo, geosphere, rgeos, S5Processor, ncdf4, ggplot2, maptools, rgdal, lubridate, cat. For the toolbox, a working version of the R-bridge needs to be available.
ArcGIS Pro 2.6 brings some new capabilities for working with geopackages. Here I take a look at some basic functions and try out editing gpkg layers as well as adding rasters and shapefiles to a gpkg. For the most part things seem to work and there's a workaround for using the Raster to Geopackage tool.
If you're working with gpkg in ArcGIS Pro and have any tips or suggestions please let me know.
No matter how you are feeling about the situation unfolding across the globe, or how it has already impacted you or your organization, I think one thing that rings true for all is that how we approach our work (and lives) has likely changed already. For some this approach has been dramatic.
The global response to COVID-19 while varied, has highlighted how GIS technology is being used to understand, chronicle and assist. This could be for medical response efforts, planning or simple awareness. During this pandemic, GIS has already been a commanding tool for inquiry, analysis and mapping. Some of the applications you may have already seen that merge science, global health and GIS are the Johns Hopkins and the World Health Organization dashboards.
As this emerging threat evolves GIS can be leveraged in many ways to understand the spread of this infectious disease, the populations most at risk and the overall impact to our communities. Where to start? What can we do foundationally with the added challenge of social distancing? To help answer these questions we will look at picks to help your organization with foundational work as many of our social dynamics change. Topics include virtualization, an example of analysis done to assess populations at risk, and general resources to further assist in responding during this critical time.
Virtualization of ArcGIS from the Cloud and On-Premise platforms to support Higher Education
Most educational institutions have virtualized their classes to this point. When it comes to GIS classes, there are several considerations for virtualizing ArcGIS Pro and even ArcMap. If you are looking at how to expand your virtualized offering, virtualization experts at esri have written a detailed blog to assist in this effort. They outline the current options covering on-premise and cloud solutions, requirements for virtualizing Pro, best practices and the latest online documentation.
Age and Social Vulnerability in the Context of Coronavirus
John Nelson has written a blog showing how to do an analysis of populations most at risk in the U.S. The purpose of this analysis is to ultimately illustrate which communities in America may have healthcare services and infrastructure that could become overextended in the coming months. He walks through the data he used, where to get it and how to work with the data before publishing as a highly informative StoryMap. He also provides links to work with and understand the social vulnerability index.
Resources to assist your efforts
Over the past couple of weeks, I have been bookmarking various resources Esri has put out in response to the pandemic. These are applicable to not just a singular product or phase of analysis. They are out there to be used whether you are mapping your data in a desktop product or publishing and developing maps and apps.
The first is the COVID-19 GIS Hub. This is the central repository if you will and has news and updates, example apps, best practices and authoritative data you can work with. You can even ask for GIS assistance. Esri recently released a Coronavirus Response Solution, which is a collection of maps and apps that can be used by Public Health agencies to understand the impact of the Coronavirus (COVID-19) and share authoritative information about the pandemic in your communities. This can be deployed with the ArcGIS Solutions Deployment Tool. You can also catch this webinar that discusses how to use the solution today, Thursday, March 19th at 11:00 AM Pacific time. This last blog is relevant to any GIS division and provides a number of additional resources including feature layers you can tap into for your COVID-19 mapping needs.
I sincerely hope these resources are useful in any pandemic related efforts you are involved with. I wish you and yours the best during these challenging times. Thanks for reading and stay tuned for future picks. If you are interested please also check out This Week's Picks - ArcGIS Online and ArcGIS Enterprise by my colleagues.
Not being a regular user of pandas/numpy I find using such libraries difficult as I cannot visualise what I'm working with... call me old skool...
I recently came across dtale a rather cool python module that displays the data and allows you to manipulate it as if it was a spreadsheet. It also has a set of rather impressive methods for charting your data.
I immediately thought it would be great to use this inside ArcPro, using the notebook capability built right into ArcPro 2.5!
If you have not already done so you need to upgrade to ArcPro 2.5 as this version supports notebooks.
First of all you need to clone your environment, I talk you through this process on this blog page whilst setting up spyder to work with ArcPro.
Cloning creates a copy with all it's dependencies in a less than obvious place and it is here you need to install d-tale. So having cloned the environment fire up the Windows command line and make sure you open it in Administrator mode:
where XXX is your user name and YYY is the cloned environment folder name
Finally in the Scripts folder, type
pip install --upgrade dtale
This will install d-tale into the cloned environment and will be accessible within ArcPro next time you open it.
So here is some sample code I then type into notebook in ArcPro, it takes a layer loaded in the map and creates a dataframe from 2 numeric fields, when you execute `d` a URL pops up and you click on it to see your data as a spreadsheet
import dtale import pandas as pd import arcpy np = arcpy.da.FeatureClassToNumPyArray("EA_Sample_2020",["LOC_NO","Z020_LOC_T"]) df = pd.DataFrame(np) d = dtale.show(df) d
So in the window I create a new field called acs and its the SUM of the previous two fields
To get this new an improved data back into a pandas data frame you can type the following code into notebook then do something with it.
Welcome to another edition of This Week’s Picks – ArcGIS Pro! In my day-to-day I often browse GeoNet and other areas where product discussions occur to get a sense for what’s coming up with the product, and after spending a fair amount of time with 2.5 there were some areas I wanted to explore further based on past questions and support cases. Fortunately, there were some resources recently published that cover these topics that I wanted to share in case you hadn’t had a chance to explore them yet.
This week we will look at the following:
Customizing your layout gallery in 2.5
Some of the improvements made to tables at 2.5
Creating and adding Python notebooks to Pro (yep, at 2.5)
Without further ado…
Customize the layout gallery
First up, customizing the layout gallery. In working with this release early on I got a chance to check out some of the layout changes coming in 2.5 and I was very excited about the ability to customize my layout gallery and have a preview before adding these (layout files in the .pagx format). Obviously not every layout is suitable for every application and building custom layouts and putting them in a gallery allows for making minor tweaks without building new layouts for every project. Aubri’s blog takes you step-by-step on how to access and customize the gallery for your own purposes, new at 2.5. Check it out right here!
Pro Tip: Aubri points out that the layout names in the gallery don’t necessarily align with the file names (or filenames if you prefer). This is because the title is read directly from the metadata, so you need to edit it there to update it (along with thumbnail, etc.):
As you may have read in the ArcGIS Ideas implemented at 2.5 video, several improvements with tables originated as community contributed ideas. Find and replace for example is one that has been in high demand. The Map Exploration team also worked on several other usability and productivity improvements that you can take advantage of. For example, default positions for attribute tables is now configurable and freezing columns is also possible. One more that I have been using heavily is configuring pop-ups using a raster field in a table. Check out the blog here which includes documentation links for each of these new areas.
Create and add Python notebooks in ArcGIS Pro
The last pick this week covers the new built-in integration with ArcGIS Notebooks that allows you to both create and edit Jupyter notebooks right inside ArcGIS Pro. This increasingly popular open-source tool has been common in the Python data science community and can now be used alongside your other project elements to stay systematized and do things like visualizing Pandas Dataframes or prototyping workflows. This blog will take you step-by-step on how to create a new notebook, import an existing notebook and launch a notebook all within Pro! Give it a look through over here.
That wraps up issue #9 of This Week’s Picks – ArcGIS Pro. I hope you found these resources useful and thanks for reading! As usual, stay tuned for future picks and if you are interested please also check out This Week's Picks - ArcGIS Online and ArcGIS Enterprise by my colleagues. Thanks again for reading and happy mapping!
I’ve been using ArcGIS Pro for several years now and strangely, I only recently learned about a quite cool capability of creating symbology based on colors that are stored as attributes (i.e. either as RGB values, HEX color codes or CMYK).
After spending an hour trawling through the Geonet Forums I saw a few threads where people were discussing this, however I was unable to find a defined process for setting this symbology up. The documentation doesn’t seem to contain it either, so I decided to quickly summarize the workflow in a quick blog post.
So, the situation that we’re going to be looking at is this:
A local council have their zoning polygons imported from MapInfo and they have three attribute fields (as below) – each storing a color (either Red, Green or Blue). Each zoning code must have a specific color used in the map to depict it, so we’re going to use these attributes to create symbology in ArcGIS Pro 2.5.
Step 1: Firstly, we need to create a new text (STRING) field:
Add a field in the Fields Designer or use a geoprocessing tool Add Field.
If you’re using the Field Designer in Pro, don’t forget to save your changes:
Step 3: Now, the interesting stuff. Setup the attribute-driven symbology
Highlight the layer under Contents (1 in the graphic that follows)
Click Symbology (2)
The Symbology Pane will open (3)
Enable the attribute-driven symbology (based on the combined RGB values). Unless you have a subtype field defined in the feature class, your starting point will be something similar to this:
Symbolise with Unique Value, remove all of the symbols listed
Check the Allow symbol property connections option under the Vary symbology by attribute tab
Return to the Primary Symbology tab, click the More drop down and select Show All Other Values.
In the Classes section, right click on the <all other values> value and then select Format Symbol(s) (double-click on the colored rectangle)
Go to Properties (#1 on the screenshot below) > Layers (2), then select the fill symbol (3).
Under Appearance, select the database icon (4) to define the attribute to map, and select Color (5) or whatever you named that text field with the combined RGB colors.
Click ok then apply changes.
The layer in the Map will show the colors that will be read directly from the attribute table. However, there’s one problem. Even though the map will show the colors from the attribute table (stored as RGB, HEX or CMYK), the layer on the contents pain will still be showing <all other values>. You won’t be able to automatically create a legend or publish this straight to AGOL.
Our tech. support people investigated this, and this seems to be an issue that had been logged as the following bug: BUG-000104316: Attribute-driven symbology assigned to features is not reflected in the legend information for the feature layer