Insights for ArcGIS keeps getting better, and our June release is no exception. We hope you enjoy the updates we've made to maps, charts, tables, and data in ArcGIS Online and ArcGIS Enterprise.
Here are your top five takeaways from this release:
#1 Charts in Maps Is that a chart in your map? Yup. Use this new map visualization to answer questions such as this one: what does the distribution of gas station brands look like between locations?
#2 Summary Tables Bring on the variables. Analyze an unlimited number of strings to see groupings of data based on interacting categories.
#3 Percentile Statistics for Reducing the Noise of Outliers With Insights, the percentile is now configurable—meaning that you can define a value (such as 90th percentile) and visualize the distribution of records that fall above and below that value.
#4 Shapefiles Drop shapefiles on an Insights page (must be zipped) just like you would an Excel spreadsheet. Also, save feature layers as shapefiles and CSV files to share with colleagues and use in other applications such as ArcGIS Pro and Excel.
#5 Scripts for Expanding the Possibilities Edit and run previously executed cells, add a script to a model, and export the scripting history to the data pane.
New Capabilities Available in Insights for ArcGIS!
We hope you enjoy the new enhancements we've added to help you manage data, utilize Python and R, and collaborate with others.
Please note that if you are an ArcGIS Enterprise customer and have deployed Insights for ArcGIS with ArcGIS Enterprise 10.7, be sure to download Insights 3.2.1. If you are running ArcGIS Enterprise 10.6.1, please download Insights 3.2. Need help upgrading to the latest Insights release in ArcGIS Enterprise? Check out our help.
Here are your top five release takeaways:
#1 Personalized Home Page
As soon as you log into Insights, see your recent workbooks, your favorites, and the most used items by you and your organization.
#2 Unlimited Analytical Methods at Your Fingertips
Work and integrate with the tools and languages you already know—Python and R. Connect to your existing Jupyter or Anaconda setup—no need for a new deployment. Please note that this is a public beta.
#3 Share Insights Items to Another ArcGIS Organizational Account
Now, Insights items (models, workbooks, themes, pages) are supported in distributed collaborations. Share Insights items to another ArcGIS Enterprise 10.7 or ArcGIS Online organizational account.
#4 Connect to PostgreSQL
Spatial data stored within PostGIS (the native spatial type) and Esri geodatabase (ST_Geometry type) are now supported.
#5 Insights Analyst User Type
The Insights Analyst user type is a new way to license Insights. It’s designed for an analyst who only wants access to Insights, and nothing else within ArcGIS. If you have a user who needs to perform analysis on your GIS content and other non-spatial business data together, all within a familiar non-disruptive BI-like environment, this user type is ideal.
Get the full scope. Check out our release blog for complete details and demos.
We have a feature service populated by data collected in Survey123. A single survey collects multiple (5) well depth readings. Using Insights, we would like to display/correlate multiple well elevations, over time, on a single card. We have created individual cards for each well reading, but would like to combine them for the sake of elevation comparison. The attached image from Excel is what we would like to achieve in Insights. Or if there is a better solution, we would like to know what that is. Thanks!
Getting Started with Insights for ArcGISis your one-stop-shop for everything you need to know when installing Insights. This guide summarizes the steps of each deployment option, information you should consider when installing, and provides links to detailed help topics.
Ready to install Insights for ArcGIS in your organization?
Several years ago, as a new mother, I started hearing about hypodermic needles showing up in local playgrounds that I had always considered safe. This was a couple of years before the opioid epidemic was national news but I wanted to understand whether this was a major issue in Massachusetts, where I live, or a series of isolated incidents. Because I work with maps every day, that was where I turned. In the years that have passed I’ve continued to gather data and use different geospatial tools to look at the opioid crisis. I’ve also been incredibly lucky that Massachusetts publishes a wealth of open/public data related to substance use disorder including fatal overdoses and treatment statistics.
In this project I’ve used Insights for ArcGIS to try to understand the epidemic over time as well as plan two different actions: One related to prevention and one related to treatment. After all, data is most useful when it’s helping people make better decisions.
Understanding Where the Problem Exists
If we compare maps of fatal opioid-related overdoses from 2000 and 2015, we see the drastic differences. As you can see in Figure 1, in 2000 Massachusetts 338 reported fatal overdoses from opioids. Only six towns or cities had more than 10 fatal overdoses with Boston having the highest number at 36. Jump ahead to 2015, shown in figure 2, and the image is much starker. In 2015, there were just shy of 1,600 deaths attributed to opioid overdose – an almost 5x increase! We also see in the heat chart that 39 towns or cities had ten or more fatal overdoses. The map tells us that these towns are spread out throughout the Commonwealth.
Figure 1. Fatal opioid-related overdoses, 2000.
Figure 2. Fatal opioid-related overdoses, 2015.
Like most chronic diseases, there is no silver bullet to fix the opioid crisis. However, there are several generally accepted helpful factors including prevention, treatment, and recovery. I am not an expert in substance use disorder, but I wanted to apply workflows from my experience working with private and public health organizations to look at potential prevention and treatment strategies. These decisions usually revolve around allocating resources, targeting the marketing or outreach for these resources, or understanding access to resources.
Figure 3. Targeting outreach for existing drop boxes.
Asking Questions to Identify a Resolution
Where would outreach be helpful?
The first question that I wanted to answer was how to increase use of existing prescription drug drop boxes. My first goal was to identify at-risk communities with existing drop boxes. I started by adding a spreadsheet of drop box locations and community-level treatment data to the map. I filtered the communities to include only those with admissions numbers for prescription drugs—at least 10 percent of treatment admissions listed another opioid, not heroin, as their primary drug. This allowed me to proxy risk for prescription drug misuse. These are the substances that we want properly disposed of in drop boxes. By then spatially filtering to only those towns with prescription drop boxes I have a short list of communities where outreach could be helpful.
Figure 4. High priority communities for drop box use.
Who I should target with a campaign?
Although you could use the same outreach tactics throughout the areas of interest, in many cases there are additional factors that make a targeted approach more effective (e.g., language or other demographic traits). We can get more of that information using segmentation data. When I enriched the list of communities with their dominant tapestry segment we see in the donut chart that several communities fall into the Green Acres, Front Porches, or Parks and Rec segments. I could try to find common behaviors between these segments to do an all-encompassing campaign but since my main goal is to reduce fatal overdoses, I’ve chosen to focus on areas with the highest numbers of fatal overdoses. By selecting the biggest community in the bubble chart, Taunton, which represents the highest number of fatal overdoses (14 in 2015) I can see the associated tapestry, Front Porches highlighted in the donut chart. People who fall into the Front Porches segment, among other things, use online gaming and dating, listen to hip hop and R&B, and watch Comedy Central—all information that could be used to tailor outreach efforts.
Figure 5. Dominant Tapestry for Taunton
Do affected communities have easy access to care?
I also wanted to understand access to care, specifically Medically Assisted Treatment (MAT), and whether that is affecting outcomes. There is evidence to show that MAT, the combination of medication and counseling/behavior therapies, can help sustain recovery. I wanted to understand whether lack of access to MAT was related to fatal overdose numbers. By using location analytics to calculate the population weighted average drive time from each town or city to the nearest MAT facility, regardless of whether it provided standalone methadone or office based suboxone/vivitrol, I could then use Insights to visualize possible relationships.
My initial findings were surprising but it led me to ask additional questions which did provide some interesting conclusions. The first thing that I determined was that there are close to 50 towns or cities in Massachusetts that are more than 30 minutes from the nearest MAT facility. As you can see on the map, this is shown with two distinct bands in the central and western regions as well as less accessible locations like the tip of Cape Cod, Nantucket, and Martha’s Vineyard. I expected to see a correlation between access to MAT and fatal overdose numbers, but I did not. As you see in the top scatter plot there is virtually no correlation.
Now, I decided to consider other factors that affect access such as public transportation. In many situations, patients are reliant on public transportation to get medical care so if it is difficult or impossible to get somewhere via public transportation the distance of that care is irrelevant. To model public transportation reliance, I enriched the towns/cities with the number of households that did not have access to a vehicle. The bottom scatter plot shows a strong correlation between the number of households without access to a vehicle and the number of fatal overdoses. A quick note, this analysis is not normalized by population so it’s possible that population is confounding. Nonetheless, I felt that this opens a door to additional questions about whether MAT facilities are really accessible to the populations that need them most.
Although I’ve been visualizing and analyzing variations of this data for several years I feel like I gained a new understanding using Insights for ArcGIS. Let’s start with the most obvious benefit. Being able to couple the map, charts, and tables in the same interface – and have them respond to each other – has let me explore the data more quickly and in more ways than I could have previously done. But beyond that, the ability to do ad hoc spatial analytics, from spatial filtering to enrichment, lets me do more than just visual analytics. It lets me ask questions to refine my analysis and manipulate the data to find answers that often lead to more questions. Insights for ArcGIS allows me to go as deep into the problem as necessary, giving me confidence in the analysis I share with others.
I encourage you to give Insights for ArcGIS a try with your data. Discover how quickly you can come to results you may have explored with other tools.
As a continued effort to improve the product and get you the insights you need (pun intended), we want to hear your wants and needs around geodatabases. For example, do you want to make use of existing relationships and domains?
Influence future capabilities of geodatabases within Insights. Leave us your comments below.
Thanks in advance,
Mallory Delgadillo, Doug Cochran, and The Insights Development Team
Insights for ArcGISjust got better! In the 2.0 release, you get optimized analysis tools to reveal more of the patterns, trends, and relationships that are hidden in your data. Now you can do powerful exploratory analysis faster and easier than ever.
What makes it faster and easier? Take a look at the arsenal of capabilities you can take advantage of with Insights for ArcGIS 2.0 and ArcGIS Enterprise 10.5.1.
Examine data with greater speed and performance
Just when you thought Insights couldn’t perform any better, it’s now faster at loading data and performing analysis. This makes it so much easier for you to explore large datasets or multiple data sources, and identify gaps. Maybe most importantly, it means you can quickly visualize and understand your data before you take it to ArcGIS Pro for advanced analysis work.
More data sources available and ways to manipulate data
The Add to Page pane now has new tabsIs your organization an Oracle shop? Insights for ArcGIS 2.0 has added support for Oracle databases. You can now connect directly to Oracle databases from within Insights, as well as previously supported SQL Server, SAP HANA, and Teradata databases. This includes support for R scripts in views and stored procedures within supported native relational databases. Creating direct database connections means that you can easily access your data from Insights and see increased performance with database data.
Comma separated value (CSV) files can now be used the same as Excel workbooks, meaning you do not have to convert your CSV data to an Excel Workbook before you begin your analysis.
It is also easier to access your previously available data from Living Atlas, standard boundaries, and Groups in ArcGIS Enterprise. Tabs for these data sources have been added to theAdd to Pagepane so that you can easily find and access the data you need.
Additionally, we added new functions for calculating fields so you can manipulate your data and perform analysis directly in Insights. Big time saver!
More chart options to explore and analyze your data
Visualize and analyze your data in new ways with four chart types that are now available in Insights: Box plots, data clocks, chord diagrams, and heat charts. Charts are an important non-spatial addition to your analysis that can help you learn more about your data. When paired with a map using the same dataset, charts can be used to select data and visualize both the spatial and non-spatial distribution at the same time.
Some charts, such as scatter plots, now have the ability to switch between logarithmic and linear scales. Logarithmic scales are an important tool for visualizing data that is spread across large numeric ranges or has outliers. This makes finding relationships and patterns in data so much easier.
Changing a scatter plot axis from linear to logarithmic.
More style control over your visualizations
There have been new additions to help you customize the look of your maps. The improved legend includes the ability to style your data using an expanded color pallet or custom colors using hex values. New at Insights 2.0, you can also change the basemap of a single card or an entire page from directly within Insights. These changes mean that it is easier for you to make sure that your maps tell the story that you want them to tell
Preview support for ArcGIS GeoAnalytics Server
Support for ArcGIS GeoAnalytics Server is now in Preview mode. If GeoAnalytics Server is configured with your ArcGIS Enterprise organization, then you now have access to hosted feature layers with data stored in the spatiotemporal big data store within Insights. Distributed storage and analytics from GeoAnalytics Server will be used automatically when applicable.
Don’t wait to act on Insights
Waiting for ArcGIS Online support? Running on ArcGIS 10.4 or older and concerned that a migration to 10.5.1 will impact your business? We’re here to help. You can use Esri Managed Cloud Services to get your organization up and running quickly with world-class hosting and management support. Teaming with Esri gives you peace of mind knowing that experts in cloud and GIS are managing your system. This frees you up to focus on discovering and sharing information.Learn more about Esri Managed Cloud Services.
Did you know that the release of ArcGIS Enterprise 10.5.1 includes anArcGIS Enterprise Builder? It’s now easier to install and configure a base ArcGIS Enterprise deployment on a single machine.
The quick exercise Get started with Insights for ArcGIS has been updated for version 2.0. This exercise is a great resource if you are new to Insights, or if you are looking for some inspiration in your analysis.
Insights for ArcGIS at the 2017 Esri User Conference
With the2017 Esri User Conferencejust around the corner, please make sure to visit the Insights team at SDCC - Exhibit Hall B1. Insights for ArcGIS, version 2.0 has just released with more capabilities such as new chart types, additional database support, significant performance gains, and more... You’ll want to see it in action and learn all about itsnew and powerful abilitiesfor discovering exploratory spatial analysis. See our “What’s New” blog for more detailed information on Insights for ArcGIS version 2.0.
Insights for ArcGIS is a web-based, data analytics workbench where you can explore spatial and non-spatial data to perform iterative and exploratory data analysis. With a drag-and-drop interface, you can answer questions, and quickly deliver powerful results using data from ArcGIS, Excel spreadsheets, and business databases. Learn how to create maps, charts, and tables to visualize and tell your story like never before using advanced algorithmic spatial analysis with minimal GIS knowledge.
Stop by and meet the team, ask questions, and share your feedback. We look forward to seeing you at UC 2017!
Here is a list of Insights for ArcGIS events and workshops at the UC:
We would like to personally invite you to attend our Insights social occurring at The Shout! House on Tuesday, July 11th at 7:30 pm, meet the Insights team and others that share interest in all things ArcGIS and Insights for more casual conversation.
With Insights for ArcGIS, you can quickly discover the secrets your data holds. Insights works alongside and takes advantage of other leading ArcGIS products. Visualize, analyze, and tell your story like never before. Insights works alongside and takes advantage of other leading ArcGIS products. This presentation will focus on the basic configurations requirements and usage of Insights for ArcGIS
Tuesday, July 11: 4:40-5:15 PM, SDCC - Demo Theater 09
Insights for ArcGIS is a new user experience for analysis available at ArcGIS 10.5 that focuses on simpler ways to work with your data to answer questions and share results including visualization first, drag and drop analytics and, on the fly filtering and aggregations. Insights for ArcGIS is designed to make interactive and exploratory analysis fast and intuitive so you can gain understanding from your data in a spatial context. Learn techniques in wrangling your data for analysis, best practices in working with Insights and its analysis capabilities and more.
Tuesday, July 11: 3:15-4:30 PM, SDCC - Ballroom 06 E
Wednesday, July 12: 10:15-11:30 AM, SDCC - Ballroom 06 E
Thursday, July 13: 10:150-11:30 AM, SDCC - Ballroom 06 E
In this session, we will focus on advanced topics in working with Insights for ArcGIS at 10.5 to include understanding the architecture, getting to know patterns and best practices, as well as tips and tricks. To get the most from this session, it would be best to attend the Introduction session as a prerequisite first.
The workshop will introduce key spatial approaches for solving real world problems using ArcGIS analysis techniques to make better decisions. During the session we will present several standardized workflows for conducting spatial analysis. Following the workflows, we will translate the spatial problems into spatial techniques, and demonstrate different analytical tools in ArcGIS for Desktop to find an appropriate solution for the problem. The workshop will include overlay analysis, proximity analysis, and surface analysis, with use cases and software demonstrations. It serves as an overview and introduction to spatial analysis using ArcGIS for Desktop including the new desktop application ArcGIS Pro.
Tuesday, July 11: 10:15-11:30 AM, SDCC – Room 17 B
At the recent American Planning Association's National Planning Conference, I worked at the Esri booth and had the chance to talk with a lot of planning professionals about their use of GIS. A lot of attendees have been using ArcGIS for a long time, and a common question was "what's new?" Insights for ArcGIS, of course!
I had prepared an Insights workbook for the conference that I thought would be appealing to planners - it focuses on Commercial Permits data for Montgomery County, MD (which I downloaded from the County's Open Data site). Many times, a county invests in technology systems to manage review and issuing of permits, and many counties have data going back several years or more. This historical data can be very informative to planners, but it might not be a dataset they have access to or know how to use.
Insights for ArcGIS can address this - by supporting multiple data formats, it allows planners to bring in permit data using a variety of methods. Once the data is brought in, Insights offers tools that make exploratory and spatial analysis easy and intuitive. To demonstrate this, I wanted to highlight what I showed at APA with the permitting data, and point out some usability tips along the way. Let the data exploration begin!
First, I created a table showing the permits sorted by Use Code - this showed me the predominant categories of permits were Businesses and Multi-Family Homes. I created a second card showing the permits by Status - this is over 6+ years, so it makes sense that only a small percent (7%) are open. To see the distribution across the county, I made a map card.
Tip #1 - When using the location field to drag and make a new map card, select another field (in this case, Status) to style the map - this gives you a more useful visual display in fewer clicks!
Next, I wanted to look at the distribution over time - by year, quarter and day of week on which the permit was added. We can see that with an annual average of 1,400 and 456 permits already issued in 2017 with the busy time of year still coming up (Q3), we can probably anticipate a high year:
Tip #2 - Take advantage of the way that Insights recognizes date components in your Date/Time fields. I didn't have to calculate year, quarter, and day of week - they were listed for me when I expanded the Date Added field!
When I look at the permitting activity as a line graph, a new pattern emerges - I see a really high spike in activity in the summer of 2011. My first question - what happened here? Styling my line chart by use code, I see that the spike can be attributed to Multifamily Housing - there must have been a bunch of new development. Now I want to see where! Clicking on the spike filters my map to just those incidents - and when I change my map styling to a Heat Map, the pattern pops out - the bulk of the development occurred right outside Clarksburg:
Tip #3 - Changing the styling of your map layers can make all the difference when doing exploratory analysis! Before I use the Heat Map, the dots were so close together, I couldn't see the clustering.
There's so much more to explore with this dataset - is there a relationship between duration and cost of a permitted project? What are the characteristics of the neighborhoods where we are seeing recent growth? Are there under-served areas of the county for certain types of development? Insights for ArcGIS can help answer these questions, and more.