Spatial Data Science Blog

Esri's no-cost Spatial Data Science massive open online course (MOOC) returns on October 5, 2022, and we invite you to register. Over six weeks, explore the latest ArcGIS capabilities for spatial analysis, predictive modeling, machine learning, and more.

Cool new features released with ArcGIS JS APIs 4.17, my first take aways:

• View Constraints: now you can limit the user navigation to be around certain geometry, help the user to focus on specific area
• Layer Visibility Animation: the APIs now supports fading effect while toggling layer visibility
• OGC Support: this is emphasized with 2 new features
  • WCSLayer: new layer allows rendering raster data from OGC Web Coverage Service
  • WMSLayer time enabled: now you can configured widgets like Time Slider with time-enabled WMS layers
• Widgets updates, now Bookmarks widgets saves the bookmark thumbnail, the user will have to set thumbnail URL 
  
  

More details can be found in this ESRI blog: What’s New in ArcGIS API for JavaScript (October 2020) 
New Code Samples for 4.17 Sample Code | ArcGIS API for JavaScript 4.17 

Ozri 2020 showcased the latest ArcGIS solutions and case studies from Australia and around the world. To access the video recordings and slides from each session, simply select a presentation from the list here

Ozri on-demand | Esri Australia 

The Federal Geographic Data Committee (FGDC) is seeking public comment on the draft strategic plan for the National Spatial Data Infrastructure (NSDI). Comments are due by September 17, 2020.

For a quick and focused tech-talks on latest ESRI trends and topics, tune in to GIS Directions podcast | Esri Australia 

I was sitting in a presentation a few weeks ago on the R-ArcGIS bridge by Marjean Pobuda and I had a question: "Can I use the R-ArcGIS bridge in my Jupyter Notebook?"  So I asked Mark Janikas (who was also attending the presentation) if this would be possible an he said, "Yes".  So, after the presentation, I set out to get the R-ArcGIS bridge running in Jupyter.

Installing the ArcGIS-R Bridge

The first thing I did was install the R-ArcGIS bridge. I installed it using ArcGIS Pro by following the installation instructions. I am currently using R-3.4.2 and arcgisbinding '1.0.1.208'. I can verify this by going to the Geoprocessing tab in the Options section in ArcGIS Pro.

Cloning My arcgispro-py3 Environment

I did not want to break my arcgispro-py3 conda environment (the default ArcGIS Pro Python environment) so the first thing I did was clone the environment. I named the cloned environment arcgispro-r. I did this from the command line as follows:

Switching Environments

Next, I switched to the cloned environment by deactivating the arcgispro-py3 environment and activating the arcgispro-r environment. This can be done at the command line with two commands, deactivate and activate, as shown her:

Installing R-Essentials

Then, I installed r-essentials, a bundle of over 80 of the most used R packages created by the Anaconda team. I did that from the command line while in my arcgispro-r environment as shown here:

Running arcgisbinding in Jupyter

After I installed r-essentials, when I launched my Jupyter Notebook, I had the option to create a Python or an R notebook. I created an R notebook. To test whether the ArcGIS-R bridge is installed and accessible to my notebook, I loaded the arcgisbinding package and checked the product version number and there it was, package version '1.0.1.208', the same one I see listed in ArcGIS Pro above!

But Does It Work?

Yes, I can use the arcgisbinding package to read spatial data into R! In order to test whether I could read in data, I used arc.open to read in a point feature class of seagrass data. I was also able to use arc.select to put that feature class into a dataframe. I shared my sample notebook on GitHub at the repo arcgisbinding-in-jupyter. I am interested to know if there is anyone else out there who has tried this or is interested in using R, ArcGIS, and Jupyter. If you are, let me know!

What's in your workflow? A presentation by Emily Riederer (emily.riederer@capitalone.com), a Sr. Analyst for Capital One from Data Pop Up Chicago in November of 2017.

While business analysis rapidly grows more data-driven, the analyst community is slow to adapt the best practices of data science workflows. Many parallels exists between data science “top topics” (e.g. reproducibility) and business pain points, but these common needs are obscured by the different “languages” of these two communities. The opportunity cost is greatest in heavily regulated industries such as finance and insurance where documentation and compliance are paramount. 

In this talk, we will review our experience transitioning Capital One business analysts from legacy systems to open-source workflows by developing user-friendly tools. We incentivized business analysts to adopt the data science mindset by curating open-source tools and developing code packages which simplify workflows and eliminate pain points. 

Our internal R package, tidycf, reimagines cumbersome Excel cashflow statements as dataframes and uses RMarkdown templates and the RStudio IDE for an intuitive, user-friendly experience without the overhead of maintaining a custom GUI. We tackle challenges in documentation and communication while immersing new users in the R language. 

We will share best practices and lessons learned from our experience designing tools for non-technical end-users, standardizing workflows based on the RStudio IDE’s infrastructure, and evangelizing data science methods. 

Check out her slide deck for this presentation here.

A few weeks ago, Esri released an update to the ArcGIS API for Python. The newest release includes:

• An intuitive and powerful raster API
• Powerful GIS administration
• Spatial DataFrames!!!

Hopefully, you can tell that the new functionality in the API that I am most excited about is the spatial dataframe! The spatial dataframe extends the pandas dataframe by adding geometry, spatial reference, and other spatial components to the dataframe. In adding the spatial dataframe to the API, ArcGIS users can now read feature classes, feature services, and image services directly into a dataframe. Once in a spatial dataframe, users can perform fast statistical and spatial analysis on the data, update existing feature services, and convert the dataframe to a feature class or shapefile. These are just a few examples of how you can use the spatial dataframe.

What really interests me is how this can be used with an ArcGIS image service. Can I use the spatial dataframe to extract image footprints from an image service? Can I use it to perform statistical analysis image footprints over a specific part of the world?

The answer to both of these questions is Yes! In this post, I’ll walk through how to use the API for Python to extract image service footprints from the Landsat 8 Views image service, show how to use a spatial filter to extract only footprints over New Jersey, determine the mean cloud cover and most recent acquisition date of the images, and share those image footprints as a feature service. If you have ever been interested in doing any of these, check out my post at gavinr.com!

I have been using Jupyter notebooks for a while now and what I have really wanted to do is figure out how to turn my notebooks into slideshows.  Fortunately, someone has already figured that out and posted a video on YouTube! Check out James Church's presentation on How to Create a Slideshow using Jupyter+Markdown+Reveal.js to see how it's done.

I can vouch for the fact that James Chrurch's methods work because I was able to very easily turn one of my notebooks into a rich, interactive Reveal.js presentation. Check out my presentation!

Reveal.js is a powerful medium for presentation Python code. This will make for an engaging means to present projects that leverage spatial analysis and the ArcGIS Python API. Let the Data Science group know if you end up using Jupyter and Reveal.js!

Did you know that you can bring a Python Notebook into a Story Map?

One of the many nice features of Python Notebooks is that you can export them. 

To embed it into my Story Map, I exported it as HTML. Once I downloaded the HTML file, I hosted it as a webpage in my Github project through gh-pages. Then, it just becomes a matter of adding the HTML to the Story Map.

The exported notebook actually tags the different sections.

Here, I've highlighted the tag that corresponds to a certain section in my notebook.

What I think is really cool is that that you can drive to the difference sections in the Notebook through the Story Map using these section tags. Here is how I used my Python Notebook inside of the Story Map:

I used the Story Map Journal and thought it was really cool that I could connect different sections of the Story Map to different sections in the Python Notebook.

Just wanted to share this data storytelling tip with any Pythonistas out there who love Story Maps!