Latest Contributions by GregoryBrunner

I was sitting in a presentation a few weeks ago on the R-ArcGIS bridge by Marjean Pobuda and I had a question: "Can I use the R-ArcGIS bridge in my Jupyter Notebook?"  So I asked Mark Janikas (who was also attending the presentation) if this would be possible an he said, "Yes".  So, after the presentation, I set out to get the R-ArcGIS bridge running in Jupyter. Installing the ArcGIS-R Bridge The first thing I did was install the R-ArcGIS bridge. I installed it using ArcGIS Pro by following the installation instructions. I am currently using R-3.4.2 and arcgisbinding '1.0.1.208'. I can verify this by going to the Geoprocessing tab in the Options section in ArcGIS Pro. Cloning My arcgispro-py3 Environment I did not want to break my arcgispro-py3 conda environment (the default ArcGIS Pro Python environment) so the first thing I did was clone the environment. I named the cloned environment arcgispro-r. I did this from the command line as follows: Switching Environments Next, I switched to the cloned environment by deactivating the arcgispro-py3 environment and activating the arcgispro-r environment. This can be done at the command line with two commands, deactivate and activate, as shown her: Installing R-Essentials Then, I installed r-essentials, a bundle of over 80 of the most used R packages created by the Anaconda team. I did that from the command line while in my arcgispro-r environment as shown here: Running arcgisbinding in Jupyter After I installed r-essentials, when I launched my Jupyter Notebook, I had the option to create a Python or an R notebook. I created an R notebook. To test whether the ArcGIS-R bridge is installed and accessible to my notebook, I loaded the arcgisbinding package and checked the product version number and there it was, package version '1.0.1.208', the same one I see listed in ArcGIS Pro above! But Does It Work? Yes, I can use the arcgisbinding package to read spatial data into R! In order to test whether I could read in data, I used arc.open to read in a point feature class of seagrass data. I was also able to use arc.select to put that feature class into a dataframe. I shared my sample notebook on GitHub at the repo arcgisbinding-in-jupyter. I am interested to know if there is anyone else out there who has tried this or is interested in using R, ArcGIS, and Jupyter. If you are, let me know! ... View more

What's in your workflow? A presentation by Emily Riederer ([email protected]), a Sr. Analyst for Capital One from Data Pop Up Chicago in November of 2017. While business analysis rapidly grows more data-driven, the analyst community is slow to adapt the best practices of data science workflows. Many parallels exists between data science “top topics” (e.g. reproducibility) and business pain points, but these common needs are obscured by the different “languages” of these two communities. The opportunity cost is greatest in heavily regulated industries such as finance and insurance where documentation and compliance are paramount.  In this talk, we will review our experience transitioning Capital One business analysts from legacy systems to open-source workflows by developing user-friendly tools. We incentivized business analysts to adopt the data science mindset by curating open-source tools and developing code packages which simplify workflows and eliminate pain points.  Our internal R package, tidycf, reimagines cumbersome Excel cashflow statements as dataframes and uses RMarkdown templates and the RStudio IDE for an intuitive, user-friendly experience without the overhead of maintaining a custom GUI. We tackle challenges in documentation and communication while immersing new users in the R language.  We will share best practices and lessons learned from our experience designing tools for non-technical end-users, standardizing workflows based on the RStudio IDE’s infrastructure, and evangelizing data science methods.  Check out her slide deck for this presentation here. ... View more

A few weeks ago, Esri released an update to the ArcGIS API for Python. The newest release includes: An intuitive and powerful raster API Powerful GIS administration Spatial DataFrames!!! Hopefully, you can tell that the new functionality in the API that I am most excited about is the spatial dataframe! The spatial dataframe extends the pandas dataframe by adding geometry, spatial reference, and other spatial components to the dataframe. In adding the spatial dataframe to the API, ArcGIS users can now read feature classes, feature services, and image services directly into a dataframe. Once in a spatial dataframe, users can perform fast statistical and spatial analysis on the data, update existing feature services, and convert the dataframe to a feature class or shapefile. These are just a few examples of how you can use the spatial dataframe. What really interests me is how this can be used with an ArcGIS image service. Can I use the spatial dataframe to extract image footprints from an image service? Can I use it to perform statistical analysis image footprints over a specific part of the world? The answer to both of these questions is Yes! In this post, I’ll walk through how to use the API for Python to extract image service footprints from the Landsat 8 Views image service, show how to use a spatial filter to extract only footprints over New Jersey, determine the mean cloud cover and most recent acquisition date of the images, and share those image footprints as a feature service. If you have ever been interested in doing any of these, check out my post on gavinr.com! ... View more

A few weeks ago, Esri released an update to the ArcGIS API for Python. The newest release includes: An intuitive and powerful raster API Powerful GIS administration Spatial DataFrames!!! Hopefully, you can tell that the new functionality in the API that I am most excited about is the spatial dataframe! The spatial dataframe extends the pandas dataframe by adding geometry, spatial reference, and other spatial components to the dataframe. In adding the spatial dataframe to the API, ArcGIS users can now read feature classes, feature services, and image services directly into a dataframe. Once in a spatial dataframe, users can perform fast statistical and spatial analysis on the data, update existing feature services, and convert the dataframe to a feature class or shapefile. These are just a few examples of how you can use the spatial dataframe. What really interests me is how this can be used with an ArcGIS image service. Can I use the spatial dataframe to extract image footprints from an image service? Can I use it to perform statistical analysis image footprints over a specific part of the world? The answer to both of these questions is Yes! In this post, I’ll walk through how to use the API for Python to extract image service footprints from the Landsat 8 Views image service, show how to use a spatial filter to extract only footprints over New Jersey, determine the mean cloud cover and most recent acquisition date of the images, and share those image footprints as a feature service. If you have ever been interested in doing any of these, check out my post at gavinr.com! ... View more

Interested in writing image processing algorithms in ArcGIS? Wondering how to write customer raster analysis functions using #python? Then check out this video from devsummit 2017 on Writing Image Processing Algorithms Using the Python Raster Function! The presenters review what the python raster function is; how it it implemented; and best practices for creating, using, and publishing python raster functions. Along the way they show compelling examples of custom python raster functions that have been written to calculate compound topographic index and site suitability. Enjoy! ... View more

I have been using Jupyter notebooks for a while now and what I have really wanted to do is figure out how to turn my notebooks into slideshows.  Fortunately, someone has already figured that out and posted a video on YouTube! Check out James Church's presentation on How to Create a Slideshow using Jupyter+Markdown+Reveal.js to see how it's done. I can vouch for the fact that James Chrurch's methods work because I was able to very easily turn one of my notebooks into a rich, interactive Reveal.js presentation. Check out my presentation! Reveal.js is a powerful medium for presentation Python code. This will make for an engaging means to present projects that leverage spatial analysis and the ArcGIS Python API. Let the Data Science group know if you end up using Jupyter and Reveal.js! ... View more

Did you know that you can bring a Python Notebook into a Story Map? One of the many nice features of Python Notebooks is that you can export them.  To embed it into my Story Map, I exported it as HTML. Once I downloaded the HTML file, I hosted it as a webpage in my Github project through gh-pages. Then, it just becomes a matter of adding the HTML to the Story Map. The exported notebook actually tags the different sections. Here, I've highlighted the tag that corresponds to a certain section in my notebook. What I think is really cool is that that you can drive to the difference sections in the Notebook through the Story Map using these section tags. Here is how I used my Python Notebook inside of the Story Map: I used the Story Map Journal and thought it was really cool that I could connect different sections of the Story Map to different sections in the Python Notebook. Just wanted to share this data storytelling tip with any Pythonistas out there who love Story Maps! ... View more

Have you ever wanted to interact with a webmap using Python? Would you want to see those updates or modifications as soon as they happen?  Using ArcREST in your Jupyter Notebook, you can create and modify webmaps and services and visualize those changes within seconds of them occurring without having to leave your Jupyter Notebook.  In this post, I'll demonstrate how to display a webmap in your Jupyter Notebook and then how to use ArcREST to create a webmap and visualize it in Jupyter as soon as it's created. Displaying a Webmap Displaying a webmap is pretty straightforward. First, I imported IFrame: from IPython.display import IFrame Next, I wrote a function to show the webmap in the IFrame given a webmap ID. def show_web_map(id):     url = 'http://www.arcgis.com/home/webmap/viewer.html?webmap=' + id     return IFrame(url, width='100%', height=500) Then, I called show_web_map and passed it a webmap ID. show_web_map('7d1e96212b06491b9c3cc457c3dc1835') Here's the result: Using Jupyter with ArcREST to Create and Display a Webmap The above example is cool, but you might be asking, "How is this useful?" Displaying the web map in your Jupyter Notebook can become really powerful if you use ArcREST​ and are creating or editing a service or a webmap. To demonstrate this, I downloaded and installed ArcREST.  I wanted to show that I can dynamically create a webmap and visualize it in my Jupyter Notebook in a matter of seconds, so I copied the additem_webmap_sample.py script into my Notebook.  This sample script grabs the all_hour.csv file and 2.5_hour.csv file from the USGS earthquake feed and adds those earthquake features to a webmap. Here's the script in my Jupyter Notebook: I concealed my username and password above, so you'd have to insert your info there.  I modified and added these last few lines: item = user.addItem(itemParameters=itemParams,overwrite=True,                     text=json.dumps(webmap_json_as_dict)) #Share the item with everyone item.shareItem(everyone=True) print(item.id) I made sure to share the resulting webmap with everyone so that it shows in the Jupyter Notebook without having to login. After running the additem_webmap_sample.py script, I showed the webmap by calling the item ID. show_web_map(item.id) Here's the resulting webmap in my Jupyter Notebook: I was able to verify that those were the only three earthquakes in the last hour by downloading the all_hour.csv file. A Quick Way to Create, Update, and Visualize Webmaps and Services While the example above is pretty simple, I think it lays a good framework for how Jupyter can be used to work with webmaps and services.  If you have the need to periodically update a webmap and use ArcREST to do it, Jupyter makes it easy to run the script and check the result in a single location.  If you want to batch create some webmaps and see the results, Jupyter makes reviewing the webmaps easy. Or if you're looking for a playground for creating and updating webmaps, Jupyter is good for that too. Hopefully the examples above demonstrate that and give you some ideas for how to work with webmaps and services using Python and Jupyter. For more spatial data science tips and tricks, check out the Data Science group or reach out to Gregory Brunner. ... View more