-
Creating Interactive Maps in GeoAnalytics Engine Using the ArcGIS Maps SDK for JavaScript
GeoAnalytics Engine allows you to plot geometries using st.plot and rt.plot. These generate static map graphics in your PySpark notebook; however, you may want interactivity in your visualizations. Using a Python script and the ArcGIS Maps SDK for JavaScript, you can create embedded interactive maps of both vector and…
-
Raster change calculations and zonal statistics at scale
Raster data is often a snapshot in time, for instance representing land cover, vegetation, or impervious surfaces. But the real value shows up when you compare those snapshots. With the raster capabilities in ArcGIS GeoAnalytics Engine, you can compute and summarize change at scale directly in your Spark workflows, no need…
-
What’s new in ArcGIS GeoAnalytics Engine 2.1
ArcGIS GeoAnalytics Engine 2.1 is now available, introducing GeoEnrichment at scale, including a sample demographic dataset with U.S. demographic attributes to help you add geographic context to your analysis. This release also includes enhancements to raster functionality, plotting performance, and overall product…
-
Upsert for selective feature service updating using GeoAnalytics Engine
Keeping spatial data up to date is a common challenge when working at scale, especially when analytics results need to be continuously published back to operational systems. With ArcGIS GeoAnalytics Engine, you can write analysis outputs directly to feature services. When updating feature services, you may not want to…
-
ArcGIS GeoAnalytics Engine 2.0: Release Resources
The ArcGIS GeoAnalytics Engine 2.0 release brings new capabilities that expand the data you can work with and enhance large-scale spatial analysis workflows. It also introduces some breaking changes that could affect workflows created with earlier versions. Explore what’s new and what’s changed: * What’s new in ArcGIS…
-
Setting up a Linux-based local Spark environment for GeoAnalytics Engine
Co-authored with Randall Whitman, Lawrence Khadka, and Ashwin Shashidharan from the GeoAnalytics software development team In this blog post we will walk through the process of setting up a local Apache Spark environment to work with ArcGIS GeoAnalytics Engine, on a machine running GNU/Linux (henceforth “Linux” here).…
-
GeoAnalytics Engine 2.0 Breaking Changes: What You Need to Know
With the release of GeoAnalytics Engine 2.0, changes have been introduced that may impact workflows, development practices, and compatibility. In this post, we’ll not only outline the breaking changes but also include code examples to illustrate what’s new, guiding you through each one step by step. Why Breaking Changes?…
-
How ArcGIS StreetMap Premium and ArcGIS GeoAnalytics Engine Transform Location Analysis Across Industries
Want to enhance your geocoding and network analysis at scale with authoritative street data? Read this blog to see how ArcGIS StreetMap Premium for ArcGIS GeoAnalytics Engine licensing supports location analysis across different industries. Already licensed? Read Accessing and working with StreetMap Premium data in…
-
Be able to see limit hour consumption by API Key
In order to activate the GeoAnalytics Engine libraries, we need to generate an API key using the GeoAnalytics Engine portal. But regardless of how many API keys we create, we cannot see how much of our compute hours is consumed per key in the dashboard. Because we may have different teams we are letting use our key and…
-
Be able to see compute hour consumption by API Key
In order to activate the GeoAnalytics Engine libraries, we need to generate an API key using the GeoAnalytics Engine portal. But regardless of how many API keys we create, we cannot see how much of our compute hours is consumed per key in the dashboard. Because we may have different teams we are letting use our key and…
-
Accessing and working with StreetMap Premium data in GeoAnalytics Engine
In this blog post we will walk through the process of accessing and using StreetMap Premium data and the corresponding license within a Spark-based computing environment using ArcGIS GeoAnalytics Engine for geocoding and/or network analysis. In order to work with StreetMap Premium data in GeoAnalytics Engine, you must have…
-
GeoAnalytics Engine: Databricks vs AWS EMR Performance Comparison
For large-scale spatial-temporal aggregation pipelines, has anyone benchmarked GeoAnalytics Engine on Databricks vs EMR? Curious about cost, cluster startup time, and data source (S3/Delta) performance differences.
-
Tuning GeoAnalytics Engine on Spark 4 for Large Spatial Joins
Running GeoAnalytics Engine 1.7 on Spark 4.0 with 500M+ records. What Spark tuning configs (executors, partitions, shuffle service) give the most stable performance for large spatial joins and aggregations?
-
GeoAnalytics Data Pipeline
Hi everyone, We’re setting up an Azure synapse pipeline where we ingest data from various ArcGIS datasets and services, convert it to Parquet files in our Data Lake where we’ll have GIS data and company data, and then create SQL views from those Parquet datasets. From there, the goal is to bring those views into our…
-
Setting up a local spark environment for GeoAnalytics Engine
In this blog post we will walk through the process of setting up a local Apache Spark environment in Microsoft Windows to work with ArcGIS GeoAnalytics Engine. While the GeoAnalytics Engine developer documentation provides great instructions for setting up Spark Local Mode, I will walk through the process with more…
-
Getting Started with GeoAnalytics Engine in Amazon EMR
In this post, we are going to walk through setting up and getting started with ArcGIS GeoAnalytics Engine using Amazon EMR. We will cover the basics of configuring the workspace with the GeoAnalytics Engine library, creating your environment, and offer a few examples to test out GeoAnalytics Engine and its functionality.…
-
Setting up your new GeoAnalytics Engine account
In this blog post, we walk through the details of setting up a new ArcGIS GeoAnalytics Engine account. For this discussion, we are assuming that you are currently setting up an evaluation or newly purchased license and have received an email prompting you to set up your account for use. When your account is created, your…
-
GeoAnalytics Engine Local Mode Py4JNetworkError
Hi all, I have been working in GeoAnalytics Engine Local Mode configuration for a few months now. Everything was working, and then one day I tried to run my code and this error started popping up. I have tried everything - checking system variables, reinstalling Java, re-downloading dependency files, even re-writing the…
-
Using API key-based authentication in GeoAnalytics Engine
This video demonstrates the use of API keys for authentication in GeoAnalytics Engine 1.5. This expands on the GeoAnalytics Engine authorization methods for the connected license, by allowing an API key to be used in place of a username and password. With API keys, a single GeoAnalytics Engine subscription can be used by…
-
Using API key-based authentication with GeoAnalytics Engine
Security and governance for authorized users is important in managing your analytics environment. To help with this, GeoAnalytics Engine 1.5.0 introduced the ability to create (and delete) API keys to provide access to users across your organization. This expands on the GeoAnalytics Engine authorization methods for the…
-
Error Uploading Linestring Dataframe from Databricks to ArcGIS Enterprise Portal: "No supported append format found."
Background: We've collected road data using an API and imported it into Databricks with the intention of displaying it on maps in ArcGIS Enterprise Portal. However, I'm encountering a problem when trying to upload a dataframe containing linestrings to the feature server in ArcGIS Enterprise Portal. Issue: When I attempt to…
-
Calculating vessel density and speed using the new GeoAnalytics Engine Track functions
Introduction Global Positioning System (GPS) data is often challenging to work with due to data volume and quality issues. For example, imagine a fleet of vehicles generating location data every second of every day. Until the data is cleaned up, the amount of data with inaccurate locations will increase, as will the…
-
Working with Snowflake data with GeoAnalytics Engine in Databricks
I feel like it’s rare to find an organization where all of the data is in one nice, tidy data warehouse or data lake. More likely than not, your data lives in a variety of locations, but you still want to be able to work with it in a single environment like Databricks. Lately, I’ve had a number of questions about working…
-
Understanding core-hours in ArcGIS GeoAnalytics Engine
There are two licensing models for ArcGIS GeoAnalytics Engine: Connected and Disconnected. In this post we’ll explore the concept of core-hours associated with the Connected licensing model, learn about the functions for tracking core-hours usage, and examine how many core-hours are used for a few example tasks. Core-hours…
-
Multipoint to Point
Pretty sure this is going to be an easy one, but looking for the most efficient way to take a multipoint layer from a public feature service where there is ever only one point in each feature, and convert it to point when loading a dataframe. url =…
