Latest Contributions by BrianBaldwin

I recently put together a simple demonstration to show how easy it is to integrate IoT sensors from Disruptive Technologies with ArcGIS. Within just a few hours, you can start getting real-time updates from field devices (cabinets, transformers, server rooms, etc) into your maps and dashboards. Some of the use cases for the integration included getting real-time alerts around potentially hazardous situations (open cabinets/manholes) and assisting with operations by providing real-time alerts on non-SCADA devices.  If you missed the recent demonstration video, you can find it here. This blog is going to walk through the configuration that was demonstrated in the video (with help from Niklas of Disruptive Technologies... thanks Niklas!). It will take you step-by-step through the setup and provide the configuration files for getting an ArcGIS/Disruptive Technologies environment configured. The first thing we are going to work on, is getting GeoEvent Server configured. If you are new to Esri’s GeoEvent for ArcGIS, please take a look at some of the Getting Started material. The most basic definition of GeoEvent, is that it is a hub for real-time data that can be associated with a map. If you want to visualize real-time temperature, watch status changes on a device, or monitor weather feeds, then GeoEvent is the tool that can be used to connect A to B.  In GeoEvent, you basically need to get 3 things to get the integration in place with the Disruptive Technologies sensors: define an input, create a GeoEvent service, and then create an output. We will be walking through all 3 steps. GeoEvent - Defining the Input The first step in GeoEvent, is to build out your input. In this case, we are going to be using the ‘Receive JSON on a REST Endpoint’ input. This is installed out of the box, so you just need to select it to get started. After you select ‘Receive JSON on a REST Endpoint, you need to configure it. The only ‘gotcha’ is to ensure that you DO NOT fill in the ‘JSON Object Name’ parameter. You can see I left it blank in the configuration below. Lastly, provide a ‘Name’ and then provide a name for the ‘GeoEvent Definition’ (we will change this later on, so the name doesn’t matter). After you hit ‘Save’, go back and edit the input you just created. You should now see a URL that was generated. It should look something similar to this: https://MYSERVER.com:6143/geoevent/rest/receiver/disruptive-tech This will be the URL that you will provide to Disruptive Technologies to create your Adapter. So, copy it down. This newly created input is now just waiting for data/messages. Now, we need to go over to Disruptive Technologies Studio to set up an Adapter, which will send messages to GeoEvent. Disruptive Technologies Studio Disruptive Technologies is a company based out of Norway, which also has offices in the US. They create some of the world’s smallest IoT sensors and have a companion software platform that can be used to manage all of the sensors. You can learn more about them at their website: https://www.disruptive-technologies.com/  All of Disruptive Technologies sensors communicate, via gateways called Cloud Connectors, back to the Disruptive Technologies cloud. All data is end-to-end encrypted, the Cloud Connectors are plug-n-play with built-in cellular and the data is available both via a web application and standard APIs. The web application used to manage your fleet of sensors, set up integrations and give users access, is called Studio. Creating a Data Connector The standard cloud-to-cloud integration between Disruptive Technologies and third-party cloud services is done via a real-time push integration called a Data Connector.  For a quick introduction to Data Connectors, watch the video Data Connector in 50 seconds. With GeoEvent set up to receive sensor events, it is now time to set up the Data Connector that will send the data over to GeoEvent. The first step is to go to your project in Disruptive Technologies’ Studio, navigate to Data Connectors and press “Create new”. Next, fill in a name and, most importantly, the Endpoint URL which will be the URL that you saved from GeoEvent earlier. We only want to send the events which contain sensor data, so only check these event types, and press save. All sensor data from this Project will now be sent to GeoEvent. To verify that data is being sent and received correctly, you can open your new Data Connector and take a look at the “Activity Last 24 Hours”. Now, lets head back over to GeoEvent to finish defining our input and get some data on a map! GeoEvent - Creating the GeoEvent Definition We have our input defined in GeoEvent, we have the Data Connector created in Studio… now we just need to define the message format that GeoEvent will receive in the GeoEvent Definitions. Back in the GeoEvent Manager, you should now see that you are receiving messages on your input. These should be visible on the Monitor pane.  At the top of the GeoEvent Manager, select Site, and then select GeoEvent Definitions. You should see all of your input and output schemas defined here. For this blog, I created a sample ‘input’ schema that you can use which contains all possible fields (can download this at the bottom of the blog). Click ‘Configuration Store’, and then select Import Configuration. After you import the configuration, go back to GeoEvent Definitions, and you should see the new GeoEvent Definition that was just imported. Lastly, you need to go back to your input and change your setting for ‘Create GeoEvent Definition’ to NO and then select the name of the definition you just imported. Defining the Output After you get your input created, you need to send the data somewhere! With GeoEvent, you have a few different options available. You can either archive your data in the Spatiotemporal Big Data Store, or you can Update Features to provide a ‘real-time snapshot’ of your input data.  For this demonstration, I created a basic feature service in ArcGIS Online. Here are the fields I included: I could have included fields to capture the signal strength, transmission mode, etc, but my main goal was to capture the edit time, temperature, and a boolean for the object present sensor. Creating the GeoEvent Service The last step is stringing it all together. When you create a GeoEvent Service, you are basically creating the logic that connects your input to your output. In this GeoEvent Service, I created a Field Mapper processor, to link the Source and Target fields together.  The one caveat with this, is that the input is not sending any geographic data or coordinates as part of the message. So, you need to manually place the features in your map or edit their position after they get created. Now, whenever the sensors sends data, it will update the target fields you defined. Now you just need to build some awesome maps, apps, and dashboards! Real-time IoT and GIS If you have any questions around the configuration, please do not hesitate to reach out. The ability to integrate the Disruptive Technologies API with GeoEvent Server was incredibly seamless and honestly fun to do. I can’t wait to see how customers continue to deploy this technology. Brian Baldwin - bbaldwin@esri.com Niklas Norin - niklas.norin@disruptive-technologies.com ... View more

The goal of this document is to keep a running, updated list of the common technical questions that users are asking related to Esri's utility network. Please feel free to post questions to the bottom of the blog post and we will work on incorporating these answers into the updated post. There are already of wealth of resources available through the ArcGIS Pro help documentation and other sources. Please ensure your questions are not already answered on this site: What is a utility network? Can any device be choosen as a subnetwork controller?  For a partitioned network, a device with directional terminals is required, for a hierarchical network, devices with directional or bi-directional terminals is required.  The device requires the Subnetwork Controller Category and also be defined as a controller in the subnetwork definition.(revised 3/27/2019) Revision history: First Posted: 3/26/2019 Revised: 3/27/2019 Is there a way to create multiple connectivity associations at a time? Connectivity associations can be built into the migration framework using Data Interoperability or the FME workbenches.  The Import Associations geoprocessing tool can be used to load associations in bulk from a CSV file after the network has been created and features loaded. Revision history: First Posted: 3/26/2019 Revised: 3/27/2019 Where are group and pre-set templates stored? These templates are stored in the map inside an ArcGIS Pro Project.  The maps can be exported to Mapx and shared to other projects.  At this time, there is no process to export the templates to a file themself. Revision history: First Posted: 3/26/2019 Revised: 3/27/2019 If I need to make an update to the 'service territory' feature, what do I do? You cannot expand the extent of an existing Utility Network.  The easiest process is to export the Utility Network to an Asset Package.  In the Asset Package, expand or adjust the extent.  Create a brand new EGDB, stage a new Utility Network and apply the asset package.  Then republish your services.  If you use the same name or overwrite your services, this should not affect any downstream maps. Note: This will not bring any versioning information or history, so that information will be lost.   Revision history: First Posted: 3/26/2019 Revised: 3/27/2019 What domain values should be used? Should everything use unique domain codes? As a field may be used for different values across the asset group, it is best practice to not use the same code for different descriptions.  Example:  Layer: Water Network Device Asset Groups: Valve and Hydrant: Valid values for Valves 1: Open 2: Closed Valid values for Hydrants 3: In Service 4: Out of Service We could use the same code, of 1 for hydrants and use the description of In Service, but when we run a query or perform a trace and are looking to stop of open Valves, we would have to add a second part to that configure to ensure we are stopping at open valve and not open hydrants.  By ensure codes are unique, we can ensure we are only stopping or returning open devices, not other devices that happen to share the same code, but the descirption is completely different. Revision history: First Posted: 3/26/2019 Revised: 3/27/2019 I would like to support the utility network in my application, how should I interact with the layers. You will need to be able to interact with services.  Your software will interact with either a Feature Service(for read and write) or layers in a Map Service(Read only). We recommend you support a web map as the interface into those services.  The industry solution templates ship with a Network Viewer.  This network viewer is very similar and look and feel to the Editor maps we provide.  The difference is it is not using Subtype layers.  For each subtype in a subtype layer, a web map layer is used with an definition expression by subtype.   If you cannot use a web map and need to interact with a service, in the Utility Network solution template includes a simple network publishing map.  This map replaces the subtype layers with group layers with definition expressions per layer.  This is designed for a Map Service and cannot be published as a Feature Service.  You software can interact with the individual layers of this service, similar to the layers in a Feature Service. If you want to use the Utility Network feature service, you will have to take advantage of the Query Data Elements capabilities of the feature service to interact with the schema of that data Revision history: First Posted: 3/26/2019 Is there any way to validate the schema of the Asset Package without having to create an EGDB and apply it? We shared a simple pyt to help you validate the Asset Package, available here: https://community.esri.com/docs/DOC-13155-validate-asset-package Revision history: Moved from parking lot: 3/26/2019 Parking Lot: Please ask your questions in the comments and we will add them here and work on answers Can I change an Asset Group in the utility network? Can I change an Asset Type in the utility network? Can I add or remove fields in the utility network? How do you(Esri) design and develop the models?  Do you work in the asset package, do you work in scripts, do you work in the utility network? If I add an asset group in the Asset Package, what are my steps to ensure the schema is valid? ... View more