Scripting tasks using the GeoEvent Admin API - Update Inputs

1916
0
03-21-2019 09:11 AM
EricIronside
Esri Regular Contributor
4 0 1,916

As RJ Sunderman mentioned in his blog, automating administration of the GeoEvent Server using python scripts can be a powerful way to augment the way you manage your real-time system (such as stopping outputs before a reboot or starting those outputs after a restart). a blog post by Andy Ommen has an excellent description of how to do this (I've even updated his script here) - Scripting tasks using the GeoEvent Admin API

Scripting can also be useful for those times when GeoEvent needs a bit of help. For example if a property on an input or output must be periodically updated. One of those situations recently arose when a user's AVL provider required a URL Parameter to supply a timestamp for the data being requested.  It is this scenario that I'll use as an example of how to modify properties of a GeoEvent component using a script.  This methodology could be used to update any of the editable properties on any of your GeoEvent components that allow modification, as Jake Skinner shows in his article on updating a token.

An Input's "since" property...

An AVL provider supplies a HTTP RESTful API for polling vehicle data in JSON format.  Yeah! The good news is that GeoEvent provides an Input Connector that does exactly that.  But on further reading you find that the API expects the polling request to include a 'since' parameter in the URL. An example of the expected format is below:

https://avl.trackingcompany.com/api1/json/currentLocations?user=john&pw=secret&since=2012-10-08T09:20:30Z 

Unfortunately, GeoEvent has no way to automatically update that date next to the 'since' parameter.  So without any other solution, you'd be stuck manually updating the GeoEvent Input's properties every day (or every hour). What we need is a way to set that timestamp to something recent, so we can reduce the amount of duplicate data we recieve and reduce the load on both servers.

The GeoEvent Admin API to the rescue...

Fortunately, the GeoEvent Admin API allows us to update an input programatically and we can automate this using a little python scripting.  To see what is offered from the GeoEvent Admin API, you can view the Swagger documentation on your GeoEvent Server by going to the following URL.

https://<YourGES>:6143/geoevent/admin/api/index.html 

Using this API we can do operations like get, add, update, delete, start, and stop on most of the compents within GeoEvent.  So conceptually, to solve the problem above, we just need to:

  1. get the input with the 'since' parameter,
  2. set the value of that parameter,
  3. and update the input. 

Step by Step

Create your GeoEvent Input

In GeoEvent create a new 'Poll an external website for JSON' Input.  Be sure to split the URL path from the parameters by using the Input's 'URL' and 'Parameters' properties.  URL Paramters are any part of the URL after the '?'. For example

   Given the URL:

            https://myserver.domain.com/api1/json/gateway?user=john&since=2019-03-21T14:22:12Z&pw=se...

   Your Input properties should look like this:

   

   

Identify your GeoEvent Input

By default the script will look through all inputs for URL Parameter properties containing a 'since' parameter.  You can help the script narrow its search by providing the 'Name' of the input(s) you want it to look at.  To do this, go into your GeoEvent's REST end poing and select your input.

         https://<yourGES>:6143/geoevent/rest/ 

After you click on your input, you will see the list of properties.  Note and/or Copy the value for the 'Name' property.

So in this case, we want to copy and save the 'Name' value "a0e151b8-0c00-4c12-838d-16cdefb36ba8".

Set up the script

You can get the script from this GitHub repository. You want to download the file 'UpdateGEEInputURLParam.py'.

Once downloaded, the following parameters will need to be set before the script will run properly.

  • server_fqdn – This is the fully qualified domain name of the server where GeoEvent is installed.
    • Example: server.domain.com
  • username – This is the username for the GeoEvent Server Manager login credentials
  • password – This is the password for the GeoEvent Server Manager login credentials
  • geeTargetInputList – This is a list of ‘name’ strings (leave empty if you want the script to look at all inputs). This is where you'd paste (inside of quotes) the 'Name' parameter of your input above.

Other script parameters you may update

  • urlParamName – This is the property on the URL Parameters you want to change.
    • The default is ‘since’ to match our context, but you might change this to anything that matches your use case.
    • Examples: timeSince, token, startTime, ...
  • time_offset_sec – This is the amount of time to add (or subtract if you supply a negative number) to the current time.
    • Positive numbers will add time the current time. Example: now() + 30 seconds. 
    • Negatvie numbers will subtract time from the current time. Example: now() - 60 seconds.
    • The default is -60 seconds (subtract 1 minute from the current time stamp)

Script constants

There are a number of constants that follow the parameters above that shouldn't be modified in most cases.  However, if you intend to extend this script, or have a non-default installation of GeoEvent, you might need to add or modify some of these.

Running the script

Installing

The script requires an external library to be installed into your Python before it will run correctly.  The library is located here in GitHub, but to install, all you need to do is the folloiwing:

  1. Open a command prompt on your machine
  2. Navigate (cd) to the scripts directory in your Python2.7 install (usually this is C:\Python27\ArcGISx6410.6\Scripts)
  3. Type ‘pip install arcrest_package’ and hit enter

Testing

If you are editing the script in an IDE you will be able to run the script there.  Outside of the IDE, you will want to be sure that Python is installed on your machine and is executable (on Windows, you can register .py files to run as python.exe files, or you can add the folder containing the python.exe to your system's PATH variable). This script can be run/tested on any machine that has network access to the GeoEvent machine.

As the script runs, it will print out log messages indicating the input(s) it is modifying:

         Looking at clientParameters on input named 'a0e151b8-0c00-4c12-838d-16cdefb36ba8'
         Updating clientParameters 'since' parameter value =2019-03-21T04:43:46Z to 2019-03-21T14:22:12Z
      Updated clientParameters 'since' parameter on input named a0e151b8-0c00-4c12-838d-16cdefb36ba8

Checking your Inputs, the Parameter property should be updated and the value of the 'since' parameter will be the current time minus 1 minute.

Scheduling

You can use a scheduler (such as Windows Task Scheduler) to run your script at a specified time or interval (e.g. hourly).  

About the Author
Esri Professional Services Real-Time GIS Team GeoEvent Sr. Product Enginner