Enhancements to the GeoEvent Server's outbound HTTP transport (10.5)

[RJSunderman]

This article is the second of two articles examining enhancements made to the HTTP transport for the GeoEvent Server 10.5 release. This article examines the outbound transport. The previous article examining the inbound transport can be found here.

In this article, I would like to provide detail for an enhancement made to the HTTP outbound transport for the GeoEvent Server 10.5 release. The following capability is listed on the What's new in ArcGIS GeoEvent Server web help page:

• HTTP outbound transport now supports field value substitutions in the HTTP GET mode

Beginning with the 10.5 product release, an output leveraging the HTTP transport can be configured to substitute event attribute values into the URL of a request GeoEvent Server will send to an external server. The attribute values are incorporated as query parameters (as opposed to the request’s content body).

The new capabilities of the HTTP transport will be described below with exercise steps you can follow to demonstrate the capabilities.

~~~

When you want to send data from event records to an external server or application you typically configure an outbound connector – such as the Push JSON to an External Website output. GeoEvent Server will incorporate the event data into the content body of a REST request and send the request to the external server as an HTTP/POST. This capability has been available in the last several releases.

A device on the edge of the Internet of Things, however, might prefer to receive requests with event data organized as query parameters rather than in a request's content body. This way the entire data payload is in the URL of the request -- leaving the content body of the request empty.

It might seem a little odd for a GeoEvent Server output, which is not intended to receive or process any type response, to make an HTTP/GET request. But the capability was introduced to enable GeoEvent Server to issue activation requests to devices which require data values be sent using query parameters.

~~~

Exercise 2A – Use HTTP/GET to send event data as query parameters to an external server

Why exactly are we configuring a custom outbound connector?

How's it different than the the Push JSON to an External Website connector available out-of-the-box?

For this exercise:

1. Configure the following GeoEvent Server output connector.
   Browse to Site > GeoEvent > Connectors and select to create a new outbound connector. Default values for the "Shown", "Advanced", and "Hidden" properties are included beneath the illustration.
   
   
   

   Shown Properties	Default Value
   URL	[ no default value defined ]

   Advanced Properties	Default Value
   Use URL Proxy	False
   URL Proxy	[ no default value defined ]
   HTTP Timeout (in seconds)	30

   Hidden Properties	Default Value
   Formatted JSON	False
   MIME Type	text/plain
   Acceptable MIME Types	text/plain
   Post/Put body MIME Type	text/plain
   Parameters	[ no default value defined ]
   Header Parameter Name:Value List	( blank )
   HTTP Method	Get
   Mode	Client

2. Save your newly configured custom outbound connector.
3. Navigate to Services > Outputs and select to create a new (Custom) HTTP/GET request with event data as query parameters output. Configure the output as illustrated below, replacing yourServer and yourDomain with a valid server and domain for your organization.
   
   
   Note the URL specified in the illustration:
   

   https ://yourServer.yourDomain/server/rest/services/SampleWorldCities/MapServer/0/query?where=city_name='${Origin}'&f=json
   

   
   The format of the URL assumes that an ArcGIS web adapter (named 'server') has been configured and that an external server or client application receiving this URL could use it to query the "Sample World Cities" map service on your ArcGIS Server. GeoEvent Server will substitute the variable ${Origin} in the URL's query parameter with an actual attribute value from a received event record, enabling the external server or client application to make a more specific query based on real-time events. 
4. Save your updated output, then publish a GeoEvent Service which incorporates your output and an input of your choice. You can use any type of input, so long as the GeoEvent Definition associated with event records received by the input includes an attribute field named Origin.
   
   Queries through a web adapter to a Portal secured web service from an unauthenticated source will return an error. Since the Sample World Cities web service is secured by Portal in my current deployment, I expect the request made by GeoEvent Server will generate an error. In order to complete the demonstration we will use the GeoEvent Server's debug logs to confirm that the output has constructed a valid query and sent the request to the ArcGIS Server map service.
   
5. Navigate to the Logs page in GeoEvent Manager. Click 'Settings' and enable DEBUG logging for the feature service outbound transport logger (com.esri.ges.transport.http.HttpOutboundTransport).
6. Send an event record to your GeoEvent Server input whose Origin attribute is the name of one of the cities in the Sample World Cities map service (e.g. Chicago). Refresh the Logs page in GeoEvent Manager and you should see log messages with information similar to the following:
   
   

• com.esri.ges.transport.http.HttpOutboundTransport
  
  • https ://caramon.esri.com/server/rest/services/SampleWorldCities/MapServer/0/query?where=city_name='... Request succeeded
  • {"error":{"code":499,"message":"Token Required","details":[]}}

The first message shows that 'Chicago' was indeed substituted into the query parameters by the GeoEvent Server output and a request was made. The error may or may not be displayed; as indicated above, the map service in my case is Portal secured and this request did not include a token authenticating the request.

There are a couple of things you'll want to keep in mind. The URL you use to configure the the output must URL Encode its query parameters to make them HTTP safe. But the value is being substituted by GeoEvent Server is based on a string received from a real-time data source. This means you may have some work to do to make sure that "San Francisco" is represented as San%20Francisco not San Francisco before an event record is sent to an output.

Also, the enhancement being introduced in this article was designed specifically for HTTP/GET since those requests do not include a JSON payload in the request’s body. However, some rudimentary testing suggests that you can use HTTP/POST as well; I suppose it would be up to the external server receiving the request whether or not to honor an HTTP/POST and either ignore the request’s JSON payload or potentially consider its content in addition to the values in the query parameter.

Finally, you do have some freedom in how the request’s query string is specified. For example, you could construct a parameterized string something like; GeoEvent Server will handle the substitution of the multiple parameter values:

query?where=city_name+IN+%28%27${CityA}%27%2C%27${CityB}%27%29&f=json

If you send the string highlighted above through an HTML decoder you'll see that it is equivalent to:

where=city_name IN ('${CityA}','${CityB}')&f=json

I hope these two blogs were helpful.  Please comment below with questions and I'll do my best to answer them.

-- RJ