SOAP Input

4332
3
09-08-2015 03:41 PM
DarrylWilson
New Contributor II

Hi there, Is it possible to add an input to the GEP consume data from a SOAP service?

has anyone created a custom adapter to achieve this?

Are there any gotcha's?

Best Regards

Darryl

3 Replies
RJSunderman
Esri Regular Contributor

Hello Darryl -

I don't have any references for you of someone who has developed a custom adapter for handling a response from a request made on a SOAP service. GeoEvent is fundamentally RESTful, meaning that we interact with services using REST requests. You might take a look at the thread Is it possible to send a SOAP request with GeoEvent?

For background, here's what I know about the differences between REST and SOAP.

  • In a REST query you might have some JSON (JavaScript Object Notation) in the body of the request, but often you do not. This is because everything you need to communicate to the service being queried is contained in the request’s URL as parameters. In this way a REST call is sort of like invoking a function with parameters you pass in the function call.

  • SOAP is fundamentally different. It is XML based – as opposed to REST which more “object-oriented”, using JSON “objects” to pass information. When making a SOAP request you always have some sort of XML body which contains the authentication and request parameters. You get back an XML response which you then have to parse. SOAP fell out of favor primarily because XML parsing is considered expensive.

A JSON response from a REST service request would, of course, have some sort of header. But the response is primarily a single object or a list of objects expressed as JSON. This object (or list of objects) returned from your request is something GeoEvent would consider an event (or a list of events).

XML can also be used to represent an event or a list of events – and we introduced the XML adapter in the 10.3 product release to handle that sort of message content. So a SOAP / XML response might be something the XML adapter out-of-the-box could parse, but since the XML is not “object-oriented” (like what we usually see with JSON) there is going to be quite a lot of work needed to delve into the XML response you receive and pull out the event data you are looking for.

The amount of work required would probably make using an out-of-the-box GeoEvent Service filters and processors impractical. You would probably need to develop a custom processor if you wanted to process a SOAP response to extract event data within a GeoEvent Service. Alternatively you could develop a custom adapter to parse the SOAP / XML response and construct a set of GeoEvents your adapter would then pass into your GeoEvent Service for some basic filtering and/or processing. Which brings us back to your original question, has anyone done this?

So … the recommendation is to avoid services which use SOAP. In addition to a custom processor or custom adapter you will probably also have to develop a custom transport to handle service polling, since you will need to construct the XML body the SOAP service expects to authenticate and communicate your request. This is definitely not something you can just configure GeoEvent to do. More than likely it is going to be a development project in which you use the GeoEvent SDK to develop custom components and extend the product to support SOAP – not in general – but for a specific service which you want to poll for data.

Hope this information helps -

RJ

GuillaumeBARRAS
New Contributor II

Hello,

I'm working on something very similar : I need to request a SOAP WebService to get vehicule current locations on the field in order to update a FeatureService who will be consume by sereral GIS applications (ArcMap, Operation Dashboard, Web/Mobile applications and so forth).

I managed to request one operation of my WebService with the "Poll an External Website for XML" Input connector. My problem is that a need to make 3 operations on that WebService to achieve my target (1/ onpening a session to get a token, 2/ request locations with the token in parameter, 3/ close the session). And as far as I can see it doesn't look possible with out of the box GeoEvent Extension components. Please can you confirm it RJ Sunderman​​ ?

I'll take a look at the Geovent Extension SDK but my first feeling is that maybe it's easier to develope an external WebService/Handler/batch treatment who will manage to request the Website (with a SOAP Toolkit :-)) and generate an output who could directly be consume by GeoEvent (like JSON, CSV file, ...).

thanks for your help.

Guillaume

Reply
0 Kudos
RJSunderman
Esri Regular Contributor

Hello Guillaume -

I don't think you will be able to use GeoEvent, out-of-the-box, to do what you are trying to do.

While the SOAP protocol does use XML to structure its requests and responses, SOAP messages are not the type of event structure GeoEvent is expecting when XML is used to format event data provided by a real-time data stream. You might configure a 'Poll an External Website for XML' to poll a SOAP service, but the GeoEvent's input will only make a single attempt to parse the response it receives.

When a GeoEvent input polls an external server, it expects a single response to its request, consistent with the architectural style of REST. The data structure, or byte payload, sent to GeoEvent in response to its request is parsed by the input's adapter (a transport is responsible only for delivering the byte payload to the adapter). The adapter expects a specific format (JSON, XML, Delimited Text) and uses a GeoEvent Definition to take attribute values from the response and construct a GeoEvent object.

There is no provision in the event ingest workflow described above to perform multiple actions or operations with a data provider's service. Integrating a SOAP toolkit into your solution to handle the token acquisition and data request is one approach. You also might be able to use the GeoEvent SDK to implement a custom transport which will handle the token acquisition, data request, and session closure. This might be preferable if the SOAP toolkit uses Java.

If you plan on using an external batch script to handle the service request and write the data to a system file for GeoEvent to then ingest, please be aware that there are several problems inherent when working with file-based input. One example is that GeoEvent may attempt to acquire a handle to a file (and begin reading the file) before the external process is finished writing the file to disk. Another example is related to the issue above; GeoEvent needs to read the contents of a file into memory in “chunks” and may encounter a problem emulating a data stream when retrieving blocks of event data from a system file.

Inputs which watch a system folder for files are generally intended to prove that event filtering and real-time analytics you have designed into a GeoEvent Service behave as intended. Files give you the ability to repeatedly send a small sample of event data to GeoEvent to test the behavior of a GeoEvent Service before transitioning to a production configuration in which real-time data feeds arrive via a stream - such as an HTTP/POST from an external server or as a reply to a query made on an external server's URL.

Sorry this reply is coming to you so late.

Hope that it is of some help.

- RJ