GeoEvent connectors (setting up a mockup)

539
6
Jump to solution
09-12-2019 10:33 AM
SatwikMishra
New Contributor II

I am a beginner and any help will be very much appreciated.

I want to set input and output connectors using the GeoEvent manager such that the input side receives text in XML format, and the output connector sends the converted JSON.

Using the manager I have set up a GeoEvent service where I have set the input and output connectors to different ports on the localhost. (Probably not the right way to go about, will try to set up servers and try, however just want to know if it's possible doing this simple way)

For example: input connector: https://<localhost ip>:3000/ (this returns XML)

output connector: https://<localhost ip>:2121/ (This will log the JSON on receiving from the output connector)

1. I need help in selecting the right connectors (poll XML from an external website or rest endpoint)

2. Should I make any changes to my js code?

3. Output connector side: Is the JSON sent in the body of the POST request to the target link?

```
//-----------XML Side------------

const express = require('express');
const srv = express();
const port = 3000;
let xml = `<note>
                     <to>Satwik</to>
                     <from>Harry</from>
                     <heading>Hi</heading>
                     <body>Sup, let us catchup.</body>
               </note>`

srv.get('/', (req, res) => {
   res.set('Content-Type', 'text/xml');
   res.send(xml);
});
srv.listen(port);

//----------------JSON Side---------

const express = require('express');
const srv = express();
const port = 2121;
const fs = require('fs');

const parser = require('body-parser');
srv.use(parser.urlencoded(express.json()));
srv.use(parser.urlencoded({extended:true}));

srv.post('/', (req, res) => {
   let jS = JSON.string(req.body);
   fs.writeFile("xml2js.json", jS);
   //fs.writeFile("xml2js_text.txt", jS);
   console.log(req.body);
});

srv.listen(port);

```

Update: Now, I have set up an AWS EC2 instance, and running the server there. On starting the input connector getting this error: 'Failed to get a response from <url>'. 

Reply
0 Kudos
1 Solution

Accepted Solutions
Stefan_Jung
Esri Contributor

Hi,

a good start could be to have a look at this blog post: https://community.esri.com/community/gis/enterprise-gis/geoevent/blog/2018/07/26/xml-data-structures... 

I've just created an input connector "Receive XML on REST Endpoint". I did not set the XML Object Name to "note", just leave it blank.

Next Step I used a REST Client to send some XML to the provided endpoint. You could also use Curl:

curl -X POST -k -H 'Content-Type: application/xml' -i 'https://hostname:6143/geoevent/rest/receiver/rest-xml-in' --data '<note><to>Satwik</to><from>Harry</from><heading>Hi</heading><body>Some Body</body></note>'

On the monitor site the count for the input connector should increase by 1 and a GeoEvent Definition ("fetch_xml_from_localhost") with should be created with the fields (type String): to,from, heading, body

You could also check the logfiles to make sure that no error occured.

Now the Incoming data has been transformed into geoevents and can be processed with GeoEvent Server.

There are a lot of output connectors to have a look at the data. Instead of directly sending the Data to another Server you could also use the output connector "Write to a JSON File". This will create a JSON File on the defined Folder. Just to see if the JSON is what you expeted. (Make sure to set the File Rollover Size to a higher value than 1kb).

If you have connected the Input and Output in the GeoEvent Service and everything Looks like exprected you could try to send it to your Endpoint.

So just do it step by step. If you want the GeoEvent to Pull the data instead, just try to change only the input and see if the data gets received, check the logmessages, check the output JSON etc.

Hope this helps.

Best,

Stefan

View solution in original post

6 Replies
Stefan_Jung
Esri Contributor

Hi Satwik Mishra‌,

I am not sure if you want help how to configure your GeoEvent Service (Input / Output connectors) or how to develop some mockups to send XML and receive JSON that should be accessed by the GeoEvent Server.

Receive XML on a REST Endpoint: GeoEvent is a server and provides a REST Endpoint and an external Tools can send XML to this endpoint. 

Poll an Exernal Website for XML:  GeoEvent is a client and polls the provided Endpoint in the given frequency.

In both cases you need a GeoEvent Definition that matches the incoming data. GeoEvent is going to transform the data into GeoEvents. So internally you should get the same GeoEvents in both Scenarios.

In general the output connector always sends data somewhere (push, write, send..) and you can define if it sould be JSON, XML, Text etc. 

It would be helpful if you provide some more Information. 

Best Stefan

SatwikMishra
New Contributor II

Hi Stefan P. Jung, I need help to develop a mockup (send XML from one server to the other and store it as a JSON file there). Currently, for the input connector, I am using GET request, but simply setting ip:port as the address is not working. (Planning to run in it on an EC2 server). In the geoEvent definition, I have added the fields.  It'd be great if you could tell me what exact information is needed. 

Reply
0 Kudos
Stefan_Jung
Esri Contributor

Hi,

a good start could be to have a look at this blog post: https://community.esri.com/community/gis/enterprise-gis/geoevent/blog/2018/07/26/xml-data-structures... 

I've just created an input connector "Receive XML on REST Endpoint". I did not set the XML Object Name to "note", just leave it blank.

Next Step I used a REST Client to send some XML to the provided endpoint. You could also use Curl:

curl -X POST -k -H 'Content-Type: application/xml' -i 'https://hostname:6143/geoevent/rest/receiver/rest-xml-in' --data '<note><to>Satwik</to><from>Harry</from><heading>Hi</heading><body>Some Body</body></note>'

On the monitor site the count for the input connector should increase by 1 and a GeoEvent Definition ("fetch_xml_from_localhost") with should be created with the fields (type String): to,from, heading, body

You could also check the logfiles to make sure that no error occured.

Now the Incoming data has been transformed into geoevents and can be processed with GeoEvent Server.

There are a lot of output connectors to have a look at the data. Instead of directly sending the Data to another Server you could also use the output connector "Write to a JSON File". This will create a JSON File on the defined Folder. Just to see if the JSON is what you expeted. (Make sure to set the File Rollover Size to a higher value than 1kb).

If you have connected the Input and Output in the GeoEvent Service and everything Looks like exprected you could try to send it to your Endpoint.

So just do it step by step. If you want the GeoEvent to Pull the data instead, just try to change only the input and see if the data gets received, check the logmessages, check the output JSON etc.

Hope this helps.

Best,

Stefan

View solution in original post

SatwikMishra
New Contributor II

Thanks a lot Stefan P. Jung‌. For me, it worked when I set the XML object name to 'note', without that I was getting null. I have another doubt, when I run the Curl command from my EC2 instance, it can't locate the https://hostname:6143/geoevent/rest/receiver/rest-xml-in, but I can do that from localhost. Is there a way I can access the rest endpoint from the ec2 instance?

Reply
0 Kudos
Stefan_Jung
Esri Contributor

Hey Satwik Mishra‌,

the Endpoint is only available if you create a REST Input Connector with the Name "rest-xml-in".

You could open https://[your-hostname]:6143/geoevent/rest in a WebBrowser to see which Input connectors are available.

In my case the list of Inputs contains:

  • rest-xml-in (esri-receive-xml-rest-in)

Klicking on the link gives me some detailed Information About the Input. The Endpoint you sould send data to is named "POST Content to Url". You will also see the same Link when Opening the Input Connector in the GeoEvent Manager.

Best,

Stefan

SatwikMishra
New Contributor II

Thanks, I had used the link mentioned against 'POST Content to Url' in curl cmd from my ec2 server, it somehow can't resolve the host. I am able to send the XML to the ArcGIS rest endpoint from the systems connected to my private network only. 

Reply
0 Kudos