Create a custom connector - Receive XML via TCP

4169
2
09-14-2015 11:36 AM
SharonMeier
New Contributor III

RJ,  I have question related to an XML input connector.  I've created an input connector to receive data over a TCP port and am trying to get that data in XML.   I don't seem to be getting any data, though.   Any thoughts?

0 Kudos
2 Replies
RJSunderman
Esri Regular Contributor

Hello Sharon -

What you are trying to do is definitely possible. I'm limited in my options for sending data to GeoEvent via TCP ... basically I have to use the GeoEvent Simulator ... but I can walk through what I did with you.

The GeoEvent Simulator normally sends delimited text, specifically comma separated values. But that's not some hard-coded property or behavior. We could use it to send XML - we would just have to configure a GeoEvent input to expect the data to arrive via TCP and couple the TCP transport with the XML adapter so the input would know how to interpret the data.

I've attached an input file (FeatureFlightsXML.csv) you can use as an example. Ignore the *.csv extension ... it's only there because the GeoEvent Simulator is looking for files named *.csv ... the file contains XML with each event on a separate line. This is necessary because the Simulator is sending data one line of text at a time, so we have to account for that.

<Flight><FlightNumber>SWA2706</FlightNumber><StartTime>3/16/2012 02:25:30 PM</StartTime><OriginAirportCode>IAD</OriginAirportCode><DestinationAirportCode>TPA</DestinationAirportCode><AircraftType>B733</AircraftType><Altitude>37000</Altitude><Longitude>-79.585739</Longitude><Latitude>34.265521</Latitude></Flight>
<Flight><FlightNumber>SWA724</FlightNumber><StartTime>3/16/2012 02:25:30 PM</StartTime><OriginAirportCode>IAD</OriginAirportCode><DestinationAirportCode>ABE</DestinationAirportCode><AircraftType>SF34</AircraftType><Altitude>10000</Altitude><Longitude>-76.405289</Longitude><Latitude>39.573271</Latitude></Flight>
<Flight><FlightNumber>SWA992</FlightNumber><StartTime>3/16/2012 02:25:30 PM</StartTime><OriginAirportCode>IAD</OriginAirportCode><DestinationAirportCode>MDW</DestinationAirportCode><AircraftType>B737</AircraftType><Altitude>36400</Altitude><Longitude>-82.067414</Longitude><Latitude>39.630957</Latitude></Flight>
<Flight><FlightNumber>SWA2358</FlightNumber><StartTime>3/16/2012 02:25:30 PM</StartTime><OriginAirportCode>TPA</OriginAirportCode><DestinationAirportCode>IAD</DestinationAirportCode><AircraftType>B733</AircraftType><Altitude>37000</Altitude><Longitude>-81.20226</Longitude><Latitude>32.099263</Latitude></Flight>

I've also attached a GeoEvent configuration file (Xml-to-JSON-Config.xml) which has a custom inbound connector I configured, pairing the XML adapter with the TCP transport. The configuration includes three inputs: one expects delimited text via tcp, one expects to receive XML via a REST endpoint, and the third is my custom input which expects XML via TCP. The output included in the configuration expects to be able to write generic JSON to a file it can create in the C:\GeoEvent\output folder on your local server.

The GeoEvent Service XML-to-JSON feeds data from each input to the file-json-out output. Its the job of the adapters of each input to create GeoEvents from the data they receive. The output will deconstruct each GeoEvent to create generic JSON and will write that JSON to the system file.

The tcp-text-in input is what I used to verify that I can use the GeoEvent Simulator to send XML. I created a  GeoEvent Definition with a single field of type String. When I send an event from the simulator to TCP port 5565, the tcp-text-in input interprets the received data using the Text adapter and brings the data in as one long String.

The rest-xml-in input is what I used to verify that I was sending valid XML. I configured this input to create a GeoEvent Definition for me. When I select any one line of data from the simulation file and HTTP / POST it to the rest-xml-in input's endpoint, I can verify that the received data can be interpreted using the XML adapter by seeing a GeoEvent get created and written out to the system file.

The custom-xml-via-tcp input is what you're looking for. I configured this input to use TCP port 5566 (rather than the default 5565). So all I have to do is tell the GeoEvent Simulator to disconnect from the default port and reconnect on port 5566 ... then I can send the same event data one line at a time to the custom input, which will interpret the received data using the XML adapter, create GeoEvents, and write the data out to the system file.

Hope this helps -

RJ

0 Kudos
MatthewLewis3
New Contributor II

Hi RJ, the example you have provided for TCP is very thorough.

What is advised if a user needs to send XML using UDP. I have tried building a custom connector using UDP transport and the XML adaptor a similar format to the TCP example. However I cannot seem to get GeoEvent to pick up the packet sent to the server. I have written a custom python script as my "Simulator".

MESSAGE = "<Flight><FlightNumber>SWA2706</FlightNumber><StartTime>3/16/2012 02:25:30 PM</StartTime><OriginAirportCode>IAD</OriginAirportCode><DestinationAirportCode>TPA</DestinationAirportCode><AircraftType>B733</AircraftType><Altitude>37000</Altitude><Longitude>-79.585739</Longitude><Latitude>34.265521</Latitude></Flight>"

from socket import *
sq = socket(AF_INET, SOCK_DGRAM)
sq.connect(("10.0.0.8", 5000))
print(sq.getsockname()[0])
sq.sendto(MESSAGE, ("10.0.0.8", 5000))‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Can you advise on the best way to achieve this?

0 Kudos