Latest Contributions by RJSunderman

Hello Sharon - What you are trying to do is definitely possible. I'm limited in my options for sending data to GeoEvent via TCP ... basically I have to use the GeoEvent Simulator ... but I can walk through what I did with you. The GeoEvent Simulator normally sends delimited text, specifically comma separated values. But that's not some hard-coded property or behavior. We could use it to send XML - we would just have to configure a GeoEvent input to expect the data to arrive via TCP and couple the TCP transport with the XML adapter so the input would know how to interpret the data. I've attached an input file (FeatureFlightsXML.csv) you can use as an example. Ignore the *.csv extension ... it's only there because the GeoEvent Simulator is looking for files named *.csv ... the file contains XML with each event on a separate line. This is necessary because the Simulator is sending data one line of text at a time, so we have to account for that. <Flight><FlightNumber>SWA2706</FlightNumber><StartTime>3/16/2012 02:25:30 PM</StartTime><OriginAirportCode>IAD</OriginAirportCode><DestinationAirportCode>TPA</DestinationAirportCode><AircraftType>B733</AircraftType><Altitude>37000</Altitude><Longitude>-79.585739</Longitude><Latitude>34.265521</Latitude></Flight> <Flight><FlightNumber>SWA724</FlightNumber><StartTime>3/16/2012 02:25:30 PM</StartTime><OriginAirportCode>IAD</OriginAirportCode><DestinationAirportCode>ABE</DestinationAirportCode><AircraftType>SF34</AircraftType><Altitude>10000</Altitude><Longitude>-76.405289</Longitude><Latitude>39.573271</Latitude></Flight> <Flight><FlightNumber>SWA992</FlightNumber><StartTime>3/16/2012 02:25:30 PM</StartTime><OriginAirportCode>IAD</OriginAirportCode><DestinationAirportCode>MDW</DestinationAirportCode><AircraftType>B737</AircraftType><Altitude>36400</Altitude><Longitude>-82.067414</Longitude><Latitude>39.630957</Latitude></Flight> <Flight><FlightNumber>SWA2358</FlightNumber><StartTime>3/16/2012 02:25:30 PM</StartTime><OriginAirportCode>TPA</OriginAirportCode><DestinationAirportCode>IAD</DestinationAirportCode><AircraftType>B733</AircraftType><Altitude>37000</Altitude><Longitude>-81.20226</Longitude><Latitude>32.099263</Latitude></Flight> I've also attached a GeoEvent configuration file (Xml-to-JSON-Config.xml) which has a custom inbound connector I configured, pairing the XML adapter with the TCP transport. The configuration includes three inputs: one expects delimited text via tcp, one expects to receive XML via a REST endpoint, and the third is my custom input which expects XML via TCP. The output included in the configuration expects to be able to write generic JSON to a file it can create in the C:\GeoEvent\output folder on your local server. The GeoEvent Service XML-to-JSON feeds data from each input to the file-json-out output. Its the job of the adapters of each input to create GeoEvents from the data they receive. The output will deconstruct each GeoEvent to create generic JSON and will write that JSON to the system file. The tcp-text-in input is what I used to verify that I can use the GeoEvent Simulator to send XML. I created a  GeoEvent Definition with a single field of type String. When I send an event from the simulator to TCP port 5565, the tcp-text-in input interprets the received data using the Text adapter and brings the data in as one long String. The rest-xml-in input is what I used to verify that I was sending valid XML. I configured this input to create a GeoEvent Definition for me. When I select any one line of data from the simulation file and HTTP / POST it to the rest-xml-in input's endpoint, I can verify that the received data can be interpreted using the XML adapter by seeing a GeoEvent get created and written out to the system file. The custom-xml-via-tcp input is what you're looking for. I configured this input to use TCP port 5566 (rather than the default 5565). So all I have to do is tell the GeoEvent Simulator to disconnect from the default port and reconnect on port 5566 ... then I can send the same event data one line at a time to the custom input, which will interpret the received data using the XML adapter, create GeoEvents, and write the data out to the system file. Hope this helps - RJ ... View more

Hello Adriana - I was assuming that you were using the Esri/geoevent-datastore-proxy. Is this true, or are you using the Esri/resource-proxy? The former was developed specifically for GeoEvent users who are unable to adopt the latest release of GeoEvent. Specifically it provides token management for ArcGIS Server connections registered with GeoEvent as Data Stores. Users who must remain at the 10.2.2 or 10.3.0 release who need functionality built-in to the GeoEvent 10.3.1 release can find what they need by using the GeoEvent Data Store proxy. If you're using the .NET version of the "resource-proxy" to enhance a web application, I don't think we'll be able to help; this forum is specific to the use of the ArcGIS GeoEvent Extension for Server. If you're using the GeoEvent Data Store proxy, but are not using GeoEvent, that would be an unsupported configuration. Either way, the GeoEvent product team does not have the experience you are looking for. I think you are correct that this may be a security, proxy, JavaScript API related problem. You might have better luck if you take your question to another forum. Perhaps: ArcGIS API for JavaScript​​, or Web AppBuilder for ArcGIS​. or Web AppBuilder Custom Widgets​ Good Luck - RJ ... View more

Hello Gal - http://www.ims.gov.il/ims/PublicXML/observ.xml I'm not seeing any problem with the XML returned from the above URL. The GeoEvent's XML adapter is able to parse it just fine. I've attached illustrations of my 'Receive XML on a REST Endpoint' input configuration and the generated GeoEvent Definition. I chose to have the adapter look specifically for <surface_station> nodes, rather than receiving the <israel_surface_data> structure in as a single event. I used a Chrome Poster plug-in to HTTP / POST the attached XML to the GeoEvent input's REST endpoint. What I did notice was that, when using a 'Poll an External Website for XML' input, there were errors logged by the HTTP inbound transport to the effect that requests to http://www.ims.gov.il/ims/PublicXML/observ.xml returned an HTTP 406 'Not Acceptable' error. 2015-09-15 11:50:58,156 | ERROR | gov.il/ims/PublicXML/observ.xml] | HttpInboundTransport             | nsport.http.HttpInboundTransport  456 | 272 - com.esri.ges.framework.transport.http-transport - 10.3.1 | http://www.ims.gov.il/ims/PublicXML/observ.xml: Request failed (HTTP/1.1 406 Not Acceptable). I'll ask to see if one of the developers can tell me why the request to your server are being returned as 'Not Acceptable', but it doesn't appear to be an issue with the XML formatting or content. - RJ ... View more

Lloyd - If you navigate to Site > GeoEvent > Connectors and search for "XML" ... you are not seeing the following two inbound connectors? These should be out-of-the-box at the 10.3.0 release. What do you see when browsing to the URL https://localhost:6143/geoevent/manager/version.txt (You should see the product version 10.3.0.4322) Are you able to locate the XML adapter's JAR file (xml-adapter-10.3.0.jar) down beneath the framework? C:\Program Files\ArcGIS\Server\GeoEvent\system\com\esri\ges\framework\adapter\xml-adapter\10.3.0 - RJ ... View more

Hello Darryl - I don't have any references for you of someone who has developed a custom adapter for handling a response from a request made on a SOAP service. GeoEvent is fundamentally RESTful, meaning that we interact with services using REST requests. You might take a look at the thread Is it possible to send a SOAP request with GeoEvent?​ For background, here's what I know about the differences between REST and SOAP. In a REST query you might have some JSON (JavaScript Object Notation) in the body of the request, but often you do not. This is because everything you need to communicate to the service being queried is contained in the request’s URL as parameters. In this way a REST call is sort of like invoking a function with parameters you pass in the function call. SOAP is fundamentally different. It is XML based – as opposed to REST which more “object-oriented”, using JSON “objects” to pass information. When making a SOAP request you always have some sort of XML body which contains the authentication and request parameters. You get back an XML response which you then have to parse. SOAP fell out of favor primarily because XML parsing is considered expensive. A JSON response from a REST service request would, of course, have some sort of header. But the response is primarily a single object or a list of objects expressed as JSON. This object (or list of objects) returned from your request is something GeoEvent would consider an event (or a list of events). XML can also be used to represent an event or a list of events – and we introduced the XML adapter in the 10.3 product release to handle that sort of message content. So a SOAP / XML response might be something the XML adapter out-of-the-box could parse, but since the XML is not “object-oriented” (like what we usually see with JSON) there is going to be quite a lot of work needed to delve into the XML response you receive and pull out the event data you are looking for. The amount of work required would probably make using an out-of-the-box GeoEvent Service filters and processors impractical. You would probably need to develop a custom processor if you wanted to process a SOAP response to extract event data within a GeoEvent Service. Alternatively you could develop a custom adapter to parse the SOAP / XML response and construct a set of GeoEvents your adapter would then pass into your GeoEvent Service for some basic filtering and/or processing. Which brings us back to your original question, has anyone done this? So … the recommendation is to avoid services which use SOAP. In addition to a custom processor or custom adapter you will probably also have to develop a custom transport to handle service polling, since you will need to construct the XML body the SOAP service expects to authenticate and communicate your request. This is definitely not something you can just configure GeoEvent to do. More than likely it is going to be a development project in which you use the GeoEvent SDK to develop custom components and extend the product to support SOAP – not in general – but for a specific service which you want to poll for data. Hope this information helps - RJ ... View more

Hello Adriana - Are you using a GeoEvent output -- either the Add a Feature output or the Update a Feature output -- to make edits to your feature service? If so, you can review the Debugging the Add a Feature / Update a Feature Output Connectors​ blog to see how to configure GeoEvent to log debug messages which should tell you something about the JSON being sent in the requests to Server and the responses returned from Server. If you are not using GeoEvent and the application you are developing is making its own requests on a feature service's .../FeatureServer/0/addFeatures or .../FeatureServer/0/updateFeatures REST endpoint, you might want to ask your question on the Web AppBuilder for ArcGIS​​ forum. Someone there can provide you advice on developing your web application. Hope this information helps - RJ ... View more

Jon - Thank you for your feedback and taking the time to submit this enhancement as an idea. The product team has created a work item with the following proposed acceptance criteria. You can refer to this as item #960 in the product team's GitHub repository if you want to provide additional feedback or inquire as to the enhancement's status. The property UI will have a new dropdown property for Update Sequence Time Field. From the dropdown, user can choose an existing esriFieldTypeDate attribute from the the feature layer. If the new property is left unspecified, the behavior will be as it is today; the date/time an event is received will drive whether or not a request is made by GeoEvent to update a feature in the targeted feature class. If the user chooses a field (e.g. a feature attribute named 'timefield') the connector will do the following: -- A -- Update its internal cache when an event with a given TRACK_ID is received only if the event's 'timefield' value is greater than an event record already in the cache. -- B -- When iterating through its internal cache to create a transaction request, consider not only each event's TRACK_ID but also the event's 'timefield' value so that features in the targeted feature class are only updated if the event's date/time is greater than the date/time of the existing feature. ... View more

Hello Evan / Greg - I would remove the line specifying the attribute field names from the CSV file you are writing into the folder the 'Watch a folder for new .csv files' inbound connector is watching ... better to include only the actual event data in the file and leave out the attribute field names. In a reply to Greg, Evan cross-references an older thread, watching a folder with csv files, in which I state: "the first two lines of the CSV file are reserved for a comma separated list of field name and a comma separated list of field data types - your actual event data needs to start with the 3rd line of the text file." But then in this thread I suggest the opposite, to remove specification of attribute field names from the CSV file. So which is it, you're asking. Should a CSV file include a two-line header with field names and types, or shouldn't it? My apologies for the confusion. In the older thread I must have been thinking of when a CSV file is used as the source of event enrichment, not when the CSV file is used as the source of the event data. An enrichment file would need the header rows to specify field names and field types. You would not want a CSV input file to include anything more than the (comma) delimited data values. Evan is correct, above, when he suggests that the OS level user responsible for running the GeoEvent Windows service is the account which needs permission to access a registered folder. I don't remember if it was at 10.2.2 or a later release that we added visual indication of the apparent permissions on a registered folder to the Site > GeoEvent > Data Stores page in Manager. That particular enhancement might help clear up potential issues with whether or not an input connector will be able to retrieve data from a registered folder. Also, Greg points out that when registering the system folder, the user-interface's default is to place a value 'input' in the 'Input Directory' parameter. I always remove this value and leave the 'Input Directory' parameter unspecified. The default would only work if you wanted your 'Watch a Folder for New CSV Files' input to look for a sub-directory named input beneath a registered system path C:\GeoEvent\csv (for example). When working with file-based input and output (and enrichment) I usually register separate folders: GeoEvent_Input  --  C:\GeoEvent\input GeoEvent_Output  --  C:\GeoEvent\output GeoEvent_Enrich  -- C:\GeoEvent\enrichment I don't configure my 'Watch a Folder for New CSV Files' to expect an 'input'  sub-directory, and I don't allow it to 'Include Subfolders'. Finally, please keep in mind that the GeoEvent Extension generally supports RESTful data streams. The inputs capable of watching a system folder for files have some significant limitations and are generally intended to prove that real-time analytics you have designed in a GeoEvent Service behave as you intend. Moving toward production we expect real-time data feeds to arrive via HTTP/POST or as replies to queries you make on an external server's URL. Best Regards - RJ ... View more

Thanks Peter for the follow-up.  It looks like somehow there are two "RabbitMQ" messaging service / platform services configured, each with its own GUID / id. I don't know how this would happen; maybe there is an OS factor to consider in issue replication. I think I would expect some sort of failure if GeoEvent went searching for a platform service of type "MESSAGE_BUS" and found more than one candidate. I don't know that I would have expected the ACCESS_REFUSED log-in failure ... but in any case you're tracking to the root cause of the issue. Good luck - RJ ... View more

Hello Matthew - It might help if you think of a "connector" as pairing a specific "adapter" with a specific "transport". It is the transport's job it to deliver a payload (raw bytes) to an adapter. It's the adapter's job to extract data from the received stream and use a GeoEvent Definition to create a GeoEvent. The 'Receive Text from a TCP Socket' inbound connector you mention does not reach out to a IP address to interrogate a device. It opens a TCP socket (as a server application) so that some external client can send delimited text to that socket. The TCP socket in this scenario is the transport. The out-of-the-box text adapter is what is expecting simple delimited text that it can parse in order to retrieve data values and construct a GeoEvent. The GeoEvent Extension does not have an ODBC / JDBC connector you can use to retrieve data directly from an RDBMS table. Please take a look at the following thread:  Re: GeoEvent process  Oracle/SQL connector​ There are exercises in the product introduction tutorial, available here​, with the other product tutorials, which introduces you to the 'Receive Text from a TCP Socket' inbound connector and the GeoEvent Simulator you can use to send data to the input. If we shift our focus away from the TCP socket protocol for a moment, the 10.3.x release has out-of-the-box inbound connectors which allow a data provider to HTTP/POST data directly to REST endpoint hosted by the GeoEvent Extension. Could you configure your AVL to HTTP / POST position data as generic JSON or generic XML to a GeoEvent REST endpoint? Could you develop an external Python script to retrieve records from your database and then POST the data to GeoEvent? Using an HTTP / POST mechanism to send data to GeoEvent is not AVL specific -- and that's sort of my point. When considering GeoEvent you need to identify how the extension should receiving a feed from an external system, or how you can make a REST request to an external system that will return data in a well known format (such as delimited text, generic JSON, geoJSON, or generic XML). Once you've had a chance to go through the product introduction tutorial, maybe you could take a look at the Sierra Wireless RAP Connector tutorial. This would only be useful if you happen to be using Sierra Wireless gateway devices in your vehicles, but the exercises will underscore the concept of how GeoEvent receives data, this time in an AVL context. There are also a number of commercial AVL connectors we could look at: CompassLDE (CompassCom), Zonar, Networkfleet … Hope this information helps - RJ ... View more

Hello Jamal - The message your IE browser is displaying ("page can't be displayed") indicates that the CXF framework has not initialized correctly. This what handles all of the REST endpoints beneath https : //localhost:6143/geoevent/. Because the CXF framework is the very last component to initialize, we are going to have to look around a bit to try and identify what exactly might be failing. Are you working with the latest release (10.3.1) of GeoEvent? Do you see any errors in either the karaf.log or wrapper.log (found beneath C:\Program Files\ArcGIS\Server\GeoEvent\data\log) ... particularly with regard to "container" or "bundle" unresolved dependencies? Are you sure that you have the GeoEvent Extension licensed on this server? Messages logged in the karaf.log from LicenseChecker and LicenseEnforcer will  indicate an error if the product is not licensed. If you try to browse to https : //localhost:6143/geoevent/manager/ from another browser (Chrome or Firefox) ... do you see a message other than "page can't be displayed"? Are you running any anti-virus software?  McAfee On-Access Scanner has proven to cause problems with GeoEvent startup (refer to Esri Support KB Article #44817​). How many bundle files do you see in the C:\Program Files\ArcGIS\Server\GeoEvent\data\cache folder?  For the 10.3.1 release I'm expecting to see 370+ numbered bundle sub-folders (bundle0, bundle1, bundle2 ... bundle371). What files do you see in the product's configuration store?  C:\ProgramData\Esri\GeoEvent\: zookeeper (folder) zookeeper.properties (text file) zookeeper-dynamic.properties (text file) Does the zookeeper.properties identify a dataDir beneath C:\ProgramData\Esri\GeoEvent\zookeeper\default ? Is there a server identification string like server.1=MY_MACHINE:2182:2190:participant;0.0.0.0:2181 in the zookeeper-dynamic.properties? Can you identify the PID for the java.exe process running GeoEvent? In task manager, look for the processing running as the user used to run the GeoEvent Windows service whose Command Line contains the switch -Dkaraf.home, or look in the instance.properties file beneath C:\Program Files\ArcGIS\Server\GeoEvent\instances for the item.0.pid value. As a user with administrative privileges, from a Windows command line, run a netstat -a -o command to show you all of the active TCP connections and ports on which particular processes are listening. In my example below, the java.exe process running my GeoEvent is PID 11608 ... so I can search through the netstat results to find the ports GeoEvent is using: TCP 0.0.0.0:1100 HOSTNAME:0 LISTENING 11608 TCP 0.0.0.0:2181 HOSTNAME:0 LISTENING 11608 TCP 0.0.0.0:6143 HOSTNAME:0 LISTENING 11608 TCP 0.0.0.0:6180 HOSTNAME:0 LISTENING 11608 TCP 0.0.0.0:8101 HOSTNAME:0 LISTENING 11608 TCP 0.0.0.0:44445 HOSTNAME:0 LISTENING 11608 TCP 0.0.0.0:64009 HOSTNAME:0 LISTENING 11608 TCP 10.27.51.242:2182 HOSTNAME:0 LISTENING 11608 TCP 10.27.51.242:2190 HOSTNAME:0 LISTENING 11608 If all of the above check-out ... you might try an in-place removal of the product's configuration store and run-time generated files, then restart GeoEvent: Stop the GeoEvent Windows service Delete all files and folders beneath C:\Program Files\ArcGIS\Server\GeoEvent\data Delete all files and folders except certs beneath C:\ProgramData\Esri\GeoEvent Restart the GeoEvent Windows service Hope this information helps. You might need to contact Esri Technical Support and open an incident. Working with technical support will probably be better than exchanging messages on the forum for this issue. I'm happy to answer questions on any of the above, and please post if you're able to identify the issue. - RJ ... View more

I'm experiencing the same issue. Server is very unresponsive. Registered geodatabase data stores refuse to validate. Requests to stop and delete map / feature services spin for many minutes before returning control to the user -- sometimes returning with the "deleted" items still listed in Manager. A refresh of the ArcGIS REST Services Directory will eventually show the items I've selected to delete as having been deleted. My Server log is filled with Unable to process request. Error handling service request :Failed to register the activation group. Failed to construct instance of service 'SampleWorldCities.MapServer'. Failed to register the activation group. This after (eventually) being able to delete all other map / feature services. Any advice? - RJ ... View more

Thank you Peter. If you'll send me the technical support incident number I'll work to follow-up with my POC in Esri Support. Sorry, but I'm having trouble finding it in Esri Support's database... I've been told by the Server product team that the ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN error message is returned when: The guest account of Rabbit MQ is used to log-in. A RabbitMQ client connection to the RabbitMQ service is attempted with a different version number. The Server team has indicated that the first cannot be the case as we (Esri) delete the guest account and create a new account for log-in. The second option, then, could occur when a client (GeoEvent or an external / non-Esri app) whose RabbitMQ version is different than the one used by the ArcGIS Server platform service attempts a connection. For example, an ArcGIS Server 10.3.1 machine I accessed specifies that version 3.2.3 of RabbitMQ is being run (https://esri-server:6443/arcgis/admin/system/platformservices/?f=pjson). If a client leveraging a JAR file from RabbitMQ 3.2.0 attempted to connect, we'd receive the ACCESS_REFUSED error. I don't see how this can be the case, though, if GeoEvent is the only client attempting a connection. Let's plan on using the technical support incident to collaborate further. I can post a SUM back here if / when we figure out what's going on. Please feel free to post back to this thread if you discover a work around or solution yourself. Thanks - RJ ... View more