|
POST
|
MBramer-esristaff had to tackle this issue recently. I think what you need to do is use a Field Calculator to cast the Date to a Long integer value. Use an expression like myDateAttributeField + 0 and write the value into a new field whose type is Long (not Date). Then you can use a second Field Calculator to compute a Long integer representing epoch milliseconds five minutes ago: currentTime() - (5 * 60 * 1000). Write this value into another new field whose type is Long. Then use a filter to compare the two integer values. - RJ
... View more
12-01-2016
11:45 AM
|
2
|
3
|
7345
|
|
BLOG
|
A couple of times a year a script developer will ask me about using the GeoEvent Admin API to automate some administrative task - such as stopping and restarting a GeoEvent input. Any user action taken through the GeoEvent Manager web application makes a request against a URL in our GeoEvent Admin API. So, in theory, once you authenticate with the GeoEvent Admin API, you should be able to script some fairly simple tasks, like stopping a running input, modifying one of the input's parameters, saving the input's new configuration and restarting the input to begin receiving event data. I'd like to share a blog post by Andy Ommen, a solution engineer working with Esri Database Services out of our Boston regional office. Take a look and let him know if you find his information useful. I really appreciate him sharing this out through his blog. Here's the link: https://community.esri.com/people/aommen-esristaff/blog/2016/10/19/scripting-tasks-using-the-geoevent-admin-api?sr=search&searchId=cfa9354e-f275-46d3-9993-373294a8fc57&searchIndex=0 Update March 2019 – Eric Ironside, a product engineer on the Real-Time team, has created a second blog illustrating how to update the properties of a GeoEvent Input. Much appreciated Eric! https://community.esri.com/people/eironside-esristaff/blog/2019/03/21/scripting-tasks-using-the-geoevent-admin-api-update-inputs?sr=search&searchId=f382e18c-085e-48e6-b73a-d8917b48fd27&searchIndex=3 Update November 2019 – Jake Skinner, a platform configuration engineer in Esri's Philadelphia region, has another blog illustrating how to authenticate with the GeoEvent Server administrative API in order to script administrative actions like updating the configurable properties of inputs and outputs. Thanks Jake! Using the GeoEvent Admin API with Python It might also be helpful to know that the ArcGIS Server Administrative REST API is documented using Swagger. You can review available operations exposed by the Admin API: Browse to https://my-machine.domain:6143/geoevent/admin Acquire a token from your ArcGIS Server (or Portal for ArcGIS if federated) and log-in In the top-left corner, click the API link to take you to the Swagger Doc for the GeoEvent Server Admin API Note the advice at the top of the page on how to authenticate your admin script's requests with the API Hope this information is helpful – RJ
... View more
12-01-2016
11:21 AM
|
3
|
0
|
2286
|
|
POST
|
John - I wanted to update you to let you know that enhancements are being rolled into the 10.5 release for the Intersector processor. At 10.4.x the assumption was that the output geometry would be the same dimension as the event's geometry. That's why the illustration in the product introduction tutorial worked and showed the intersection of an event's polyline with a polygon GeoFence output as a polyline. Beginning with the 10.5 release, you will be able to configure an Intersector processor to output a specified 'Output Geometry Type'. So, in your case, if you are processing an event with an associated polyline geometry and you want to obtain the point at which your event's polyline intersects a polyline GeoFence - you will be able to do that. Illustrated below are a couple of the use cases I've used to verify the enhancements to the Intersector processor at 10.5 You'll notice that you can still obtain a multipart polyline when an event's polyline intersects a polygon GeoFence at two separate line segments. You can also get the point intersection between two polylines. There are several other test cases which I will try to lay out in a blog, which will serve as the documentation for the enhancements to the Intersector processor. Hope you find the 10.5 enhancements useful - RJ
... View more
07-29-2016
05:33 PM
|
0
|
0
|
1038
|
|
BLOG
|
When conducting our benchmarks for the upcoming 10.5 release we encountered an unexpected drop in the raw number of events per second we can ingest in a TCP/Text input and send out a TCP/Text output (without conducting any processing or filtering on the event data). The issue turned out to be that we had accidentally staged the software on a different type of Amazon EC2 than what we had used in previous benchmark tests. It turns out that the c4.2xlarge class supports Intel® AVX2 whereas the c3.2xlarge class does not. Support for AVX2 can double the number of floating-point operations per second an application can perform. For our benchmarks that meant that event throughput for simple events which included coordinates for a point geometry fell from over 4,000 events per second to well below 3,000 events per second. Instance Type vCPU Memory (GiB) Storage (GB) Networking Performance Physical Processor Clock Speed (GHz) Intel AVX† Intel AVX2† Intel Turbo EBS OPT Enhanced Networking† c3.2xlarge 8 15 2 x 80 SSD High Intel Xeon E5-2680 v2 2.8 Yes - Yes Yes Yes c4.2xlarge 8 15 EBS Only High Intel Xeon E5-2666 v3 2.9 Yes Yes Yes Yes Yes I mention this because, looking at the two class specifications in the table above, it would be very easy to overlook something "minor" like support for Intel AVX2 technology. (Scroll the table display to the right to see that the "C3" instance does not support AVX2.) As with our discovery that different types of RAM (e.g. DDR3 vs. DDR4) are very significant - capabilities supported by your chosen server's processor can be very significant. Cross Reference: - https://aws.amazon.com/ec2/instance-types - http://www.intel.com/content/www/us/en/benchmarks/performance-xeon-e5-v3-advanced-vector-extensions-paper.html
... View more
07-19-2016
11:23 AM
|
7
|
0
|
1894
|
|
POST
|
Christine Wiltawsky - Yes, I just verified using a system configured with 10.4 Patch1. Note that I've chosen to blur irrelevant property values since, in my example, I'm writing to an existing field of type Date.
... View more
07-06-2016
02:33 PM
|
2
|
0
|
6079
|
|
POST
|
See Also: Announcing the Next Generation of ArcGIS Ideas in the Esri Community on GeoNet! In conjunction with the launch of the new Esri Support website, the old "ArcGIS Ideas" site http://ideas.arcgis.com has been moved into GeoNet (https://community.esri.com/community/arcgis-ideas) The GeoEvent product team is currently making a push to review both bugs and enhancements from the product's backlog. Some of these work items are a year or more old and may be partially addressed with changes made for the 10.4.1 release -- which should be publicly available the week of June 1st 2016. Some changes, particularly those dealing with Field Calculator, Field Mapper, and the handling of date/time values continue to be deferred. The next generation Real-Time event processing, being developed now and scheduled for release December 2016 as an Esri Managed Cloud Services offering, may incorporate some of these long awaited changes with plans to eventually bring them into the on-premises product. If you have changes you have been looking for, please take a look at the new ArcGIS Ideas space in GeoNet and vote up ideas you want to see implemented. Make sure, if you are submitting a new idea to tag it geoevent to help get it routed to the GeoEvent product team. Best Regards – RJ
... View more
05-27-2016
02:09 PM
|
4
|
0
|
2225
|
|
POST
|
Robert - I cannot comment on why publishing your survey to Portal vs. ArcGIS Online would produce the differences in schema you are observing. If you want to try and dig into why GeoEvent is receiving event records from Portal which have not been recently updated (when polling an ArcGIS Online feature service appears to work), you will need to configure GeoEvent to include DEBUG messages in its log. You might try setting DEBUG logging on just the following loggers within GeoEvent Manager. If you see errors or warnings associated with these, please let us know. com.esri.ges.datastore.agsconnection.DefaultAGOLConnection com.esri.ges.datastore.agsconnection.DefaultArcGISServerConnection com.esri.ges.datastore.agsconnection.DefaultPortalConnection Unfortunately the messages you need which include the WHERE clause GeoEvent uses to identify event records which have been recently added or changed is provided by HTTP client log messages. I say this is unfortunate because that particular component is very, very chatty. You can only imagine as GeoEvent is communicating via HTTP / REST with all sorts of stuff ... If you set DEBUG logging on the GeoEvent <root> component -- which you can do for a short period of time, just long enough to catch your input polling and receiving newly updated records -- you can catch the HTTP client log messages and review exactly what GeoEvent is requesting from the feature service it is polling. You will want open the karaf.log in a text editor and search for a logged message with the following substrings: Executing following request: GET http://services.arcgis.com arcgis/rest/services/ your_feature_service_name /FeatureServer/ layer_index /query where=1%3D1+and+LastUpdated+%3E%3D+%272016-05-10+21%3A34%3A07 (decoded: where=1=1 and LastUpdated >= '2016-05-10 21:34:07) That last message is URL encoded and can be difficult to read, but that's how it's going to appear in the log file. You can copy/paste the URL encoded string into an online decoder (I like using the URL Decoder/Encoder) to get the decoded string I included above. Notice that the decoded string contains a date/time allowing GeoEvent to specify that any feature returned should have a date/time greater than that value. Crawling through the karaf.log is going to be a chore, but since you know the name of the feature service being polled and the name of the attribute field suppling the date/time value you should be able to zero in on messages of interest fairly quickly (click the illustrations below to open the thumbnails): If you filter your karaf.log down to just messages with "Executing following request" and "Got response from HTTP request" you should start to see a pattern, at least in the ArcGIS Online example. This pair of messages is requesting information on the feature service: This pair is telling GeoEvent that there are no features which satisfy the query: Eventually, when there *are* features which satisfy the query, you should see messages like the following: This third example is interesting because GeoEvent is following its query for features WHERE LastUpdated+%3E%3D+%271970-01-01+00%3A00%3A00.000%27 (e.g. LastUpdated >= '1970-01-01 00:00:00.000') with a query requesting only the OBJECTIDs be returned (e.g. returnIdsOnly=true). GeoEvent has to do this so that it knows whether it got all of the features or whether a feature service is limiting the number of features returned in response to a client's query. You might be asking "Why the date/time back in 1970??" When an input which polls for incremental updates is first created a false date/time is incorporated in the query’s WHERE clause. This is because the new input has not had a chance to cache a date/time value and needs to make one up which will likely get everything the feature service has. Thereafter GeoEvent will substitute the greatest date/time value from the set of features returned in response to its query ... and the next query's WHERE clause will ask for only the features whose date/time is strictly greater-than the previous observed date/time. Hopefully all that technical background and log file illustrations were helpful. You, and possibly whomever you work with in Esri Tech Support, will need that background on what GeoEvent is doing in order to diagnose why the query to the Portal item is not returning just the recently updated event records. - RJ
... View more
05-10-2016
05:38 PM
|
0
|
0
|
1564
|
|
POST
|
Hello Rao – Yes, the XML sample you provided above can easily be adapted by the product’s XML adapter and processed by a GeoEvent Service. The first remark I have is that XML is translated to JSON by the product’s adapter. So you will want to avoid using XML attributes which can contend with sub-elements of a given node. Your data sample is pretty clean, but to illustrate, this is what it looks like as JSON: Your challenge, using GeoEvent, will be that you want to access items in a list which is part of a larger structure. If you were to stipulate that the list <Assetstatus> would always have exactly 5 items, then we could assume that you could pull specific items out by index. For example: ez_data.fz.Statuslist.Assetstatus[3].Status This isn’t normally something that you can stipulate and we cannot therefore assume there will be a fixed number of items in the list. In that case you have one opportunity – as part of the GeoEvent input’s configuration – to tell the adapter that it should use something other than <ez_data> as the root node. You specify this using the XML Object Name parameter: This way, the five items in your sample XML’s <Assetstatus> list are each brought in as separate events. The upside is that you don’t need to worry about how many items will be in the list. The downside is you do not have the title, id, timestamp, or coordinate information that is above the <Assetstatus> node in the data structure. There is work you can do to pull information from a XML feed’s header and incorporate it into individual event records. Please refer to the thread XML can I use this source if you are interested in taking that approach. Hope this information is helpful - RJ
... View more
04-21-2016
05:00 PM
|
0
|
0
|
1112
|
|
POST
|
Jon – adding to Sagar's advice above, I noticed that you said that your "SSL Cert is set up on the machine, registered with arcgis server and binding configured with IIS". Did you import the SSL certificate both into Server (using the Server Admin Directory) and also into Portal (using the Portal Admin Directory)? I think your IIS certificate needs to be imported into both products. – RJ
... View more
04-21-2016
02:32 PM
|
1
|
1
|
3421
|
|
POST
|
Sardar - Please take a look at the following threads: Geoevent Processor Geofence Question Overlapping geofences with different attributes What you are asking for GeoEvent will not do. The only way to load a single GeoFence representing an aggregate of several Geometries into GeoEvent would be to create a convex hull (or envelope) around the points and import that polygon Geometry as a GeoFence. You cannot maintain the identities of several different Geometries as part of a single GeoFence. Every GeoFence must have a single Geometry which can be assigned to a category. Within a category every GeoFence must have a unique identifier / name. I cannot think of a way, using GeoEvent, to determine the “closest” GeoFence to a Geometry associated with a received event. The event’s Geometry either does or does not have a spatial relationship with a GeoFence (e.g. intersects the GeoFence’s Geomtery). - RJ
... View more
04-18-2016
07:29 PM
|
0
|
0
|
699
|
|
POST
|
Hello Sardar – The Poll an External Website for JSON input connector does indeed have a property ‘Receive New Data Only’ … but the property does not provide the behavior I think many folks are looking for. When you make a request on a site’s endpoint, unless the site is configured to accept a parameter as part of your request specifying the data you want, the only way GeoEvent has to determine if the site has “new” data is to look for a specific header value in the headers returned in the response from the external server. This “Last-Modified” header (described in Section 14 of RFC 2616 “Header Field Definitions” of the HTTP/1.1 protocol) tells GeoEvent to go ahead and parse the response from the server as there is new information in the response. From the spec: 14.29 Last-Modified The Last-Modified entity-header field indicates the date and time at which the origin server believes the variant was last modified. Last-Modified = "Last-Modified" ":" HTTP-date An example of its use is Last-Modified: Tue, 15 Nov 1994 12:45:26 GMT The exact meaning of this header field depends on the implementation of the origin server and the nature of the original resource. For files, it may be just the file system last-modified time. For entities with dynamically included parts, it may be the most recent of the set of last-modify times for its component parts. For database gateways, it may be the last-update time stamp of the record. For virtual objects, it may be the last time the internal state changed. What "Last-Modified" does not do is key in on specific records which have changed. For example, a site serving information on a dozen different active wildfires might update a few of its wildfire event records and then indicate that update(s) have been made by setting its “Last-Modified” to date/time the updates were committed. When GeoEvent polls it will see that the "Last-Modified" date/time is later than the value it has cached and will ingest the event data returned from the site (all of the wildfire records in this case). The Poll an ArcGIS Server for Features input, on the other hand, is a little more flexible. That input also has a property for polling only for feature records recently added or updated – but in this case GeoEvent is able to work with the Esri Feature Service to incorporate a WHERE clause in its request in order to receive only the specific records which satisfy the WHERE clause. This is not functionality provided by the input you’re using which is making a more general request on a less well-known external server. It is difficult in the 10.3.x / 10.4.x releases to create a filter expression to compare an event’s date/time against another date/time value. You might be able to implement a work around by handling date/time value received from the external server as Long Integer values (in epoch milliseconds) rather than as Date values. I’ve included a couple of references below to discussion threads which mention this. Generally, however, using a GeoEvent Service to detect and discard “old” event data is not recommended. GeoEvent is assuming that all of the data either sent to an input via HTTP/POST or polled from an external server is “live” and should be processed as current real-time values. See Also: How to Create Temporal Filter in GeoEvent Filter out Time Anomalies
... View more
04-18-2016
06:39 PM
|
0
|
0
|
1046
|
|
POST
|
Hello Stephan – Sorry your question has gone so long without a response. Is the issue you describe still a problem for you? You might look at the thread Re: Spatiotemporal big data store isn't showing up in Geoevent Manager which Martin and I contributed to March 5th (a few weeks after you posted this question - sorry). It sounds to me like maybe you don’t have a connection to Portal registered as a GeoEvent Data Store? The GeoEvent Data Store connection registered as ‘Default’ is not sufficient to begin using the new spatiotemporal big data store – you have to register a “Portal” type connection as a GeoEvent Data Store and provide username and password credentials enabling GeoEvent to authenticate with your Portal. Once you do this, and then navigate to Site > GeoEvent > Spatiotemporal Big Data Stores in GeoEvent Manager, you should see an entry for an enabled BDS in the Spatiotemporal Big Data Store pulldown toward the top of the Manage Spatiotemporal Big Data Stores page. - RJ
... View more
04-18-2016
05:55 PM
|
2
|
0
|
1141
|
|
BLOG
|
Sharon - Sorry your questions went for so long without a reply. Are the above still issues for you? Did you perhaps get some help from Esri Tech Support? You shouldn’t have any problems deleting a GeoEvent Definition created by a processor you’ve incorporated into a GeoEvent Service. Every time I’ve done this the processor has recreated whatever event definition it thinks it needs. Generally I advise folks to copy, rather than edit, a GeoEvent Definition they didn’t create. In this case, because an event definition “upstream” from the processor changed, you could signal the processor to clean-up its managed event definitions by making a simple change to the GeoEvent Service – like removing a connecting line from between two nodes – then publishing the changes to the GeoEvent Service. You can then restore the connecting line you deleted and publish the GeoEvent service again … processors like Field Calculators, GeoTaggers, and Field Enrichers which created event definitions should delete them as part of the first publication and create them once events are received (after the second publication). If the input’s event count is increasing then I’d assume that event data is being received and successfully adapted to create a GeoEvent. If the GeoEvent Service’s ‘In’ event count is increasing, but not the ‘Out’, I’d assume that events are either being filtered or are dying when a processor node attempts some action and an exception is thrown. On the other hand if neither the ‘In’ event count nor the ‘Out’ event count are increasing I would assume that events are being placed on a queue by the running inbound connector (input) but never being taken off because the RabbitMQ message broker is not running. - RJ
... View more
04-18-2016
05:31 PM
|
0
|
0
|
1769
|
|
BLOG
|
Simon - I'm going to use the 10.3.x-r3 release of the introduction tutorial (the version marked July 2015 and Rev B) with our current 10.4.1 release candidate to try and answer your question about what needs to change in the module 2 narrative and exerices to bring them into line with 10.4.x ... On page 28, step 8: Click to edit the field tagged GEOMETRY and rename the field Location. You should leave the field named geometry as that is what the 10.4.x release is going to want to see when using the Latitude and Longitude fields to build a point Geomery for you from the Double values received by the TCP/Text input. The field named geometry must still be moved to make it the last field in the GeoEvent Definition. On page 35, step 3: Use the illustration below to configure the properties of the new processor. You should configure your Field Mapper processor to map the field name you retained (geometry from Flights-TcpTextIn) to the field named geometry (from the Flights event defintion imported from your published feature service's layer). Your output from the Flights GeoEvent Service should mirror what is illustrated on page 36: Flights-TcpTextIn,SWA2706,18-Apr-2016 15:21:59,IAD,TPA,B733,37000,-79.585739,34.265521,"-79.585739,34.265521"
Flights,,SWA724,18-Apr-2016 15:29:20,IAD,ABE,SF34,10000,"-76.405289,39.573271"
Flights,,SWA992,18-Apr-2016 15:29:34,IAD,MDW,B737,36400,"-82.067414,39.630957" The two most recently simulated events are no longer associated with the Flights-TcpTextIn event defintion, they have been mapped to the Flights event definition you imported from your published feature service's layer. The double set of commas correspond to the OBJECTID field left unmapped in your Field Mapper Processor. The Geometry (the last field in the Flights event defintion) holds the point generated for you by the TCP/Text input; its value is represented as a quoted pair of Double values. Those are the only changes I need to make to the July 2015 and Rev B version of the introduction tutorial to accomodate the 10.4.x behavior I was blogging above. I have a goal of walking through the entire product introduction tutorial before the UC this summer and updating its narrative and illustrations to reflect the upcoming 10.4.1 product release. - RJ
... View more
04-18-2016
03:55 PM
|
0
|
0
|
1769
|
|
POST
|
Daniel - For the 10.3.1 release we tried incorporating a library to improve the user's sign-in experience but discovered a rather severe security vulnerability and had to revert (remove) the code changes. We improved single-sign-on experience for the 10.4 product release. To provide you a little bit of background, GeoEvent requires anonymous access to two different REST endpoints in order to coordinate authentication with Portal when Portal has been federated with ArcGIS Server and authentication is passing through the web-tier (e.g. Basic Authentication or IWA). https://localhost:6443/arcgis/rest/info (alternatively http://localhost:6080/arcgis/rest/info) Provides ArcGIS Server Authentication Information https://portal-server-hostname/arcgis/sharing/generateToken Portal’s token generation page At 10.3 / 10.3.1 ... if GeoEvent cannot access these endpoints its failover behavior is to prompt the user to acquire and provide a token in order to log-in to the GeoEvent Manager. I think the 10.4 / 10.4.1 login experience, when federated with Portal, will be more in-line with what you expect. There is one known issue that is currently deferred to 10.5 -- after federating with Portal and completing the configuration of your SSL certificates you often have to restart the GeoEvent Windows service twice (2x) in order to get it to recognize the new security topology. Hope this information is helpful - RJ
... View more
04-05-2016
03:09 PM
|
2
|
1
|
1956
|
| Title | Kudos | Posted |
|---|---|---|
| 1 | 02-11-2026 11:38 AM | |
| 1 | 02-11-2026 10:38 AM | |
| 1 | 01-05-2023 11:37 AM | |
| 1 | 02-20-2025 03:50 PM | |
| 1 | 08-31-2015 07:23 PM |
| Online Status |
Offline
|
| Date Last Visited |
05-21-2026
12:39 PM
|