POST
|
Hi Elizabeth, I'd like to add to Morakot's answer above. You can confirm what type of payload you are receiving by attaching a JSON Output (Write to JSON Output) to the NetworkFleet-Connector. This will output the payload into a structured JSON file that you can examine. The top field will contain the the schema structure of the data from Verizon, and will let you know exactly what types of payloads are being delivered. Also, with knowing the name of this field and the exact payload type, you could re-construct the filter operation in your GeoEvent Service to pass through a more accurate representation of the data you are receiving. - Chris
... View more
11-19-2016
06:16 AM
|
0
|
0
|
586
|
POST
|
Hi Elizabeth, The Network Fleet payload is sent with many different data types attached that correspond to different Field Definitions. Have you taken a look at the following question posed regarding a similar behavior? Verizon Network Fleet - GeoEvent Definition Manipulation If so I apologize for suggesting, but it references a way to ensure that the correct Field Definition is being honored. You can setup a filter to only pass through the correct definition (NetworkfleetVehicleReg in your case) and have it correctly mapped to another definition. If you have a configuration like this already and it is still failing, would you be able to provide some screenshots of the GeoEvent Service? -Chris
... View more
11-16-2016
01:25 PM
|
0
|
2
|
586
|
POST
|
Hi Pedro, How did you deploy the processor? Did you move the file into the /deploy folder within the GeoEvent System directories, or did you use GeoEvent Manager to add the file to the system (under Site-Components- Processors-Add a Local Processor)? Would you also be able to include the processor as an attachment for testing? -Chris
... View more
11-10-2016
09:21 AM
|
0
|
2
|
468
|
POST
|
Rui, The Reverse Geocoding Processor uses the Esri World Geocoding Service to calculate the nearest address when an incoming GeoEvent contains lat/long data. I'll answer your questions below in further detail: 1. Any GeoEvent Processor that allows for a new GeoEvent Definition to be created does so on the fly. That is, a definition will only be created when GeoEvents are passing through the processor. Simply adding the processor to the configuration does not include the definition you set. 2. When working with GeoEvent definitions, many times they contain too much information for the project that we apply it to. This question is more of a personal preference depending on what type of information is pertinent to you and the project. I personally like to send the newly created definition to a field mapper first before sending it to any output. This allows me to maintain control of the fields, field types and names. I will note that both scenarios will result in successful transmission of data and that I've seen both in production systems 3. While using the World Geocoding Service, performing a single geocode operation does not consume service credits. Since the geocoding process is only being called once per geoevent, the processor will not consume credits. Batch Geocoding however will consume credits (if conducted from a spreadsheet or CSV file). I would also mention that I'm not sure if there is a threshold set for the max number of free geocodes per minute you can perform. I've run this processor in front of an input connector that was set to poll every 60 seconds and was not charged any credits, however there is the potential to consume credits if making geocode operations quicker than once every 30 seconds (as I'm unaware of what the threshold could be). Thanks, Chris
... View more
11-10-2016
09:10 AM
|
1
|
1
|
744
|
POST
|
Tucker Longinotti I've started a new question on your behalf so we can get a fresh look at what is happening in your system. If need be, I can help you get a Support case created and work with you to diagnose any problems. For now, let's look at the behavior you are seeing. The original post can be found here: https://community.esri.com/thread/175125 You are running a 10.4.1 Full Stack (formally known as a Web-GIS) with GeoEvent. All of the output connectors display a 0 count for the received Geoevents, and have been for a long period of time. Normally this still indicates a RabbitMQ or larger issue with the transport system behind GeoEvent. What we can do is start at the basic level here and see if we can determine the cause. If you're able to, please attach a zipped copy of the GeoEvent log files. These can be found at C:\Program Files\ArcGIS\Server\GeoEvent\data\log. They are quite verbose, and I'm hoping they can show us what is occurring. Also, have you noticed this as a recent error, or has it been present since the initial installation? -Chris
... View more
11-03-2016
07:58 PM
|
1
|
3
|
2332
|
POST
|
Hi Tucker, I'll add to Jake's comment with a little more detail into what could be occurring with GeoEvent. The behavior you describe sounds like a error behavior in GeoEvent related to the RabbitMQ platform service. This service facilitates the movement of messages from the inputs, through the services and to the outputs within GeoEvent. It's normally the first thing that is assumed to be wrong when both the GeoEvent Services and Output connectors are not increasing in count. We discovered this error starting at the 10.3.x release, and it has been addressed in the 10.4.x release. The only exception is that we still observe it within some user environments or within AWS environments using the Esri AMI. The latter is due to a internal software conflict within AWS that is very hard to reproduce. Can you tell me what version of GeoEvent you are using, as well as if you see any large error messages in the GeoEvent logs? Normally the GeoEvent logs will contain references to the RabbitMQ service if this is truly the cause. -Chris
... View more
11-03-2016
11:12 AM
|
0
|
2
|
1034
|
POST
|
Hi Linda, Does the NM radius fence around the primary geofence change? For example, can we assume that this secondary radius is always 2NM wide? I've got a few more conceptual ideas that I'd like to try on my own, and for these tests I'll be assuming that each point can be between 0-2NM from the primary geofence. Currently all processors can be modified by administrators by leveraging the GeoEvent SDK that is packaged with every installation. The SDK contains a Developer Guide and SDK reference for anyone interested in modifying the components. This does however require a knowledge of the code needed to make the modifications, and currently there is no intuitive builder or interface to help with non-code modifications. - Chris
... View more
11-03-2016
10:34 AM
|
1
|
0
|
505
|
POST
|
Hi Linda, GeoEvent can alert you when two features intersect or are near each other using several methods including built-in filters and the Incident Detector Processor. However, running this analysis against time could be more difficult depending on the scope of your project. Can you explain more regarding the amount of time needed for each intersection as well as the specific application of this potential GeoEvent Service? The Track Gap Processor within GeoEvent has a built-in temporal element that tracks a time interval when events are not received (think a car sitting idle and not changing positions for a given amount of time). We could attempt to configure this in reverse to fit the scope of your project (to alert when GeoEvents are received for a given time interval). Combining this with a spatial filter will ensure that the points are near each other to trigger the processor to start tracking. -Chris
... View more
11-01-2016
05:46 AM
|
1
|
2
|
505
|
BLOG
|
When processing event records which include a large number of unique track identifiers you might notice that some spatial relationships evaluated by GeoEvent filters and processors do not behave as expected. We will focus on the “Enter Any” and “Exit Any” spatial operators as they apply to a GeoTagger Processor when more than 1000 unique TRACK_ID values are present. I will explain in detail the behavior reported to me by a few users and a potential product configuration you can make to better accommodate your data. Consider, for example, a GeoTagger Processor configured to enrich event records with the name of a geofence. The processor will evaluate a set of geofences and add a new field with the name of a geofence to an event record whenever the event’s geometry enters or exits an area. As you observe the output from a GeoEvent Service however, you notice that events are being dropped at random from the processor. Events that you observed several minutes ago are being removed or are severely delayed in displaying at all within the GeoEvent Service and are not included in the output. While there may be other reasons for your observations, we’ll assume that the GeoTagger Processor is the root cause. "Enter Any" and "Exit Any" Spatial Operators maintain state The majority of spatial operators do not require GeoEvent to maintain state information. The “Enter Any” and “Exit Any” operators are the exception, and require GeoEvent to track both geometry and track identifier as events are moved forward through the processor. Maintaining state requires a prior knowledge of each event’s location; each event is treated as a dependent of its previously observed location. Without maintaining state however, the previous positions of the events remain unknown to GeoEvent and all events are treated independently. This means that a GeoTagger Processor configured with the “Enter Any” or “Exit Any” operation must utilize a cache of unique track identifiers for every observed event. In short, this is what is known as a cache‑aware processor node. GeoEvent does have a maximum cache size enabled which is in place to maintain the state of all events as efficiently as possible; the default value for this property is 1000 events. When an event arrives at a cache-aware node and the event’s TRACK_ID is not contained in the cache, one of the previously observed TRACK_ID values must be discarded if the cache is currently full. This is done to make room for the newly observed event and respect the 1000 event limit. What does this mean for your data? Conceptually it means that the processor will forget that it has ever encountered an event with the discarded TRACK_ID. When observing the behavior in real-time, events will spontaneously be dropped from the processor and certain events that you observed several minutes ago may not be displaying at all in the output destination. Though the logic remains the same for both, the definitions used for “Enter Any” and “Exit Any” are in fact different. In order for GeoEvent to recognize that an event’s geometry has entered a geofence, its prior location must have been observed outside that geofence. Conversely, GeoEvent will not recognize that an event’s geometry has exited a geofence unless the geometry’s prior location was observed inside that geofence. These definitions are honored by default, but can produce different results if an additional property that GeoEvent uses to evaluate the spatial relationships is changed. "First GeoEvent triggers Enter" and "First GeoEvent triggers Exit" When a cache-aware node must make an “enter or exit” decision, it respects the settings of two properties within the GeoEvent Manager. These are the "First GeoEvent triggers Enter" and "First GeoEvent triggers Exit" properties, which determine the importance of the “enter” or “exit” operations. By default GeoEvent assumes that “entry” is more important than “exit” and that most event geometries are already outside of a geofence. The defaults for these properties are true and false respectively. You can change the values for each, but it is recommended that the change be made only with deliberate care. If not careful you could easily configure GeoEvent to start generating unwanted analysis and notifications, particularly after a restart of the server machine. For example, even if event geometries are expected to move around inside a geofence such as an administrative boundary, they will be located outside every other geofence that is registered with GeoEvent. If you change the "First GeoEvent triggers Exit" property from false to true you may unexpectedly get a significant number of “exit” evaluations every time the server machine is rebooted. We’ll look at an example of a default configuration with GeoEvent using “entry” as the main importance. Let’s assume that a GeoTagger is set to an “Enter Any” operation and the event TRACK_ID is not in the cache. If the event’s geometry lies inside of a geofence, then the GeoTagger will read the event as having entered. Alternately, if this same GeoTagger is set to “Exit Any” and the event’s geometry is already outside of a geofence, the processor will determine that the point did not exit. Sound tricky? Just remember that GeoEvent assumes that geofences are empty 99% of the time, and that points are expected to enter at some future point. That doesn’t mean that exits are ignored, but they are placed with less importance and require more observations to determine. Changing the maximum cache size for cache‑aware nodes The default setting for the maximum cache size is exposed through a specific product configuration file. It is important to note that this cache size is a maximum for each cache-aware node, and it is not a system-wide limit. You must have administrative privilege to edit this file. Changing the default value can result in your GeoEvent Server consuming significantly more RAM. The Java process in which the GeoEvent Server runs, by default, is limited to 4GB of system RAM. If every cache-aware node begins caching event data for significantly more than 1000 unique track identifiers, a larger portion of the 4GB will be consumed leaving less room for other more basic functions. To promote system stability it is recommended that you estimate of the total number of unique track identifiers expected from your event data and set the maximum cache size value slightly higher than the estimate (to allow for more features to be added over time). Keep the value as small as possible and do not specify an arbitrarily high maximum cache size. The cache value is contained within the com.esri.ges.manager.servicemanager.cfg file located in the following directory on a default system: “C:\Program Files\ArcGIS\Server\GeoEvent\etc” Keep in mind that this location may change if GeoEvent Server was installed to directories other than the product’s default system folder. Incident Detection and the 1000 event cache limit Within GeoEvent, you could also observe that open incidents are being dropped from the output destination in a similar manner to how the GeoTagger discards events. The Incident Detector Processor is another cache-aware node in GeoEvent, and utilizes an incident cache particularly when evaluating conditions for concurrently open incidents. Much like the GeoTagger, the Incident Detector will discard open/ongoing incidents if its incident cache is full and a new event triggers an incident to open. The opening/closing of incidents is managed by the Incident Manager program, which runs in the background to maintain state. The Incident Manager has a 1000 incident cache limit enabled by default, but it is exposed through the GeoEvent Manager rather than a configuration file. This default size can also be changed, but the same recommendations as the GeoTagger Processor apply. Obtain an estimate first of how many unique features GeoEvent will be processing, and then determine how many probable incidents could be open at one time. Set the number of Open and Closed incidents to slightly higher than the calculated maximum.
... View more
10-25-2016
08:46 PM
|
4
|
0
|
1778
|
POST
|
Ashley, If this is a blank GeoEvent installation, you could stop the GeoEvent Windows Service and clear out the contents of the C:\ProgramData\Esri\ directory. This contains the Zookeeper configuration files, and can have an effect on system performance at times.While the Windows Service is down, you could also clear out the contents of the C:\Program Files\ArcGIS\Server\GeoEvent\data directory. This contains the unpacked components that are responsible for running GeoEvent. It's possible that one of them did not get unpacked correctly. Keep in mind that deleting the Zookeeper directory will delete any services you've created inside of GeoEvent Manager. After you've done this (if you can), start the GeoEvent Windows Service again and pay attention to the ArcGISGeoEvent.exe and java.exe processes. If GeoEvent is installed correctly, they should consume about 1MB and 500MB of system RAM respectively. -Chris
... View more
10-25-2016
08:26 PM
|
1
|
6
|
1836
|
POST
|
Hi Ashley, I just encountered this behavior myself on a testing environment I was configuring. What fixed it for me was to stop and start the GeoEvent Windows Service, and clear the browser cache. After that I opened a new private browser just to rule out a cache and I was able to login with the Primary Site Administrator. From what I've noticed so far this behavior is erratic at best, but there are not corruptions as far as I can tell. The browser is simply failing to pass the token to the ArcGIS Server for verification, and it's happened so infrequently that I have not had a chance to examine it with Developer Tools. Let me know if you've done this already or what the outcome is if you try. -Chris
... View more
10-24-2016
07:51 PM
|
1
|
8
|
1836
|
POST
|
Hi Allen, Without visualizing your environment first hand, to me this behavior could be with the data store connections themselves, or another program on the machine that is blocking the GeoEvent.exe and java.exe process from starting correctly. I'll pose a few more questions to see if we can get it: 1. Within the data store connections in GeoEvent Manager, are you using any authentication such as "Use Token" or "Web Tier Authentication"? 2. When the ArcGIS Server and GeoEvent Windows Services are started again, how much RAM do the "ArcGISGeoEvent.exe" and "java.exe" processes consume? 3. Do you have any antivirus on-access scanning or other 3rd party applications installed that could be blocking the processes? 4. At this point, you can start leveraging the GeoEvent logs to further determine the cause of the behavior. a. Start with the ROOT logger set at "INFO" b. Look for any Token or data store failures c. The physical files are located in the C:\Program Files\ArcGIS\Server\GeoEvent\data\logs folder, and you can zip that folder and include it in this thread Thanks - Chris
... View more
10-19-2016
06:35 AM
|
0
|
0
|
733
|
POST
|
Hi Allen, Sorry to hear about the GeoEvent Extension within your environment, and this is certainly not expected behavior. We can try to narrow down the potential cause, as from what you described could be the ArcGIS Server or GeoEvent Extension. It will be easier to pose a few questions regarding your environment first. 1. When the machine becomes responsive again after the shutdown occurs, does GeoEvent fail to read the ArcGIS Server map/feature services? Another way of looking at this is when you open the GeoEvent Manager, are the input/outputs showing as green (valid) or red (invalid)? 2. Do the map/feature services themselves become unresponsive at the REST endpoint? 3. Can you replicate the behavior without a complete system shutdown? Meaning, are your "graceful" shutdowns occurring without shutting down the machine? 4. Finally, what version of ArcGIS Server/GeoEvent do you have within this system? Thanks -Chris
... View more
10-18-2016
05:27 AM
|
0
|
3
|
733
|
POST
|
Hi Chris, I've used the JSON sample you've provided to test within GeoEvent 10.4. Do you have an accessible source for this JSON feed? With the current JSON formatting, GeoEvent will create a definition with the numbers (1,2,3) as individual fields. This seems more specific to the syntax of the JSON rather than a setting in GeoEvent. Below is the auto-generated definition when I polled the JSON you've provided Out of curiosity I'll be testing the JSON syntax to see if it's actually possible, but I would say that it's not possible to generate the Definition without using dedicated field names in the JSON. -Chris
... View more
10-17-2016
07:51 AM
|
2
|
0
|
433
|
POST
|
Wellington, Apologies for the late response, but this sounds like a great application for the Motion Calculator Processor which can be installed as an add-in processor for the GeoEvent Extension. The Motion Calculator Processor will calculate the distance, speed, heading and many other variables for a given incoming GeoEvent. Consider the following example for utilizing the Motion Calculator Processor within your workflow: The vehicle's position would be evaluated against a filter, determining if it has exited a specific geofence. The filter in the example above would only allow vehicles through if they have in fact exited. The Motion Calculator Processor would then start tracking the position of the vehicle and calculate various statistics of that vehicle. Heading, speed, min/max values and other variables will be appended to the data and pushed into a new GeoEvent Definition. You can then use this new definition to populate a database feature class, send an email notification or write to a specific file depending on the scope of your project. More information regarding the Motion Calculator Processor can be found at the first link below. You can also find additional processors and GeoEvent-related content within the GeoEvent Group on ArcGIS Online in the second link. https://www.arcgis.com/home/item.html?id=03c440742bbd4686b16420164d30e448 https://www.arcgis.com/home/search.html?q=owner:GeoEventTeam -Chris
... View more
10-15-2016
06:37 AM
|
0
|
0
|
659
|
Title | Kudos | Posted |
---|---|---|
1 | 05-30-2017 01:44 PM | |
1 | 10-09-2016 09:57 AM | |
1 | 05-22-2017 08:07 AM | |
1 | 10-25-2016 08:26 PM | |
1 | 10-05-2017 01:33 PM |
Online Status |
Offline
|
Date Last Visited |
11-11-2020
02:25 AM
|