lost geoevent services on server reboot

[BradleyNeish1]

Whenever my server gets restarted, I lose my geoevent services(open up designer, there are no services). I have them saved, so it's pretty easy to get them back. I was just wondering if this is on purpose, or a known issue, or if there's something wrong with my setup?

[RJSunderman]

Is each collection item treated as an event?

Yes. When a JSON structure is received and the node being used as the root is a list, GeoEvent Processor parses the list and sends each item in the list to the adapter as a separate GeoEvent.

5 inputs, 5 Geoevent Services, and 5 outputs. Each "Update Feature" output goes to a separate AGS feature service.

As a service layout, this is reasonable. What we have to manage is the total number of events being received and processed every second so as to not overwhelm the server. Updating a feature service is currently a bottleneck in the event processing workflow. GeoEvent Processor can generally handle on the order of 800 events per second without any processing or filtering being performed. When updating a feature service, the event traffic needs to be throttled back to 200 - 300 events per second (total, across all running GeoEvent Services). We are developing high capacity stream services with high availability and cluster processing for our next major product release - but that is not going to be publicly available until sometime mid- to late-2014. I've attached two files to this thread which you might find helpful for capacity planning and general product performance.

The other 4 inputs all retrieve more than 100 items. Usually, not all of the records in the target feature class get updated after an input has run. In fact after about 12 hours, no further updates occur to the target feature classes with inputs items > 100.

I don't believe that what the feature service does with the event data it receives will affect resource consumption. That is, it is just as expensive to query a feature service and update 50 features as it is to query the feature service and update zero features. As long as we're not making PUT and GET calls too frequently, on too large a feature dataset, GeoEvent Processor should take the JSON in, convert it to a GeoEvent, query the feature service and post the event data. It shouldn't matter that after a half-day all necessary features have been processed and the dataset is considered up-to-date. How many total features are in that dataset and how many events are being received a second is what matters.

BTW - the 'java.exe' process has remained at about 600MB since I made the change to max heap size but now one of the ArcSOC.exe services is consuming 1.4GB memory.

The ArcSOC.exe processes are part of the ArcGIS for Server product. If indeed the issue is with feature service update, it might make sense that those processes are consuming the lion share of the server's memory. I might recommend scaling back your solution to run one of the five GeoEvent Services you have for a period of time to monitor the system's resources and then slowly scale up to either add a second service or stop a "smaller" service to begin running one that handles more data. Running all five services when 80% of those are each expecting a few hundred events at a shot might just be overwhelming your server.

- RJ

[RJSunderman]

Apologies - forgot to attach the promised performance planning and capacity documents.

[DennisGeasan]

As long as we're not making PUT and GET calls too frequently, on too large a feature dataset, GeoEvent Processor should take the JSON in, convert it to a GeoEvent, query the feature service and post the event data. It shouldn't matter that after a half-day all necessary features have been processed and the dataset is considered up-to-date. How many total features are in that dataset and how many events are being received a second is what matters.

Attached is a screen capture of the GEP monitor with the 5 inputs and outputs.  First page is the GE Services.  Second page is Inputs(top) and Outputs(bottom). They are all less than 3/s. Is this the measure you are looking for? If not how do I get the number of events per second?

Each feature class has a different number of records but they range from 34 to about 333 records.

DG

[AndreasEspersen]

HI RJ,

Last night and today i have also experienced losing my geoevent services on 10.2.2 after restarting the windows service for GEP and had to set it up all from scratch again.

Is there a way to export geoevent services like you could with the old tool Service Designer and import them again afterwards. That would also be beneficial moving from test to production.

We have also experienced some weird behavior with our outputs that the feature service we are outputting to suddenly have changed after restart of GEP Windows Service

Regards

Andy

[ThomasPaschke]

Hi Andreas,

You can save the configuration of your GEP (including the service) by exporting a configuration xml file. This will save your inputs, outputs, services, GEDs, Tags, GeoFences & Data Stores.

To do so go to Site -> Configuration Store -> Export Configuration. To restore or migrate a configuration use the Import Configuration functionality on the same site. If you only want to import/export selected features the xml file can be edited.

Hope that helps.

Best,

Thomas

[AndreasEspersen]

Hi Thomas,

Much Appreciated

Do you know why GEP drops the geoevent services?

Regards

Andy

[ThomasPaschke]

Although we made a good effort to fix all the use cases, in rare cases the validation of the service can fail which is why the service is not showing up in the user interface. It should not be deleted and still be on disk.

[BradleyHardesty]

I have also experienced this loss of GEP services and have to re-import my site configuration every time my server computer is rebooted or the GEP extension is restarted in windows services.  Looking for a resolution.

[RJSunderman]

The ArcGIS 10.3 GeoEvent Extension Patch 1 for Server, now available on the Esri Support site at the link below, includes a fix which I expect will address the issue you are describing.

http://support.esri.com/en/downloads/patches-servicepacks/view/productid/66/metaid/2181 

Specifically, we discovered that GeoEvent Services, published in the 10.3 release, which included filters with spatial conditions (INSIDE, WITHIN, CROSSES, etc.) caused the GeoEvent Services to disappear when either the GeoEvent product's service was restarted, or the GIS Server was rebooted. The upcoming 10.3.1 release will also address the issues addressed in Patch 1 above.

- RJ