Allow python scripting on GeoEvent Processor

3349
10
06-21-2017 11:42 PM
Status: Open
PanagiotisPapadopoulos
Esri Regular Contributor

I would like to see the ability to add a python script as a processor on GeoEvent Processor. 

10 Comments
AnninaHirschi_Wyss1

+ 1 from me

RJSunderman

I don't know that we have a work item in the GeoEvent Server's backlog to create a processor which will allow you to enter Python Script to execute against an event record received for processing by a GeoEvent Service. However, there is work being done on the realtime module within the ArcGIS API for Python‌ which might provide functionality you are looking for.

To folks voting-up this idea ... would you please comment with some additional detail on how you would like to use Python with GeoEvent Server?  Are you looking to write python script to calculate values like the ArcGIS Desktop / ArcMap "Field Calculator" (vs. the GeoEvent Server "Field Calculator" whose expression interpreter does not support script/code)?

Or are you more interested in scripting system administration tasks, like those you perform from within GeoEvent Server's Manager web application? Are you perhaps looking for ArcGIS API for Python to expose endpoints from the GeoEvent Server REST API?

- RJ

Shannon Kalisky

AnninaHirschi_Wyss1

Hi RJ Sunderman,

Here's my use case:

We have a payload (comes via rest-json-in) containing encrypted hex attributes. To decode, we have a python script. Ideally we would like to decode that attribute before writing to an output (e.g. fs, bds) and that's why we thought it would be great to be able to run that script within a processor.

We are newbies dealing with GeoEvent and still exploring the best way to achieve that. Any thoughts from your side?

Thanks,

Annina

RJSunderman

Hello Annina

For the current 10.6 and previous 10.5.x releases I would recommend the following videos, available from the Esri YouTube Channel:

Most likely you are going to need to develop a custom processor for GeoEvent Server, using the Java SDK, to handle decoding of the encrypted hex attribute values.

The GeoEvent Server's Field Calculator is not at all the same as the Desktop / ArcMap Field Calculator. GeoEvent's processor supports some limited string manipulation using methods from the Java String class. You can read more about these in the on-line documentation (updated to include examples for the 10.6 release):  Processors - Field Calculator

Unless / Until we develop a Field Calculator that accepts Python Script - you're limited to the above.

-- RJ

AnthonyRyanEQL

+1 from me. One of the core languages for the Esri platform is python. It's used ArcMap, ArcGIS Pro, GP tools, Python API, etc. Makes sense to have it as part of geoevent server processor as to have a common language and not have to learn Java, etc.

ModyBuchbinder

+1 from me too. Just like ArcGIS let you create an ArcMap button or raster function in Python just to make it easier it will be nice to have a framework to write a processor, instead of calling Java program, to do something with the event (filter, create alert, etc) it can be easier to do it in Python (Java is less common for ArcGIS programmers) 

EricAdams

I need this processor for the following reason: my GeoEvent service runs, and then my python script is run, but I run the python using task scheduler. The script has something that checks if the GeoEvent service is still running.

It would be nice to get python integrated into the GeoEvent processor family to get away from the task scheduler.

BrianLomas

Similar to Eric's idea, I would be interested in a geoevent processor that could trigger a python script. So, if a certain condition is met a python script would be called. This would provide a lot of flexibility for those who are not familiar with java it also opens up a new ability to extend the GE functionality as a whole i.e. using python geoprocessing tools, reporting, access to custom python geoprocessing tasks etc.

BeckyC-O

@RJSundermanDid this ever get implemented?  I'm hoping to launch a python script when Geoevent detects a feature class has received a new record via Survey123.

GregoryChristakos

The development of a python processor to perform custom real-time geoprocessing or to trigger a python job elsewhere, is not on the GeoEvent Server road map at this time.

Our team had initially done some exploration into this idea around the 10.8.1/10.9.0 timeframe but found that there were significant limitations with real-time data processing and workflows. As such, there are no plans to revisit this sort of enhancement as an out-of-the-box capability.

Depending on the type of feature service you are working with, it could be worth exploring web hooks as an alternative mechanism for acting on adds, updates, or deletes to a feature layer being made by GeoEvent Server. In theory some feature services can be configured with a web hook to send a web request to an endpoint of your choosing whenever an add, update, or delete, is made to its feature layer. The endpoint can be what then executes a python script.

In lieu of web hooks, it may also be possible to have GeoEvent Server construct a POST request to a similar sort of web endpoint that then triggers a python script asynchronously.

Beyond that, we still recommend what @RJSunderman had mentioned before.