JSON input from Geoprocessing Service

[JamesCrandall]

We have existing or in-development Geoprocessing Services that essentially output string JSON that we're simply enriching with location info (x/y or ring values).  Since we're building these with Python I think we can format the output just about any way we want, but is there connectors in GeoEvent that can be configured to just specify the xy attributes of the json we want to use?

In the sample output below, all of the attributes in the "result" array would be what we'd like to push to GeoEvent and output as a feature service.

{
 "results": [
  {
   "paramName": "resultJSON",
   "dataType": "GPString",
   "value": {
    "result":  [
     {
      "phosphorusUnit":  "n/a",
       "samples":  [
      
      ],
       "phosphorus":  "n/a",
       "pointX":  -8938939.6672,
       "pointY":  3079872.605700001,
       "wmFeature":  "STA-1E",
       "phosphorusTimestamp":  "n/a",
       "_7dayFlowVolume":  0,
       "structureName":  "S377"
     },
      {
      "phosphorusUnit":  "mg/L",
       "samples":  [
       {
        "qualityCode":  "P",
         "collectMethod":  "G",
         "collectDate":  1583936160000,
         "sampleId":  "P114457-10",
         "value":  0.086,
         "testName":  "PHOSPHATE, TOTAL AS P",
         "flag":  null,
         "station":  "G311",
         "testNumber":  25,
         "units":  "mg/L"
       },
        {
        "qualityCode":  "P",
         "collectMethod":  "G",
         "collectDate":  1584535260000,
         "sampleId":  "P114450-12",
         "value":  0.065,
         "testName":  "PHOSPHATE, TOTAL AS P",
         "flag":  null,
         "station":  "G311",
         "testNumber":  25,
         "units":  "mg/L"
       },
      {
      "phosphorusUnit":  "n/a",
       "samples":  [
      
      ],
       "phosphorus":  "n/a",
       "pointX":  -8942434.6842,
       "pointY":  3083166.2346,
       "wmFeature":  "STA-1E",
       "phosphorusTimestamp":  "n/a",
       "_7dayFlowVolume":  -99999.0,
       "structureName":  "S375"
     }
    ]
   }
  }
 ],
 "messages": [
 
 ]
}

[Stefan_Jung]

Hi James Crandall‌,

i am not sure if i understand you question correct. First of all some questions from my side, is the JSON example generated by the Geoprocessing Service or is it manual generated? Looks like there are some bracket issues. But when i understand it correct, every result has the attributes pointX and pointY.

Do you want to access this attributes to have a Geometry? For just xy this is easy using Build Geometry from Fields.

Maybe I can help you if you provide some more information.

Best,

Stefan

[JamesCrandall]

Hi Stefan,

Thank you for the reply!  Yes, that is an actual output from a GP service we have running on our ArcGIS Server 10.4 at the moment.

every result has the attributes pointX and pointY

Yes, correct.

My question is: can this be used as an input to a GeoEvent server?  My understanding is that GeoEvent can "poll" other REST services at a defined interval, which we would use to update an output of a GeoEvent server.  Ultimately, we're thinking of using GeoEvent to update a Feature Service used in our other web products (AGOL and Portal).

(sorry if I'm incorrectly defining things, still new to the GeoEvent Server terminology)

[Stefan_Jung · Accepted solution]

Hi James Crandall‌,

yes GeoEvent can Poll a REST Endpoint in a interval of you choice to receive the data and update a feature class.

As Input Connector you can use "Poll an External Website for JSON". This Input Connector will perform a HTTP Request to the provided URL and expect JSON as result. 

There are a few parameters you should be aware of. "JSON Object Name" can be used to define the root node of your JSON. In this case it could be "results", but i am still not 100% clear with your JSON. It is not valid. Does result only contain on object and all the features you want to add are located at "results[0].result"?

"Construct Geometry From Fields" can be used to automatically generate a Geometry from pointX and pointY. If the inner "result" is your root element you can just write the names of the fields as X and Y because they are on the root level.

GeoEvent will also create a GeoEvent definition with a hierarchical structure that matches to your data. You should check if all datatypes are correct and maybe edit them if they are not correct. This is a automatic process according to the first feature that will be parsed.

If you want to write everything to one feature class you have to flatten the incoming data. Not sure if you already know how the output should look like, but the samples list might be a challenge. To flatten the data you create one more GeoEventDefinition that contains all field on the root level and does not contain any list. Using the Processor "Field Mapper" in the GeoEvent Service you can map the fields to the new GeoEvent Definition.

The last step would be to create an output "Update a Feature" to write all the data to your feature class. The output connector (wizard) can create publish a new feature class that matches to the GeoEvent Definition.

Hope this helps. It should also be possible to write into two different outputs. You could write the results to a feature class and the samples to a Table adding some foreign key if available. But this is an advanced topic. First of all you should try if the easy way works like expected.

Best,

Stefan

[JamesCrandall]

Thank you again, Stefan.  

I agree that's a bad example JSON (the "samples" array attribute is not useful for this question).  Also, the "Results" attribute is default for any Geoprocessing service output I believe.  

In any event, it does sound like we would have the ability to setup the appropriate out-of-the-box connectors in Geoevent server using a REST source.  

We have some requirements to process OSIsoft/Pi REST api's using our own wrappers that enrich them with the geometry attributes from our Geospatial data services.  In essence we have Geoprocessing services that consume both the REST api's and merge them together to create a new REST output.  However we need a way to format that output into an actual Feature Service that can be consumed by AGOL map viewer and we're exploring options to accomplish this.

We originally thought perhaps developing a Server Object Interceptor (SOI) for existing map/feature services could suffice, but I'm unsure if the volume of operational data would inhibit performance.  Geoevent Server seemed like a much more robust option and researching to determine if it would meet our needs.