Using JSON input - is there a way to use all elements from parent and child objects?

4113
4
11-13-2013 06:12 PM
DennisGeasan
Occasional Contributor II
I am using an input connector that is polling a REST endpoint which is delivering JSON. The results are this:

{"driverID":83910282,
"vehicleID":83887977,
"time":1384359889000,
"loc":{
   "lat":18.949661,
    "lng":-70.169067,
    "head":null,
    "time":null,
    "speed":null,
    "latitude":18.949661,
    "longitude":-70.169067,
    "heading":null},
"speed":0,
"head":5,
"odometer":null}


I need the attributes from all the fields, those in the parent and the sub object "loc".

A GeoEvent Definition was automatically created. In that the field "loc' is defined as a group and includes the four fields that are not null in the JSON object "loc".  However, when I direct the output to a CSV file the "loc" field is empty.  Also, you can't edit the group field but you can edit the items in the group but you can't add fields to the group.

In the Input configuration, if I fill in the field "JSON Object Name" with the value 'loc', then I get ONLY the four non-null values in the JSON "loc" object.  I want to get all of the fields.  How do I accomplish that?
Dennis Geasan
GIS Technologies
0 Kudos
4 Replies
AdamMollenkopf
New Contributor
It is known limitation at 10.2 that the 'Write to .csv file' output connector does not properly output fields on GeoEvents that are field groups .  Please try to use the 'Write to .json file' and expect you will see the results you are expecting.  You can copy and paste the contents of the produced .json file into an online formatter like jsonlint.org to see it in a more readable format.  The 'Write to .json file' output connector supports Field Groups and fields that have multiple cardinality.

Hope this information is helpful,
Adam Mollenkopf
ArcGIS GeoEvent Processor for Server - Product Lead
0 Kudos
DennisGeasan
Occasional Contributor II
Does output to a feature class work with the field groups and fields with multiple cardinality?  And could you explain the difference between "field groups and fields with multiple cardinality" and when each would be used?  Thanks.

Dennis Geasan
0 Kudos
RJSunderman
Esri Regular Contributor
Hello Dennis -

Interpreting a JSON structure as simple comma separated text when the JSON contains groups and fields whose cardinality is greater than one can be confusing. This is the reason for the known limit when non-trivial JSON event data is output to either TCP/Text or CSV/File output. The behavior will be the same at the 10.2.1 release as it was at the 10.2.0 release. As Adam suggests, the correct way to view your output would be to use a JSON/File output or write the event data to GEP"?s cache using the "?Publish GeoEvents on a REST endpoint"? and then query the endpoint using the f=json formatting specification.

Support for editing a GeoEvent Definition generated by GEP when the event definition has fields of type group and/or fields whose cardinality is greater than one is limited at the 10.2.0 release. Product enhancements at 10.2.1 provide the functionality I believe you are looking for.

The event definition generated by GEP will depend on the structure of the first JSON event received. As you observed, fields whose value is null are omitted from the field definition. At 10.2.1 you will be able to edit the event definition to explicitly add fields and specify data types for the fields whose published values are occasionally null.

Event data which contains fields of type group and/or fields whose cardinality is greater than one cannot be used to update a feature service without first flattening the data structure. This is the reason the Field Mapper processor element was originally developed. Since the fields in a feature class must be discrete types (e.g. String, Long, Double, Date, etc.) you would use a GEP field mapper to map the data from the JSON field loc.longitude to a field Longitude of type Double.

Your last question, to explain "field groups and fields with multiple cardinality"� is probably best addressed with an illustration. In the image below (click to enlarge), the JSON structure has a single field, a group, named FlightGroup. FlightGroup has nested within it two other groups, Pilot and FlightInfo, as well as simple/flat fields (Flight and Altitude). Within the group FlightInfo you"?ll see an element named geometry which is actually a list containing two elements. Thus you see that groups can contain groups, and sub-groups can contain lists (fields with multiple cardinality)

[ATTACH=CONFIG]29122[/ATTACH]


  • The value for the flight"?s origin airport would be:  FlightGroup.FlightInfo.OriginAirportCode

  • The geometry value of the second list item, the one with the non-zero "?z"? value, would be: FlightGroup.FlightInfo.geometry[1]


Hope this information helps -
RJ
0 Kudos
DennisGeasan
Occasional Contributor II
That helps a lot. I'll check out the Field Mapper and look forward to the updates in 10.2.1. Thanks a lot. DG
0 Kudos