Question on Multicardinal Field Splitter Processor

Question

Thank you RJ Sunderman‌ for your detailed explanation of processing a hierarchical multicardinal XML data structures. I do have some questions on issues I'm having using the Multicardinal Field Splitter.

I have a geoevent definition like this:

And I need to split every event by the groupOfLocations.globalNetworkLinear.linearWithinLinearGNElement attribute. But it seems, the Multicardinal Field Splitter has some problems with splitting on attributes which are nested in a hierarchy:

Do you have any idea how to setup this correctly?

Thanks,

Matej

MatejVrtich · Answer

RJ Sunderman‌, by chaining Multicardinal Field Splitter processors, I can handle the stream of data exactly how I need.

Thank you for your help!

Matej

RJSunderman · Answer

Matej –

Using my test data above, I can configure three MCFS (multicardinal field splitter) processors to split the data first on the group element metric_collection, then on the group element Level1_Group3, and finally on the array Group3_Items. Following this pattern I think you'll be able to achieve what you need to do.

Above, I've illustrated the three event records routed from the final field splitter to the output. Please make sure you have downloaded the latest release of the field splitter processor bundle (Release 8 - June 24, 2020) for release 10.8.1 of ArcGIS. This version includes changes which allow multiple MCFS processors to be arrayed in series. If you are using an earlier release of ArcGIS, please use Release 7 (February 27, 2020).

Hope this information is helpful –
RJ

RJSunderman · Answer

Hello Matej –

Unlike a Field Mapper or Field Calculator which needs you to specify the full "path" into a data structure, the Field Splitter and in particular Multicardinal Field Splitter processors only expect the name of the element at which you want the split to be applied. So the Field to Split parameter's value should only specify the element's base name, linearWithinLinearGNElement, not the full path to the element (e.g. groupOfLocations.globalNetworkLinear.linearWithinLinearGNElement).

That said, you may have found a limit of what the processor is able to handle in terms of a deeply nested, hierarchical, multicardinal, data structure. To test what you are trying to do I created some sample JSON data which adapts using the GeoEvent Definition illustrated below. Note that it does not matter whether the data received is formatted as XML or JSON – the inbound connector has to adapt whatever data is received to create an event record which can be routed to a GeoEvent Service for processing.

After receiving the JSON above and allowing the inbound connector to adapt it using the illustrated GeoEvent Definition, I can use a Field Mapper to extract, for example, the data value "Delta" (line 21 in the sample JSON) by specifying the full path down through the data structure to the "E301" element in the array:

metric_collection.Level1_Group3.Group3_Items[1].E301

If I specify the same full path when configuring the Multicardinal Field Splitter, I see the error message you were able to capture from the system log: Field name <name> is invalid. However, if I specify only Group3_Items (the name of the array) as the field on which to conduct the split, I see a different error message in the log indicating a null pointer exception was encountered.

I am in contact with one of our developers to review the source code of this processor. We will see what we can do to better handle the case where Group3_Items (the name of the array) is specified and why a null pointer is not being caught. The work around, for now, is to use multiple Multicardinal Field Splitter processors in series and "walk down" the hierarchy from the top to the multicardinal element on which you want the final split applied. I'll reply to this post with what I tested so you can see what I had to do to implement the work around.

Please make sure you have downloaded the latest release of the field splitter processor bundle (Release 8 - June 24, 2020) for release 10.8.1 of ArcGIS. This version includes changes which allow multiple MCFS processors to be arrayed in series. If you are using an earlier release of ArcGIS, please use Release 7 (February 27, 2020).

Hope this information is helpful –
RJ