topic Re: NMEA Adapter in ArcGIS GeoEvent Server Questions

NMEA Adapter

KeithNightlinger — Wed, 19 Feb 2014 21:12:17 GMT

I am working with the new NMEA adapter and using the socket connector it only appears to be processing the GPGGA message. I have GeoEvent definitions for for all three (GGA, RMC, and GLL) and I know my GPS device is sending all three by sending it to another port.

Would there be a reason why only one message type is coming through the GeoEvent Processor?

Also, how is the DeviceID assigned? I have 30 vehicles that will be sending in a NMEA signal. Using the GGA message, the device ID shows as 4. How can I correlate this device ID with a Vehicle ID?

Thanks in advance.

Keith

Re: NMEA Adapter

JoelSmith1 — Thu, 20 Feb 2014 12:58:41 GMT

Try changing your Filter Properties of your GeoEvent Service to something like this (using the OR operand):

       $DEFINITION_NAME = NMEAGPGGA
OR    $DEFINITION_NAME = NMEAGPGLL
       $DEFINITION_NAME = NMEAGPRMC

As far as the Device ID, I'm wondering the same thing.

Thanks,
Joel

Re: NMEA Adapter

KeithNightlinger — Thu, 20 Feb 2014 13:07:11 GMT

Thanks for the suggestion. This is actually without a filter sending directly to the tcp-console. I even setup a standalone TCP listener and can see the data coming in:

Standalone TCP Listener - pre-geoevent processor
$GPGGA,144358.0,4154.891742,N,08818.761683,W,1,06,0.8,206.4,M,-34.0,M,,*64
$GPRMC,144358.0,A,4154.891742,N,08818.761683,W,0.0,359.9,200214,,,A*71

tcp-console - after going through geoevent processor
Received Event : NMEAGPGGA,5,02/20/2014 08:37:52 AM,"-88.31268728333333,41.9148789,0.0",1,6,0.8,205.9,M,-34.0,M,,

Re: NMEA Adapter

KeithNightlinger — Thu, 20 Feb 2014 18:30:40 GMT

I am working a guess here but it looks like the GPRMC GeoEvent Definition might be out of order. It looks like the Validity check field should come before the Geometry, unless the connector was built to reorder the fields during processing. If someone from ESRI could chime in here it would be helpful.

Field order of GeoEvent Definition
DeviceId
TimeStamp
Shape
GEOMETRY
Validity
Speed
Course
Variation
EastWest

Field order of GPRMC NMEA Sentence
      1   220516     Time Stamp
      2   A          validity - A-ok, V-invalid
      3   5133.82    current Latitude
      4   N          North/South
      5   00042.24   current Longitude
      6   W          East/West
      7   173.8      Speed in knots
      8   231.8      True course
      9   130694     Date Stamp
      10 004.2      Variation
      11 W          East/West
      12 *70        checksum

Actual sentance
$GPRMC,144358.0,A,4154.891742,N,08818.761683,W,0.0,359.9,200214,,,A*71

Re: NMEA Adapter

VladislavPlechnoy — Thu, 20 Feb 2014 22:10:32 GMT

Keith,

NMEA adapter implements all three formats (GGA, RMC, and GLL). It appears that it interprets GGA messages correctly and for some reason fails on other two. We need to troubleshot this issue to find out what is going on. Source code for this adapter is available in our public GitHub repository. Please feel free to review it at 'https://github.com/Esri/nmea-for-geoevent' I will let you know when I have a fix for this issue.

DeviceID is reported by the device as part of GeoEvent. If you would like to correlate this ID with vehicle ID you need to use a field enricher processor that will enrich geoevents with vehicle ID. Required mapping should be provided during field enricher configuration. If you need help setting that up please let me know. I can assist you with that.

-Vlad.

Re: NMEA Adapter

KeithNightlinger — Thu, 27 Feb 2014 14:28:50 GMT

Keith,

Required mapping should be provided during field enricher configuration. If you need help setting that up please let me know. I can assist you with that.

-Vlad.

Vlad,

I would like to take you up on this offer, how can we get started?

Thanks,
Keith

Re: NMEA Adapter

VladislavPlechnoy — Mon, 17 Mar 2014 20:08:48 GMT

Keith,

This is how you setup your Field Enricher (File) processor:
1) Create a csv file similar to one attached to this post and put it in the folder that one of your registered folder data stores is pointed at
2) In your service designer add a new processor and choose 'Field Enricher (File)' from the drop down called 'Processor'
3) Give it a name of your choice
4) In 'Registered Folder' drop-down choose the folder data store that you are going to use or click 'Register Folder' button to create a new one
5) Put your csv file name into 'File Name' property
6) 'File Join Field' property value should represent the field name in your csv file that will be used for linking with incoming GeoEvent (For instance from the csv file attached choose: FlightNumber)
6) Fill out 'Enrichment Fields' property value with coma-separated list of fields (For instance from the csv file attached choose: PilotName,PilotPhone)
7) 'Field Tags' property is optional. Leave it blank for now.
😎 'GeoEvent Join Field' property value should specify the field name or tag on the GeoEvent that will be used for linking.
9) Specify new GeoEvent Definition name in the 'Resulting GeoEvent Definition Name' property value.

Let me know if you require more help.

Thanks,
Vlad.

Re: NMEA Adapter

JoelSmith1 — Tue, 18 Mar 2014 12:27:16 GMT

Keith,

NMEA adapter implements all three formats (GGA, RMC, and GLL). It appears that it interprets GGA messages correctly and for some reason fails on other two. We need to troubleshot this issue to find out what is going on. Source code for this adapter is available in our public GitHub repository. Please feel free to review it at 'https://github.com/Esri/nmea-for-geoevent' I will let you know when I have a fix for this issue.

DeviceID is reported by the device as part of GeoEvent. If you would like to correlate this ID with vehicle ID you need to use a field enricher processor that will enrich geoevents with vehicle ID. Required mapping should be provided during field enricher configuration. If you need help setting that up please let me know. I can assist you with that.

-Vlad.

I'm in the same boat... is there an incident / bug open for this issue? Looking forward to the fix!

Thanks,
Joel

Re: NMEA Adapter

VladislavPlechnoy — Thu, 27 Mar 2014 21:41:43 GMT

The NMEA connector has been updated with a new JAR on the Product Gallery.

The update can be downloaded from its page:
NMEA" rel="nofollow" target="_blank">http://www.arcgis.com/home/item.html?id=1717c2f7fe5b497491234ce3a352bdd2]NMEA 0183 for GeoEvent Processor

This update fixes translation of RMC and GLL messages.

Thanks!

Re: NMEA Adapter

SamuelRichmond — Sat, 11 Dec 2021 13:43:47 GMT

I believe there is still a bug in the adaptor affecting the RMC NMEA format translation. GGA works with no issues, but from looking at the code on github for the NMEAGPRMCMessageTranslator.java file I notice on line 44 it says:

geoEvent.setField(i++, toPoint(data[3], data[5], "N".equals(data[4]), "E".equals(data[6])));

I'm looking at the strings my GPS device is sending and in field 6 I have "W" not "E". I think this is causing the connector to error out. If so, I would think the "N" could be an "S" as well and cause the same issue.

GGA works fine. I can't test GLL because my device doesn't support it. I do get errors in my GeoEvent Processor logs that read "Exception while translating a NMEA message : NMEAGPRMC message data is invalid."

Here is a sample of the data I am getting from the GPS:

$GPRMC,052037.0,A,3029.916471,N,09032.533741,W,0.0,,060614,,,A*5F
$GPRMC,052038.0,A,3029.916472,N,09032.533740,W,0.0,,060614,,,A*52
$GPRMC,052039.0,A,3029.916458,N,09032.533756,W,0.0,,060614,,,A*5C
$GPRMC,052040.0,A,3029.916448,N,09032.533767,W,0.0,,060614,,,A*51
$GPRMC,052041.0,A,3029.916448,N,09032.533767,W,0.0,,060614,,,A*50
$GPRMC,052042.0,A,3029.916448,N,09032.533767,W,0.0,,060614,,,A*53
$GPRMC,052043.0,A,3029.916448,N,09032.533767,W,0.0,,060614,,,A*52
$GPRMC,052044.0,A,3029.916448,N,09032.533767,W,0.0,,060614,,,A*55

Any help is much appreciated!

Re: NMEA Adapter

SamuelRichmond — Sat, 11 Dec 2021 13:43:50 GMT

I've been doing more testing and tried using the GPS 2 IP application that was shown in the NMEA connector documentation for testing. It works with no issues for RMC, GLL, and GGA. Here is a sample of the data it is sending to GEP.

$GPRMC,063815,A,3029.918,N,09032.527,W,0.00,217.3,060614,003.1,W*4F
$GPRMC,063817,A,3029.918,N,09032.527,W,0.00,217.3,060614,003.1,W*4D
$GPRMC,063819,A,3029.918,N,09032.527,W,0.00,217.3,060614,003.1,W*43
$GPRMC,063821,A,3029.918,N,09032.527,W,0.00,217.3,060614,003.1,W*48
$GPRMC,063823,A,3029.918,N,09032.527,W,0.00,217.3,060614,003.1,W*4A
$GPRMC,063825,A,3029.918,N,09032.527,W,0.00,217.3,060614,003.1,W*4C
$GPRMC,063827,A,3029.918,N,09032.527,W,0.00,217.3,060614,003.1,W*4E
$GPRMC,063829,A,3029.918,N,09032.527,W,0.00,217.3,060614,003.1,W*40

The differences I noticed are in the time stamp field. The GPS2IP doesn't send a decimal number (seconds), the GPS device does. The app also gives True Course, and Variation where the GPS leaves them blank. The last thing I noticed is the app has 12 fields, where the GPS device has 13. I may be on to something!

Line 55-56 of the NMEAGPRMCMessageTranslator.java code has this snippet:

if (data == null || data.length != 12)
       throw new ValidationException("NMEAGPRMC message data is invalid.");

If I'm not mistaken, it is counting fields to validate that the format is correct. I've been doing some digging and the number of fields varies on just about every manufacturer's documentation for the RMC NMEA sentence between 12 and 13 fields. I haven't gone as far as purchasing the official paper from NMEA to get all of the possible sentences because it's kind of pricey, but I did find this http://www.catb.org/gpsd/NMEA.html#_rmc_recommended_minimum_navigation_information and it seems very complete.

If you disregard the "count" that the document gives the fields, remove the field for "FAA mode indicator" (#12) because it seems like a lot of equipment follows an old version of the specification that doesn't include it and THEN hand count ALL of the fields including the $--RMC field, you have 13.

I guess my point is that for RMC sentences you may have 12, 13, or even 14 fields in there and the code doesn't account for that which is causing an error for me and probably others.

Thoughts? Do you think the RMC message translation code needs to take this into account?

P.S. After looking at the GLL message translator code, it counts 7 fields and the NMEA document I found for the GLL sentence has 9 total. If you remove the FAA mode indicator, it's 8. The same problem (assuming it is the problem) is probably happening with it as well.

Re: NMEA Adapter

SamuelRichmond — Fri, 06 Jun 2014 15:59:40 GMT

I downloaded the source, modified the code, and recompiled the module this morning and was able to get the adapter to accept RMC sentences from my device. I submitted a pull request to the project on github to patch the source. Let me know if i can help in any way with testing things further.

BTW, the device I'm using is a CradlePoint IBR-600LE-PWR mobile router with GPS. The planned application is to track police cars as well as provide in-car data to laptops and other equipment in the car.

Re: NMEA Adapter

RJSunderman — Fri, 27 Jun 2014 22:36:56 GMT

Hello Samuel �??

I apologize for not replying sooner to your posts to the forum. Thank you for your initiative in downloading and reviewing the source available [url=https://github.com/Esri/nmea-for-geoevent]here[/url] on GitHub. I noticed from the exchanges on GitHub that you are working with Vlad, who is the developer you want to be in touch with.

I think that you are spot on with all of your observations. The NMEA specification is quite detailed and contains many different message sentence variants. What the GeoEvent Processor team put up on GitHub is only an example of how the product's SDK can be used to develop a custom adapter. As you've observed, the number of fields in a GPRMC message can vary. Because GeoEvent Processor requires a GeoEvent Definition to allow an adapter to parse a payload (e.g. NEMA message) from received event data, we would have to provide a potentially large number of event definitions if we were to advertise comprehensive support for all the different NMEA sentence variants. The initial offering only provides samples for three (GPGGA, GPGLL, and GPRMC).

It appears, based on your message from 5-June that the validation code has a bias for �??North�?? and �??East�?? coordinate values. I�??m considering this a bug and will submit an issue against it. The validation logic which checks the number of fields, however, is tied to what the adapter plans to do with the data in those fields. I�??ve a feeling that if we were to simply relax the validation to allow GPRMC sentences with 12, 13, or 14 fields that other users might want to know why the adapter was ignoring data values in the extended fields, which goes back to what I was saying about needing a potentially large number of event definitions to satisfy a large variety of potential NMEA sentences.

Thank you for your observations and the technical detail in your posts. If we do not immediately integrate your pull request on GitHub it is most likely because we want to maintain the NMEA adapter offering as a sample which folks can tailor for their specific devices (as you were able to do) rather than trying to maintain an adapter which supports "all possible" sentence structures.

Best Regards �??
RJ

Re: NMEA Adapter

SamuelRichmond — Wed, 02 Jul 2014 17:02:16 GMT

RJ,

I agree that due to the size of the NMEA catalog, making an adapter that includes all of the available sentences is a huge task. Perhaps a separate connector for each RMC sentence size would be in order then and they could be included in the basic NMEA adaptor download. The basic sample adapter covers the majority of what most people will use as far as sentences go, and my personal opinion is that if ESRI wants to offer a sample that with with RMC, they should get it right so it works with ANY RMC sentence. Maybe leave it up to the end user to figure out the number of fields their equipment is reporting and allow them to chose the proper one.

As for the wonder by end users as to why certain fields are being ignored, as far as I know, the fields that exist in the range of 12-14 aren't used in the sample RMC connector either. My code just extends the ignoring to the extra fields. I'm far from an expert, but the fields most will need are still covered. RMC sentences are for GPS tracking and the only fields one should need for that are UTC Time, Status, Lat, Lat Direction, Lon, Lon Direction, Speed over ground (knots), and date. All of these are still picked up with the modification I made.

Having sample adapters is great and I appreciate having something to look at to get me started, as it did with the custom adapter I made for my CradlePoint routers to add System IDs to identify which device is which between 60 separate vehicles in the fields. I feel like most users would expect basic NMEA RMC sentences to work, no matter the number of fields when downloading a sample program that says that's what it does. It didn't for me and I had to troubleshoot it to fix it myself. A lot of the products users don't have the time or possibly knowledge to do what I did. Please consider this in any of your internal discussions regarding this issue. I'm not saying my code is 100% right. It works, but it's my own personal hack, for my own application. I submitted the pull request to bring attention to the problem and possibly get it fixed for everyone's benefit, whether my modification is used or not.

Again, thanks for responding to me an taking a look at the problem. Let me know if I can be of any assistance working this out. I'd also share the CradlePoint Router adapter I made if ESRI wants to make it available for others to use. I know there are a lot of people out there trying to do the same thing I am and it would save them several all-nighters and stress to just be able to plug it in and go.

Re: NMEA Adapter

Lorena_AndreaPeñuela — Wed, 20 May 2015 19:59:14 GMT

Hi,

I´m working with Geoevent and the app GPS2IP, but I have a problem:

The app doesn’t updated the coordinates, due to the app always use the same coordinate (+27.5578,-122.778), although this coordinate is the EEUU but I’m in Colombia.

Any of you had this same problem?

The NMEA connector works perfectly, but Geoevent always get the same coordinate.

May you help me please?

Regards,

Lorena