NMEA2000 -> NMEA0183 conversion

[ronzeiller]

For openCPN or similar we need the NMEA2000 converted to NMEA0183.
Signal K is internally used for all values

The questions are:

1. What to do if incoming values will be calculated due to calibration (tables) and/or offset?
   We need variables for that, as Signal K is limited

2. What to do if we need two different Signal K values for one NMEA0183 sentence?
   E.g. VWR – Relative Wind Speed and Angle sentence
   Signal K has no combined value for that.

Therefore we have a problem doing things like:
void SKNMEAVisitor::visitSKEnvironmentWindAngleTrue(.......)
for building the VWR sentence in SKNMEAVisitor.cpp where we also need Wind Speed.

IMHO NMEA2000, Signal K, NMEA0183, SeaSmart and so on are incoming and outgoing data formats which should not used for internally storing datas as the formats do not fit together.

In other words:
Signal K is not needed at all if someone just wants KBox for NMEA2000 -> NMEA0183 (openCPN and similar)
or wants KBox for NMEA2000 -> Actisense format for Expedition (instead an Actisense NGT-1)
You know, that KBox is mentioned on the Expedition Homepage? 👍
Personally I liked your KMessage much more as it gave me much more flexibility 😞

[ronzeiller]

@sarfata
Can you advise if following would work?
(After adding EnvironmentWindAngle, EnvironmentWindSpeed etc.in signalk.json and code generation)

void SKNMEAVisitor::visitEnvironmentWindAngleTrue(const SKUpdate &u, const SKPath &p, const SKValue &v) {

  float WindAngle;
  float WindSpeed;
  WindAngle = SKRadToDeg(v.getNumberValue());
  
  // WindAngle and WindSpeed should come together......
  // TODO: which is coming first, Angle or Speed? Does this matter?
  
  // Validation
  if (WindAngle<0||WindAngle>360||!u.hasEnvironmentWindSpeedTrue()) return;

  WindSpeed = getEnvironmentWindSpeedTrue();  // in m/s

  // continuing with building NMEA0183  VWR and (deprecated) MWV sentence
  // ........

[sarfata]

1. What to do if incoming values will be calculated due to calibration (tables) and/or offset? We need variables for that, as Signal K is limited

This is a very good question. We should also think of how we could use KBox to calculate things like the true wind speed/direction.

A solution that I am thinking of would be to introduce a new concept of "DataComputer". They would be managed by SKHub and every time a new piece of data is published it would be passed through to all the registered DataComputers and they could replace the data by another one, or generate new data. I have not implemented this yet.

2. What to do if we need two different Signal K values for one NMEA0183 sentence?

What you are doing is a solution for now. I brought the visitor concept from the old KMessage style but maybe it was not such a good idea and instead we should just override the visit method and have a list of if-statement that tests if the data we need is included (and maybe some configuration) and then call a function that generates a specific sentence. Would you like this better? I think I would.

[sarfata]

IMHO NMEA2000, Signal K, NMEA0183, SeaSmart and so on are incoming and outgoing data formats which should not used for internally storing datas as the formats do not fit together.

Yes for NMEA and NMEA2000 but not really for signalk. The problem with my KMessage solution is that I was completely re-inventing signalk. They already have done a very good job of looking at every possible NMEA/NMEA2000 sentence to come up with one canonical representation of the data. With KMessage I would just be adding more and more variables and build the same thing again.

Also, having member variables for every possible variable on the boat was not scalable. The object would be huge. The SKUpdate system allows us to manipulate sets of boat data but only using a minimal amount of memory.

[ronzeiller]

OK, as longer I study the Signal K, I like it more 😄

The general problem seems to be:

1. we have event driven tasks, like (very fast) incoming NMEA2000, and NMEA0183 on Serial/USB coming in.
2. we have to calculate things depending on (time-)different or (interface-)different incoming datas, like
   Leeway (Boatspeed 1Hz, Heel around 10Hz)
   or True Wind (AWS and AWA 10Hz, Boatspeed 1Hz, Leeway, Heel 10Hz, Hdg 10Hz)
   or Ground Wind (True Wind + COG, SOG) and so on.....
3. NMEA0183 sentences depend sometimes on (time-)different and (interface-)different incoming datas too
4. Then we have not so much time critical things like interval driven monitor pages.
5. Logging could be time critical (depending on written format, SD-Card type)
6. Fast writing NMEA0183 to Serial output (immediately after a sentence is built together)
7. Filtered writing NMEA0183 at 4800 baud output (time filtered, selected sentences only)
8. For later: NMEA200issue and upcoming discussion at Signal K too: "same" datas from different sources, as Hdg from Autopilot and internal sensor, or 2 sensors on the boat, .....

A solution that I am thinking of would be to introduce a new concept of "DataComputer". They would be managed by SKHub and every time a new piece of data is published it would be passed through to all the registered DataComputers and they could replace the data by another one, or generate new data. I have not implemented this yet.

This sounds great!

What you are doing is a solution for now. I brought the visitor concept from the old KMessage style but maybe it was not such a good idea and instead we should just override the visit method and have a list of if-statement that tests if the data we need is included (and maybe some configuration) and then call a function that generates a specific sentence. Would you like this better? I think I would.

Whatever is working 👍
Who is calling this building/calculating function when?
An interval task, an event*) or a mixture?

*) e.g. NMEA2000 PGN Apparent Wind is coming -> Signal K two different values updated -> the second update drives the NMEA0183 sentence build for Apparent Wind

[sarfata]

we have event driven tasks, like (very fast) incoming NMEA2000, and NMEA0183 on Serial/USB coming in.
we have to calculate things depending on (time-)different or (interface-)different incoming datas, like
Who is calling this building/calculating function when?

All very true. I think at some point the "right" thing to do might be to switch from event driven to frequency driven.

KBox would receive data from all sensors at whatever speed they come in, and would generate new data at a frequency set by the user.

For example:

• we receive GPS at 1Hz, Wind at 4Hz, Boat Speed at 1Hz
• when data is received it is stored in memory (with IIR filter, etc)
• a "calculator" is called at fixed interval (configured by the user) and generates new message that are sent to WiFi, NMEA, NMEA2000

Advantages:

• We have all the data we need to calculate new values. We can look at the freshness of data to decide if we should publish anything (for example we do not publish if data is older than 1sec)
• We can generate data at different speed for different network (NMEA at 4800 bauds, NMEA2000, WiFi, logging, etc)

Disadvantages:

• This will introduce a delay between getting a new data and repeating it to other peripherals

An even better approach would be to synchronize the clocks so that the data generated is in phase with the data we receive. This way there is no delay but this can get very complicated very quickly.

Conclusion: I am not 100% sure what is the right solution here. Probably keeping a mix of event based and timer based is the best compromise for now.

[ronzeiller]

@sarfata

For example:

we receive GPS at 1Hz, Wind at 4Hz, Boat Speed at 1Hz
when data is received it is stored in memory (with IIR filter, etc)
a "calculator" is called at fixed interval (configured by the user) and generates new message that are sent to WiFi, NMEA, NMEA2000

I think we have to divide things into calibration/correction of incoming sensor datas and calculation of (new) datas from different incoming values.

Calibration/correction, where no other incoming datas are needed, or datas needed like heel, which are "in our hands":

• Boat Speed: in-linearity in lower speed range by a table, general offset, compensate different bows if sensor is not in center of the boat, Cosine(Leeway) correction,....
• Apparent wind speed/angle: correction sensor-height-> 10m, upwash/downwash, offset for misalignment., some geometric corrections due to heel, correction of wind due to boat movement,...
• Heading Magnetic: corrections to True North

Those calibrations/corrections have to be done, when a value arrives.

Calculations:
The sensor with the highest interval is boat speed (usually 1 Second). All other values come more often, usually up to 4x higher. (GPS could come from rapid SOG/COG)
So an incoming boat speed could trigger all calculations, because all calculations (Current, True Wind, Performance) are dependent from boat speed anyway.
The time lag error (for calculations) is then maximum quarter of a second if all values from memory (SKUpdate) are taken, but a freshness check of all the other values does not hurt.

So for the beginning I would say we could start with the boat speed triggering and not with a calculation-interval-task.

Added 8.1.2018: See PR #117

Writing to output interfaces:
N2k PGNs are written nearly immediately, or not?
Above a certain baud rate all datas on NMEA output could be written immediately too?
4800 baud writing would need something else.....

[ronzeiller]

Closed, as N2k -> NMEA conversion already implemented.
Reference to this discussion concerning DataComputer or Performance calculations made in PR #117