Encoding and sending NMEA2000 messages
Closed this issue · 18 comments
Unlike NMEA0183, which uses a simple text format, NMEA2k messages are binary and unintelligible to the eye. Further, the binary forms is proprietary and only available to licence holders. However, the canboat project has reverse-engineered many of the PGNs and published PGN descriptors.
I am aiming to
Step 1: write aJavaScript constructor that will deliver an empty object for any PGN that has a PGN descriptor.
The user can populate the object with the parameter values to be sent.
Step 2: write an encoder that will generate the relevant binary message
Step 3: add to the plugin the ability to send the binary message out through OCPN.
This work will follow the receiving and parsing work #74 as that will provide insights into how to handle the encoding.
@TwoCanPlugIn Can you fill me in here?
From opencpn_plugin.h
CommDriverResult WriteCommDriverN2K(
DriverHandle handle, int PGN, int destinationCANAddress, int priority,
const std::shared_ptr <std::vector<uint8_t>> &payload);You remark that the 'stupid' header in received data is dummy (apart from pgn) and I note that the source and destination values therein that I have seen are always 255.
So if they are dummy because OCPN does not pass them on, where does destinationCANAddress and priority come from for writing?
Hi Tony,
When I used the term 'dummy' it was only in reference to the CRC checksum for data received from the SocketCAN interface, not to the header information (pgn, priority, source & destination address).
My comment regarding 'stupid' header, was that when Dave & the powers that be decided to initially support NMEA 2000 and chose the Actisense NGT-1, they made the design decision to adopt the Actisense data format and pass the entire Actisense payload to the API (with the actual NMEA 2000 payload beginning at byte 13). They consequently adopted the same format for the SocketCAN interface by prepending the NMEA 2000 header information and appending a 'dummy' CRC checksum. I shan't go into the specifics of NMEA Fast message re-assembly other than to say it is something that is performed inherently by 'smart' devices such as the NGT-1 and is performed by OpenCPN for 'dumb' devices such as SocketCAN.
The CRC checksum is not part of the NMEA 2000 standard, it is just something that is part of the Actisense data format that enables a USB connected device to verify that a received message has not been corrupted during transmission over the USB connection. The CRC checksum should vary and be accurate for data received from the Actisense and Yacht Devices interfaces but as I pointed out, it is just a 'dummy' value for data received from the SocketCAN interface.
The header information in the received data (pgn, source & destination address, priority) should accurately reflect what was transmitted over the NMEA 2000 network and was received by OpenCPN.
If you are transmitting onto the network you should use the appropriate destination address, transmission rate & priority as received or as defined by the NMEA standard. For example the priority for PGN 127233 (Man Overboard Notification), is defined as '3', the valid destination address is defined as 'Global' (e.g. 255). You will find that most PGN's are sent to the global address (255), it is only some of the 'weird' network management and request/response PGN's that are sent to an individual address. The source address is managed & handled internally by OpenCPN.
Hope this helps.
@duichan Build b8e07d6 has a first attempt at pushing NMEA2000 out. The plugin registers a pgn the first time it is sent, as required by NMEA2000. It copes with multiple N2k connections, although that is, perhaps, unlikely.
Here is a script that attempts a send.
The script sends pgn 126208 with a request to all stations to return pgn 126998. I have no idea if this makes sense.
If you uncomment line 35, it will display the pgn 126208 it has prepared.
In ocpn_plugin.h we have
CommDriverResult WriteCommDriverN2K( DriverHandle handle, int PGN,
int destinationCANAddress, int priority,
const std::shared_ptr <std::vector<uint8_t>> &payload);
The implication is that the 'payload' has the nominal Actisense header like in the received payload, even though all the header information is provided by separate arguments. If this does not seem right, you can remove the Actsense header by uncommenting line 37. In this case, only the NMEA2000 data will be sent.
The script sets up a number of listeners for pgn 126998 before sending pgn 126208. After 10 seconds it cancels the outstanding listeners.
As I do not have NMEA2000 here at home and my boat is laid up in Norway, I cannot test it further. I have configured an NMEA2000 connection and it seems to send the pgn 126208 into the void with no complaint. Can you give it a whirl? Feel free to fiddle with the script if you can improve what I am blindly trying with my limited NMEA2000 knowledge.
Thanks in anticipation.
Just following with interest.
I can't comment on your Javascript code, however a few observations.
The payload is "the NMEA 2000 payload". No Actisense header or any other extraneous stuff.
You've picked possibly the worst PGN to experiment with. PGN 126208 Request/Command/Acknowledge Group function is rather complex, and to be honest, very few devices support it and for most requested PGN's all you will receive in response is most likely a NAK. In fact the only device that I've had success using it is to send commands to a Raymarine autopilot where PGN 126208 is used to set fields in various Raymarine proprietary PGN's which command the autopilot.
From a testing perspective, if you just want to get a response from devices on the network, use PGN 59904 (ISO Request) to request various PGN's such as 60928 (Address Claim), 126996 (Product Information) Few devices support PGN 126998 (Configuration Information). OpenCPN does respond to requests for PGN's 60928 & 126996.
This is a snippet of code illustrating the above:
std::vectorpayload;
payload.push_back(60928 & 0xFF);
payload.push_back((60928 >> 8) & 0xFF);
payload.push_back((60928 >> 16) & 0xFF);
auto sharedPointer = std::make_shared<std::vector<uint8_t> >(std::move(payload));
result = WriteCommDriverN2K(driverHandle, 59904, 255, 5, sharedPointer);
If you want to transmit for example depth data (for viewing in the dashboard) the payload will be:
unsigned char sid = 0xFF;
unsigned int depth = some value;
short offset = some value;
unsigned char maxRange = some value;
payload.push_back(sid);
payload.push_back(depth & 0xFF);
payload.push_back((depth >> 8) & 0xFF);
payload.push_back((depth >> 16) & 0xFF);
payload.push_back((depth >> 24) & 0xFF);
payload.push_back(offset & 0xFF);
payload.push_back((offset >> 8) & 0xFF);
payload.push_back(maxRange & 0xFF);
.....
result = WriteCommDriverN2K(driverHandle, 128267, 255, 5, sharedPointer);
Finally if you are using Linux as development platform, you don't need a physical NMEA 2000 network, you can configure a virtual can interface (vcan). Refer to the NMEA 2000 info at https://opencpn.org/wiki/dokuwiki/doku.php?id=opencpn:manual_basic:toolbar:options:connections:nmea2000
@TwoCanPlugIn Thank you very much for this - most helpful. Studying what you have given me has thrown up some issues in the way I am doing things, so I need to rework somewhat. Now you have confirmed the header is not included in the outgoing payload, I think it best not to include it in the JavaScript object. This will avoid clashes in attribute names used in both the header and data. Also I am not yet handling STRING_FIX.
@duichan please hold off testing for now.
Re strings, IIRC there are only two types.
Fixed length, unterminated ASCII strings Eg. the fields in PGN 126996 (Product Information), PGN 129794 (AIS Class Ai Static & Voyage Data; the vessel name),
and strings preceded with a length & encoding byte (Unicode or ASCII). Eg., PGN 129041 (AIS AtoN) for the AtoN name, 126998 (Configuration Information) for each of the three fields, 130074 (Waypoints) for the waypoint names.
These strings are transmitted as
byte[n] = string length (includes the length & encoding byte)
byte[n+1] = encoding, 1 = ASCII, 0 = Unicode
byte[n+2]....byte[n + string length -2] the string itself.
I've never seen any device transmit these strings as anything other than ASCII.
Good luck & best wishes for the festive season.
@duichan I have taken onboard the useful input from @TwoCanPlugIn and assumed only the N2k data should be sent.
Please install build fdd049d from my alpha repo, which includes an updated NMEA2000 constructor.
Here is a new script to test sending a 59904 to request a 126996 response.
testOutput 2.js.zip
When this is working, I will build the send stuff into an object push( ) method.
Best wishes for the season.
Sorry, doesn't seem to work:
Data to send:[20,240,1]
result: Time is up. Responses 0
I have confirmed that I can still receive PGNs by running your earlier script, so I guess something is wrong with the transmit side.
Have a good Christmas and New Year
I have now decoded the log from my B&G MFD. The relevant line is this:
pri 3, pgn 59904, src 72, dst 255, len 3, data: 01,01,01 --- UNKNOWN ---
Clearly, although JavaScript thinks it is sending [20, 240, 1] as data, the display is actually receiving [01,01,01]
I hope this helps you to locate the problem.
Success I think.
The js program responds thus:
[147,145,6,20,240,1,255,0,255,255,255,255,134,52,8,148,99,86,117,108,99,97,110,32,53,32,77,70,68,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,48,49,48,48,48,95,69,32,49,56,46,51,46,54,49,46,49,46,49,53,53,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,48,48,49,51,49,56,35,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,2,1]
Repeated 10 times
Happy New Year to you too
I have been out of action for a while with COVID, despite the latest booster %-#)
The above decodes to:
{
"PGN": 126996,
"id": "productInformation",
"description": "Product Information",
"priority": 6,
"desination": 255,
"origin": 0,
"nmea2000Version": "2.1000000",
"productCode": 25492,
"modelId": "Vulcan 5 MFD",
"softwareVersionCode": "01000_E 18.3.61.1.155",
"modelVersion": "",
"modelSerialCode": "001318#",
"certificationLevel": 2,
"loadEquivalency": 1
}
which makes sense.
I have now encapsulated the write stuff in the push method.
The repeated 10 times highlights a fundamental issue with my single-shot approach to listening.
Build a64bab3 is an experimental build with an enduring NMEA2000 listener. Please give it a whirl with the following script:
// experiment with sending N2000 to request all stations respond
Nmea2000 = require("NMEA2000");
pgnToSend = 59904;
pgnToReceive = 126996;
OCPNonNMEA2000(receive, pgnToReceive); // listen repeatedly
onSeconds(timeOut, 10);
responses = 0;
// construct and send message
nmeaSend = new Nmea2000(pgnToSend, null, {trace:0});
nmeaSend.pgn = pgnToReceive; // pgn to request
nmeaSend.push();
function receive(what){
print("\n", responses, "\t", what, "\n");
responses++;
}
function timeOut(){
OCPNonNMEA2000();// Cancel listener
stopScript("Time is up. Responses " + responses);
}
All the best for 2024. I hope it is better than 2023 world-wise.
Sorry you have had COVID. I am hearing about quite a lot of people that have had it recently despite being up to date with booster jabs. For the last few years I have similarly expressed the wish to everyone that the new year turns out to be better than the last. Sadly that hasn't been the case but I try to remain optimistic.
Your decode of the data looks right, certainly as far a model and serial number are concerned.
I have tried the latest script. I am reproducing it in full below because there are 5 responses, all of which are very slightly different:
0 [147,145,6,20,240,1,255,0,255,255,255,255,134,52,8,148,99,86,117,108,99,97,110,32,53,32,77,70,68,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,48,49,48,48,48,95,69,32,49,56,46,51,46,54,49,46,49,46,49,53,53,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,48,48,49,51,49,56,35,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,2,1]
1 [147,145,6,20,240,1,255,1,255,255,255,255,134,52,8,148,99,86,117,108,99,97,110,32,53,32,105,71,80,83,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,48,49,48,48,48,95,69,32,49,56,46,51,46,54,49,46,49,46,49,53,53,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,48,48,49,51,49,56,35,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,2,0]
2 [147,145,6,20,240,1,255,2,255,255,255,255,134,52,8,148,99,86,117,108,99,97,110,32,53,32,69,99,104,111,32,40,84,104,105,115,32,117,110,105,116,41,32,32,32,32,32,32,32,48,49,48,48,48,95,69,32,49,56,46,51,46,54,49,46,49,46,49,53,53,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,48,48,49,51,49,56,35,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,2,0]
3 [147,145,6,20,240,1,255,4,255,255,255,255,134,52,8,234,6,84,119,111,67,97,110,32,80,108,117,103,105,110,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,50,46,49,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,84,119,111,67,97,110,32,80,108,117,103,105,110,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,49,50,49,57,56,55,54,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1]
4 [147,145,6,20,240,1,255,3,255,255,255,255,134,52,8,148,99,86,117,108,99,97,110,32,53,32,78,97,118,105,103,97,116,111,114,32,32,32,32,32,32,32,32,32,32,32,32,32,32,48,49,48,48,48,95,69,32,49,56,46,51,46,54,49,46,49,46,49,53,53,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,48,48,49,51,49,56,35,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,2,0]
@duichan The above decodes to:
Does this make sense? Is it complete? When I configure the Garmin displays onboard, I can see a list of the NMEA2000 devices. I wonder if you can learn this about your system here.
I am interested in whether we are catching all the responses or missing some.
I attach two versions of the script which should decode and display as in the above.
whoseThere.js.zip
This one decodes and displays on the fly. If you run it repeatedly, do you get the same result? Perhaps the number and order of the responses vary?
If the responses are pretty immediate and you are waiting for the timeout, you can reduce the time from 10 seconds in line 7.
whoseThereDeferred.js.zip
This variant stashes the responses and defers the work of decoding and display until after the time out.
Is there any difference?
I have not been able to test this stashing and deferring, so you might have to fix it.
Thanks for your continuing help in this.
Your list is correct. The B&G Vulcan unit correctly appears as four different logical devices, though it is in fact only one physical device.
Output from first script:
Source ProdCode ModelId softwareVersionCode modelSerialCode Certification loadEquivalency
0 25492 Vulcan 5 MFD 01000_E 18.3.61.1.155 001318# 2 1
1 25492 Vulcan 5 iGPS 01000_E 18.3.61.1.155 001318# 2 0
2 25492 Vulcan 5 Echo (This unit) 01000_E 18.3.61.1.155 001318# 2 0
4 1770 TwoCan Plugin 2.1 1219876 0 1
3 25492 Vulcan 5 Navigator 01000_E 18.3.61.1.155 001318# 2 0
result: Time is up. Responses 5
Output from second script:
0 25492 Vulcan 5 MFD 01000_E 18.3.61.1.155 001318# 2 1
1 25492 Vulcan 5 iGPS 01000_E 18.3.61.1.155 001318# 2 0
2 25492 Vulcan 5 Echo (This unit) 01000_E 18.3.61.1.155 001318# 2 0
4 1770 TwoCan Plugin 2.1 1219876 0 1
3 25492 Vulcan 5 Navigator 01000_E 18.3.61.1.155 001318# 2 0
Time is up. Responses 5
Source ProdCode ModelId softwareVersionCode modelSerialCode Certification loadEquivalency
0 25492 Vulcan 5 MFD 01000_E 18.3.61.1.155 001318# 2 1
1 25492 Vulcan 5 iGPS 01000_E 18.3.61.1.155 001318# 2 0
2 25492 Vulcan 5 Echo (This unit) 01000_E 18.3.61.1.155 001318# 2 0
4 1770 TwoCan Plugin 2.1 1219876 0 1
3 25492 Vulcan 5 Navigator 01000_E 18.3.61.1.155 001318# 2 0
result: undefined
Apart from the missing header and outputting the data twice, the two scripts give identical results.
Glad to be of assistance.
Early on I took a design decision that listeners for any kind of event were single-shot - cancelled after the event. To listen repeatedly, the script needed to set it up again.
In the above tests, we are listening for responses to a single prompt from each address and I have been unsure how to manage that. The last tests were done with a special build that made the listening multi-shot. But if I adopt this, it would change the behaviour of existing scripts.
Avoiding that is going to be complicated. Before I embark down that path, I want to check whether it really is necessary. So, @duichan I need to ask you to do another test.
Build 09c3e90 has single-shot behaviour restored. Please test with this script.
whoseThereDeferred.js.zip
Please run a few times to see if it consistently receives all five responses. I am hoping it might. Otherwise, I have more work ahead.
OK I understand the problem. Unfortunately this latest version seems to give inconsistent and incomplete responses:
1st attempt:
0 25492 Vulcan 5 MFD 01000_E 18.3.61.1.155 001318# 2 1
1 25492 Vulcan 5 iGPS 01000_E 18.3.61.1.155 001318# 2 0
2 25492 Vulcan 5 Echo (This unit) 01000_E 18.3.61.1.155 001318# 2 0
4 1770 TwoCan Plugin 2.1 1219876 0 1
2nd & 3rd attempt:
0 25492 Vulcan 5 MFD 01000_E 18.3.61.1.155 001318# 2 1
4th & 5th attempt:
0 25492 Vulcan 5 MFD 01000_E 18.3.61.1.155 001318# 2 1
1 25492 Vulcan 5 iGPS 01000_E 18.3.61.1.155 001318# 2 0
6th attempt:
0 25492 Vulcan 5 MFD 01000_E 18.3.61.1.155 001318# 2 1
7th & 8th attempt:
0 25492 Vulcan 5 MFD 01000_E 18.3.61.1.155 001318# 2 1
1 25492 Vulcan 5 iGPS 01000_E 18.3.61.1.155 001318# 2 0
Drat!!! But thanks for testing.