Pyserial and tinyproto example

[redemption95]

Hi @lexus2k ,
I am trying to run the sketch from python3 and pyserial.
Here is my code.

import tinyproto
import time
import serial

p = tinyproto.Hdlc()
ser = serial.Serial('/dev/ttyS0', 57600, parity = serial.PARITY_NONE, timeout= 0)

def on_read(a):
    print("Received bytes: " + ','.join( [ "{:#x}".format(x) for x in a ] ) )

def on_send(b):
    print("Callback")
    print("Wrote bytes: " + ','.join( [ "{:#x}".format(x) for x in b ] ) )

# setup protocol
p.on_read = on_read
p.on_send = on_send
p.begin()

# provide rx bytes to the protocol, obtained from hardware rx channel
p.rx( bytearray([ 0x7E, 0xFF, 0x3F, 0xF3, 0x39, 0x7E ]) )

# buf = tinyproto.
# bytearray buf = [ 0x7E, 0xFF, 0x3F, 0xF3, 0x39, 0x7E  ]

while True:
    buf = bytearray([ 0x7E, 0xFF, 0x3F, 0xF3, 0x39, 0x7E  ])
    print("Writing byte array to serial")
    print(p.tx(buf))
    ser.write(p.tx(buf))

I get 0 from the p.tx function assumed packet cannot be formed.
There is no more explicit python doc except the wrapper class.
Please let me know if I am doing anything wrong or an proper
example to achieve full duplex communcation ( Threaded ).

Thank you,
Sasank Panda

[lexus2k]

Hi @githubforsank ,

I will check and let you know.

Best regards.

[lexus2k]

The modified example is provided below. There are multiple issues in the provided sketch:

1. p.tx() method provides the list of bytes to send. Each time you call the function it gives you fresh data to send to serial port. So, you need to remember the bytes first before printing to the console, because those bytes must be also send to serial port
2. You didn't use p.put() method, which actually puts message for sending.

p.tx() returns the bytes, that must be send to the serial port
p.rx(buf) processes the bytes, which were obtained earlier from the serial port by the application
p.put(msg) prepares message for sending

import tinyproto
import time
import serial

p = tinyproto.Hdlc()
ser = serial.Serial('/dev/tty0', 57600, parity = serial.PARITY_NONE, timeout= 0)

def millis():
    return round(time.time() * 1000)

def on_read(a):
    print("Received bytes: " + ','.join( [ "{:#x}".format(x) for x in a ] ) )

def on_send(b):
    print("Callback")
    print("Wrote bytes: " + ','.join( [ "{:#x}".format(x) for x in b ] ) )

# setup protocol
p.on_read = on_read
p.on_send = on_send
p.begin()

lastTs = millis() - 1000 # Force to send first message immediately

while True:
    if millis() - lastTs >= 1000:
        # Send message every second
        buf = bytearray([ 0x7E, 0xFF, 0x3F, 0xF3, 0x39, 0x7E  ])
        p.put( buf )
        lastTs += 1000
    # print("Writing byte array to serial")
    bytesForSerialPort = p.tx()
    # print( bytesForSerialPort )
    ser.write( bytesForSerialPort )

[redemption95]

Hi @lexus2k,
Apologies for reopening this.
But I am unable to get the call back receive function.
Consider this snippet.

Read function

def on_read(a):
    print("Received bytes: " + ','.join( [ "{:#x}".format(x) for x in a ] ) )

p.rx( bytearray([ 0x7E, 0xFF, 0x3F, 0xF3, 0x39, 0x7E ]) )
while True:
    if ser.in_waiting > 0:
        s = ser.read()
        print("Byte recieved :", s)
        p.rx(s)

The output of the byte received is as such :

Received bytes: 0xff,0x3f
Byte recieved : b'~'
Byte recieved : b'H'
Byte recieved : b'e'
Byte recieved : b'l'
Byte recieved : b'l'
Byte recieved : b'o'
Byte recieved : b' '
Byte recieved : b'W'
Byte recieved : b'o'
Byte recieved : b'r'
Byte recieved : b'l'
Byte recieved : b'd'
Byte recieved : b'!'
Byte recieved : b'\x00'
Byte recieved : b'~'

Although I can see I received the packets in the proper format
I am unable to get the call-back function.
The line
p.rx( bytearray([ 0x7E, 0xFF, 0x3F, 0xF3, 0x39, 0x7E ]) )
does call the callback but the recieved data doesn't call the callback.

I am doing something wrong here.
Plesae help.
Thank you ,
Sasank Panda

[redemption95]

Further observations :
The p.rx for python only works for this bytearray.
p.rx( bytearray([ 0x7E, 0xFF, 0x3F, 0xF3, 0x39, 0x7E ]) )

It fails to call the callback for even a bit changed in the payload.

p.rx(bytearray([ 0x7E, 0xFF, 0x4F,  0xF3, 0x39, 0x7E  ]))
p.rx( bytearray([ 0x7E, 0xFF, 0x3F, 0x02, 0xF3, 0x39, 0x7E ]) )
p.rx(bytearray([ 0x7E, 0xFF, 0x4F,  0x30, 0xF3, 0x39, 0x7E  ]))

Seems some issue with the wrapper arguments or library.
Can you please confirm this behavior @lexus2k

[lexus2k]

Hi @githubforsank

The root cause in your case is that you have CCITT16 enabled (this is behavior of Python implementation). Thus all incoming data passed to rx() method must have valid CRC field value. But in your example, all different packets have the same CRC 0xF3, 0x39.

You need to provide correct CRC.
If you want to disable CRC completely, let me know, I will provide Python update for you.

Best regards.

[redemption95]

Thank you @lexus2k for identifying that.
Then I am sure the error is from the Arduino library.

Consider this snippet

#define CONFIG_ENABLE_FCS16
#include <TinyProtocol.h>
#include <AltSoftSerial.h>

tinyproto::Packet<20> hdlc_packet;
int buff[256];
tinyproto::Hdlc proto_hdlc(buff, 256);
AltSoftSerial host;



void hdlcRecieveCallback(tinyproto::IPacket &pkt)
{
    Serial.println("HDLC Recieve callback");
    Serial.println(pkt.getString());
    Serial.println("Recieved data size : " + String(pkt.size()));
    Serial.println("Max buffer size : " + String(pkt.maxSize()));
    Serial.println("=========================");
}
void setup() {
    /* No timeout, since we want non-blocking UART operations. */
    Serial.setTimeout(0);
    host.setTimeout(0);
    /* Initialize serial protocol for test purposes */
    host.begin(115200);
    Serial.begin(9600);
    proto_hdlc.setReceiveCallback(hdlcRecieveCallback);
    proto_hdlc.begin();
    proto_hdlc.enableCrc16();

hdlc_packet.put((uint8_t)0xFF);
hdlc_packet.put((uint8_t)0x3F);

Serial.println(hdlc_packet.size());
proto_hdlc.write(hdlc_packet);

}
void loop()
{
    uint8_t byte;
    int count = 0;
    if ( proto_hdlc.run_tx( &byte, 1 ) == 1 ) //FD protocol fills buffer with data, we need to send to the channel
    {
        while( host.write(byte) == 0 );
    }
}

I have called the enableCrc16 method here.
But the packet I receive on the python side seems
to have no CRC bytes. As you said you have not provided
the disableCrc() in the python wrapper. As a result,
the callback function is not called as discussed in the issue above
#21 (comment)

Byte recieved : b'~'
Byte recieved : b'\xff'
Byte recieved : b'?'
Byte recieved : b'~'

Can you please tell me why the crc16 bytes are not added to the frame.
Also, it would be very helpful if you could provide the disableCrc() wrapper,
so that i can disable it for the timebeing.

[lexus2k]

Hi

The issue here is: begin() method (it initializes the protocol) is called before changing CRC field enableCrc16(). You need to swap 2 lines of code

    host.begin(115200);
    Serial.begin(9600);
    proto_hdlc.setReceiveCallback(hdlcRecieveCallback);
    proto_hdlc.enableCrc16();
    proto_hdlc.begin();

Also, it would be very helpful if you could provide the disableCrc() wrapper,

In Arduino environment such method already exists. As for Python, please check latest commit b559257 - it add new property for Python:

   p.crc = 16  # To enable CRC 16
   p.crc = 0  # To set default CRC
   p.crc = 0xFF # To disable CRC

Remember, that with CRC disabled the protocol works faster, but there is no guarantee that the delivered message is ok.

[redemption95]

Hi @lexus2k ,

Thank you for the python methods.
Really appreciate it.

I swapped the lines. Now I get the checksum bits
but I don't think those are the correct values.
The code still remains the same as here. #21 (comment)

Byte recieved : b'~'
Byte recieved : b'\xff'
Byte recieved : b'?'
Byte recieved : b'\x00'
Byte recieved : b'\x00'
Byte recieved : b'~'

[lexus2k]

You need to log all bytes being sent and received from UART on both ends. Only when you have all those data logged, it will be possible to say what's wrong in a communication.

[lexus2k]

Hi, I'm closing the issue.
if you have any question feel free to reopen it.