URC with length and binary content

[mschnell1]

Some Modems send URCs with a payload length specification followed by a stream of non transparentized bytes.
AFAIU, the ATAT "URC" Parser can't handle this.
More generally, I seem to understand that the ATAT derive macros provide handling of "streaming" (serializing / deserializing / parsing) according to the types of the elements of a struct. Here unquoted binary information can be denoted by the heapless_bytes::Bytes type. But with same the payload length is defined by the Bytes count (when sending) and can't be denoted by another field in the message (when receiving).
To me it seems that for the case in question it would be good if it would be possible to specify an alternate parser (e.g. by an attribute macro in the appropriate Urc enum arm or in the OnReceive struct denoted by same). By that any kind of message could be handled.
Or maybe the parser can be defined to decode the the message up to length field and the binary payload after that can be received by some additional feature.
(I would be inclined to contribute to ATAT in order to implement such an enhancement. But I am relatively new to Rust and only have a faint impression on macro programming. So I would need some support on finding a starting point....)
-Michael

[mschnell1]

I found that the modem I use perfectly does what I expect:

• AT+QMTPUBEX can send a block of binary data as a payload (no quotes used here)
• the URC +QMTRCV - when the optional msg_length field is enabled - perfectly provides the binary payload (but silly, the payload is additionally wrapped in double-quotes, but quotes inside the payload are transferred normally)

(Instead of allowing for defining an optional msg_length to be stated with +QMTRCV (and the payload still documented as type "String") they better ad provided an option for a completely alternate URC such as +QMTRVCEX that has a length an unquoted binary payload.)

When using a string type in the URC Receive struct, of course we get a parsing error, when the received payload contains a double quote.

Astonishingly I don't get a parsing error when the payload contains a Comma, LF or CR character.

(Astonishingly) I do get parsing errors, when the received payload contains characters greater or equal 128. Supposedly this is as Rust requires Strings to be UTF-8 code.

-Michael

[mschnell1]

I found that the modem I use, when doing TCP transfer rather than MQTT transfer, provides a similar answer scheme with the +QIRD block. Other than with +QMTRECV here the binary information is separated from the length by CR LF ( rather than by , " with MQTT). (These modems are weird :( )
Did anybody already use such TCP communication via Modem by means of ATAT ?

[mschnell1]

I found that it's possible to link one's own parser to an arm of the AtatUrc derived Urc enum by not deriving the OnReceived... struct from AtatResp .
For a test I did this:

#[derive(Clone /* , AtatResp*/)]
pub struct OnReceivedTest {
    pub client_idx: u32,
    pub msgid: u32,
    pub topic: String<0x100>,
    pub payload_len: u32,
    pub payload: atat::heapless_bytes::Bytes<0x100>,
}

impl<'a> atat::serde_at::serde::Deserialize<'a> for OnReceivedTest {
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: atat::serde_at::serde::Deserializer<'a>,
    {
        let r = OnReceivedTest {
            client_idx: 0,
            msgid: 0,
            topic: "".into(),
            payload_len: 0,
            payload: atat::heapless_bytes::Bytes::<0x100>::new(),
        };
        Ok(r)
    }
}

Now I suppose I can do the deserialize() function to decode any data from the modem following the appropriate keyword denoted by the Urc enum.

I wonder if / how I can call the standard deserializer to decode the fields coming in before the binary payload.

[mschnell1]

Ranting on ....

In order to use serde and the "other site" of serde provided by ATAT, for parsing the non-binary part of the +QMTRECV urc (or similar stuff), I suppose I should do something like

#[derive(Clone, AtatResp)]
pub struct QMTRecvHeader {
    pub client_idx: u32,
    pub msgid: u32,
    pub topic: String<0x100>,
    pub payload_len: u32,
}

#[derive(Clone)]
pub struct OnReceivedQMTPayload {
    pub header: QMTRecvHeader,
    pub payload: atat::heapless_bytes::Bytes<0x100>,
}

impl<'a> atat::serde_at::serde::Deserialize<'a> for OnReceivedQMTPayload {
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: atat::serde_at::serde::Deserializer<'a>,
    {
        // create a resul for testing
        let mut r = OnReceivedQMTPayload {
            header: QMTRecvHeader {
                client_idx: 0,
                msgid: 0,
                topic: "".into(),
                payload_len: 0,
            },
            payload: atat::heapless_bytes::Bytes::<0x100>::new(),
        };

        let visitor =  ???
        r.header = deserializer.deserialize_any(visitor)?;

or somehow using
       atat::serde_at::deserialize_struct();

        Ok(r)
    }
}

Now trying to learn

• what deserialize_struct might do
• how such a serdes visitor needs to be constructed, then
• checking if deserialize_any() indeed makes sense and finally
• finding out how my parser can receive the following payload_len bytes from ATAT. (maybe by deserialize_bytes ? )

-Michael

[mschnell1]

I suppse the appropriate serde documentation will be needed to be adhered.
-> https://serde.rs/custom-serialization.html

[MathiasKoch]

I think for this it would be nice if we could provide a struct type that can be used for the field, with the general structure along https://docs.rs/tokio-util/latest/tokio_util/codec/length_delimited/

[ijager]

See #186