Token parameter in Message object is ignored

[vixtor]

No matter what token is set in Message object constructor, actual request contains some randomly generated token.

This is the simple code from example, only setting a custom token.

   context = await Context.create_client_context()

    await asyncio.sleep(1)
    
    #payload = b"The quick brown fox jumps over the lazy dog.\n" * 30
    request = Message(code=GET, payload=payload, token=b']\xb4\xba,\xd1\x05\xd2\xf3', uri="coap+tcp://.../...")
    print(request)
    response = await context.request(request).response

If we print the message object, we see that it has token 5db4ba2cd105d2f3

DEBUG:asyncio:Using selector: EpollSelector
<aiocoap.Message at 0x7f609cf18070: no mtype, GET (no MID, token 5db4ba2cd105d2f3) remote UndecidedRemote(scheme='coap+tcp', hostinfo='...:...'), 1 option(s)>

But debug shows that actual message object being sent is a different one, without the token.

DEBUG:coap:Sending message: <aiocoap.Message at 0x7f609c9e2dc0: no mtype, 7.01 Csm (no MID, empty token) remote None, 2 option(s)>

Response from server shows that even though that this other message did not contain token, it still got one, and server sent it along with the response:

DEBUG:coap:Sending request - Token: ca21, Remote: <TcpConnection at 0x7f609cf180a0, hostinfo ..., local 10.0.2.15:55948>
DEBUG:coap:Sending message: <aiocoap.Message at 0x7f609cf18070: no mtype, GET (no MID, token ca21) remote <TcpConnection at 0x7f609cf180a0, hostinfo ..., local 10.0.2.15:55948>, 1 option(s)>
DEBUG:coap:Received message: <aiocoap.Message at 0x7f609cf18040: no mtype, 7.01 Csm (no MID, token 5e02ae63188f4818) remote <TcpConnection at 0x7f609cf180a0, hostinfo ..., local 10.0.2.15:55948>>
DEBUG:coap:Received message: <aiocoap.Message at 0x7f609cefed00: no mtype, 2.05 Content (no MID, token ca21) remote <TcpConnection at 0x7f609cf180a0, hostinfo ..., local 10.0.2.15:55948>, 2 option(s), 284 byte(s) payload>
DEBUG:coap:Response <aiocoap.Message at 0x7f609cefed00: no mtype, 2.05 Content (no MID, token ca21) remote <TcpConnection at 0x7f609cf180a0, hostinfo ..., local 10.0.2.15:55948>, 2 option(s), 284 byte(s) payload> matched to request <PlumbingRequest at 0x7f609cefed60 around <aiocoap.Message at 0x7f609cf18070: no mtype, GET (no MID, token ca21) remote <TcpConnection at 0x7f609cf180a0, hostinfo ..., local 10.0.2.15:55948>, 1 option(s)> with 2 callbacks>
Response token: b'\xca!'

Tokens are optional by specification, and it should be possible to set any token.

Debug info:

Python version: 3.9.0 (default, Oct  6 2020, 00:00:00) 
[GCC 10.2.1 20200826 (Red Hat 10.2.1-3)]
aiocoap version: 0.4.1
Modules missing for subsystems:
    dtls: missing DTLSSocket
    oscore: missing cbor2, filelock, ge25519
    linkheader: missing LinkHeader
    prettyprint: missing LinkHeader, cbor2, termcolor, pygments
Python platform: linux
Default server transports:  tcpserver:tcpclient:tlsserver:tlsclient:udp6
Selected server transports: tcpserver:tcpclient:tlsserver:tlsclient:udp6
Default client transports:  tcpclient:tlsclient:udp6
Selected client transports: tcpclient:tlsclient:udp6
SO_REUSEPORT available (default, selected): True, True

[chrysn]

In the transition from handling messages to handling requests/responses, the token setting is replaced with programmatic matching (ie. the response being fed into the request's .response awaitable). That the token can be set at all is more an artifact of the message class being used all through the stack. Same for tie message ID -- even though not all transports use them. What do you want to set the token for? In CoAP that may be an indication of using it in a transport-dependent way.

[vixtor]

I was testing an application, so I needed to set a specific token.

[chrysn]

If it's just about producing a particular token, you can reach into the producing machinery like this: ``` for interface in ctx.request_interfaces: interface.next_token = iter([b"tok1", b"tok2", b"tok3"]).__next__ ``` (This is going over all interfaces, so if you sent requests over different transports, *all* of them would send with tok1 first and then tok2; once exhausted, they'll completely fail to send and raise weird StopIteration errors -- don't let it get there). It's definitely not recommended for any real application, and frankly I'm not a big fan of test vectors that prescribe to applications which particular value to use (as that leads to APIs that are confusing in production), but for your application it should do the right thing. The interface between token manager and context is not exactly public API, so you may want to introduce a note there to look at this again when updating aiocoap, but at least I don't plan to make incompatible changes around there any time soon.

[vixtor]

Thanks.