SQS Record Shema - BinaryValue could be null

powertools-lambda-typescript/packages/parser/src/schemas/sqs.ts

Line 13 in fc87eb3

binaryValue: z.string().optional(),

When sending sqs messages with otel instrumented aws-sdk, we can receive the following attribute.

"messageAttributes": {
    "baggage": {
        "stringValue": "baggageAttr=someVal",
        "binaryValue": null, <-- binaryValue in this case can be null
        "stringListValues": [],
        "binaryListValues": [],
        "dataType": "String"
     },
     ...
}

resulting in a ParseError

"cause": [
    {
       "expected": "string",
       "code": "invalid_type",
       "path": [
           "messageAttributes",
           "baggage",
           "binaryValue"
       ],
       "message": "Invalid input: expected string, received null"
    }
...
]

Having had a quick look at the aws-sdk sqs otel instrumentation, I don't see binaryValue explicitly set anywhere to be null, hence I assume this could possibly be an OK behaviour.

BinaryValue Type - Base64-encoded binary data object (I guess since it's an object it can be null)

So, I think we should make it nullable()

Hi @artkhod, thanks for raising the issue. Could you provide more details on how you were trying to send the message to reproduce this.

This is the second time that a customer report this. We need to find a way to reproduce this.

See this: #3398

Hi @sdangol, producer lambda's handler can simply send any message to the SQS queue. After it's deployed, we can enable application signals for it from the console (configuration -> monitoring) or with a few statements in cdk. This would be the quickest way to instrument it with otel, and that would be my scenario.

Another lambda with SQS source would try to parse the first message with the SQS Record schema (originally, error occurred while parsing the record inside of the batch processor's recordHandler, but I doubt that it's necessary to reproduce this).

Thanks for the quick reply @artkhod. Will try to reproduce this as you said and look into it.

Hi @artkhod, thanks for reporting this. We were able to reproduce the behavior you described and indeed the SQS schema should be update to have the field as .nullable().

I created a reproduction example of the issue happening, which you can find here https://github.com/dreamorosi/repro-4436.

Interestingly enough, and also why we weren't able to reproduce it right away last time, the AWS CLI performs client-side validation against the binaryValue field being set to null, but using the AWS SDK works - which means the value is allowed.

Warning

This issue is now closed. Please be mindful that future comments are hard for our team to see.
If you need more assistance, please either reopen the issue, or open a new issue referencing this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

This is now released under v2.26.0 version!