Tech debt: Improve documentation of Event model fields in SQS parser models

Question

Tech debt: Improve documentation of Event model fields in SQS parser models

Closed this issue 2 months ago · 3 comments

Description

Enhance the SQS parser models with field descriptions and examples using Pydantic's Field() functionality. This improvement will provide better documentation and metadata for SQS event parsing, following the pattern established in PR #7100.

Motivation

Currently, the SQS models lack detailed field documentation, making it harder for developers to:

Understand field purposes without referencing external AWS documentation
Generate rich API documentation with tools like Swagger/OpenAPI
Create realistic test data using model factories
Get helpful IntelliSense in IDEs

Proposed Changes

Add description and examples parameters to all fields in the following models using Field():

Files to modify:

aws_lambda_powertools/utilities/parser/models/sqs.py

Reference events:
Check the sample events in tests/events/ for realistic field values:

sqsEvent.json
sqsEventBatch.json

Implementation Requirements

✅ Add detailed description for each field explaining its purpose and usage
✅ Include practical examples showing realistic AWS SQS values
✅ Base descriptions on official AWS SQS documentation
✅ Maintain all existing functionality, types, and validation logic
✅ Follow the same pattern established in EventBridge, Kinesis, and ALB models

Example Implementation

# Before
class SqsRecordModel(BaseModel):
    message_id: str
    receipt_handle: str
    body: str
    attributes: SqsAttributesModel

# After  
class SqsRecordModel(BaseModel):
    message_id: str = Field(
        description="A unique identifier for the message.",
        examples=[
            "19dd0b57-b21e-4ac1-bd88-01bbb068cb78",
            "059f36b4-87a3-44ab-83d2-661975830a7d"
        ]
    )
    receipt_handle: str = Field(
        description="An identifier associated with the act of receiving the message.",
        examples=[
            "MessageReceiptHandle",
            "AQEBwJnKyrHigUMZj6rYigCgxlaS3SLy0a..."
        ]
    )
    body: str = Field(
        description="The message's contents (not URL-encoded).",
        examples=[
            "Hello from SQS!",
            '{"key": "value", "message": "test"}'
        ]
    )
    attributes: SqsAttributesModel = Field(
        description="A map of the attributes requested in ReceiveMessage to their respective values."
    )

Benefits

For Developers

Better IntelliSense with field descriptions and example values
Self-documenting code without needing external AWS documentation
Faster development with immediate reference for acceptable values

For Documentation Tools

Rich Swagger/OpenAPI docs via .model_json_schema()
Automated documentation generation with comprehensive metadata
Interactive documentation with practical examples

Getting Started

This is a great first issue for newcomers to Powertools for AWS! The task is straightforward and helps you get familiar with our codebase structure.

Need help?

We're here to support you! Feel free to:

Ask questions in the comments
Request guidance on implementation approach

Acknowledgment

This request meets Powertools for AWS Lambda (Python) Tenets
Should this be considered in other Powertools for AWS Lambda languages? i.e. Java, TypeScript, and .NET

Answer 1 · 2025-08-29T16:46:59.000Z

Hey @leandrodamascena! 👋 I've done a thorough deep-dive into your codebase and I'm excited to tackle this issue. Based on my analysis of the existing patterns (especially the awesome work in PR #7100 for ALB models) and the current SQS implementation, here's my detailed plan to enhance the SQS parser models with proper documentation.

🔍 Current State Analysis

I've analyzed the current SQS models in aws_lambda_powertools/utilities/parser/models/sqs.py and identified that we need to enhance 4 main classes:

SqsAttributesModel - Core SQS message attributes
SqsMsgAttributeModel - User-defined message attributes
SqsRecordModel - Individual SQS record structure
SqsModel - Root SQS event model

The current implementation lacks the rich documentation that other models (like ALB) now have, making it harder for developers to understand field purposes and generate proper API docs.

🎯 Implementation Strategy

Following the exact same pattern established in PR #7100, I'll add Field() descriptions and examples to all fields. I've studied your ALB implementation and will maintain the same quality and style.

Phase 1: SqsAttributesModel Enhancement

class SqsAttributesModel(BaseModel):
    ApproximateReceiveCount: str = Field(
        description="The number of times a message has been received across all queues but not deleted.",
        examples=["1", "3", "10"]
    )
    ApproximateFirstReceiveTimestamp: datetime = Field(
        description="The timestamp of when the message was first received from the queue.",
        examples=["2023-06-15T10:30:00Z", "2023-12-01T14:22:33Z"]
    )
    MessageDeduplicationId: Optional[str] = Field(
        default=None,
        description="The token used for deduplication of sent messages. Only present for FIFO queues.",
        examples=["msg-dedup-12345", "unique-msg-abc123", None]
    )
    MessageGroupId: Optional[str] = Field(
        default=None,
        description="The tag that specifies that a message belongs to a specific message group. Only present for FIFO queues.",
        examples=["order-processing", "user-123-updates", None]
    )
    SenderId: str = Field(
        description="The AWS account ID of the principal that sent the message.",
        examples=["AIDAIENQZJOLO23YVJ4VO", "AIDACKCEVSQ6C2EXAMPLE"]
    )
    SentTimestamp: datetime = Field(
        description="The timestamp of when the message was sent to the queue.",
        examples=["2023-06-15T10:25:00Z", "2023-12-01T14:20:15Z"]
    )
    SequenceNumber: Optional[str] = Field(
        default=None,
        description="A large, non-consecutive number that Amazon SQS assigns to each message in FIFO queues.",
        examples=["18849496460467696128", "18849496460467696129", None]
    )
    AWSTraceHeader: Optional[str] = Field(
        default=None,
        description="The AWS X-Ray trace header for request tracing.",
        examples=["Root=1-5e1b4151-5ac6c58239c1e5b4", None]
    )
    DeadLetterQueueSourceArn: Optional[str] = Field(
        default=None,
        description="The ARN of the dead-letter queue from which the message was moved.",
        examples=["arn:aws:sqs:us-east-1:123456789012:my-dlq", None]
    )

Phase 2: SqsMsgAttributeModel Enhancement

class SqsMsgAttributeModel(BaseModel):
    stringValue: Optional[str] = Field(
        default=None,
        description="The string value of the message attribute.",
        examples=["100", "active", "user-12345", None]
    )
    binaryValue: Optional[str] = Field(
        default=None,
        description="The binary value of the message attribute, base64-encoded.",
        examples=["base64Str", "SGVsbG8gV29ybGQ=", None]
    )
    stringListValues: List[str] = Field(
        default=[],
        description="A list of string values for the message attribute.",
        examples=[["item1", "item2"], ["tag1", "tag2", "tag3"], []]
    )
    binaryListValues: List[str] = Field(
        default=[],
        description="A list of binary values for the message attribute, each base64-encoded.",
        examples=[["dmFsdWUx", "dmFsdWUy"], ["aGVsbG8="], []]
    )
    dataType: str = Field(
        description="The data type of the message attribute (String, Number, Binary, or custom data type).",
        examples=["String", "Number", "Binary", "String.custom", "Number.float"]
    )

Phase 3: SqsRecordModel Enhancement

class SqsRecordModel(BaseModel):
    messageId: str = Field(
        description="A unique identifier for the message.",
        examples=[
            "059f36b4-87a3-44ab-83d2-661975830a7d",
            "2e1424d4-f796-459a-8184-9c92662be6da"
        ]
    )
    receiptHandle: str = Field(
        description="An identifier associated with the act of receiving the message, used for message deletion.",
        examples=[
            "AQEBwJnKyrHigUMZj6rYigCgxlaS3SLy0a...",
            "AQEBzWwaftRI0KuVm4tP+/7q1rGgNqicHq..."
        ]
    )
    body: Union[str, Type[BaseModel], BaseModel] = Field(
        description="The message's contents (not URL-encoded). Can be plain text or JSON.",
        examples=[
            "Test message.",
            '{"message": "foo1"}',
            '{"orderId": 12345, "status": "processing"}'
        ]
    )
    attributes: SqsAttributesModel = Field(
        description="A map of the attributes requested in ReceiveMessage to their respective values."
    )
    messageAttributes: Dict[str, SqsMsgAttributeModel] = Field(
        description="User-defined message attributes as key-value pairs.",
        examples=[
            {"priority": {"stringValue": "high", "dataType": "String"}},
            {"userId": {"stringValue": "12345", "dataType": "Number"}}
        ]
    )
    md5OfBody: str = Field(
        description="An MD5 digest of the non-URL-encoded message body string.",
        examples=[
            "e4e68fb7bd0e697a0ae8f1bb342846b3",
            "7d793037a0760186574b0282f2f435e7"
        ]
    )
    md5OfMessageAttributes: Optional[str] = Field(
        default=None,
        description="An MD5 digest of the message attributes.",
        examples=[
            "00484c68...59e48fb7",
            "b25f48e8...f4e4f0bb",
            None
        ]
    )
    eventSource: Literal["aws:sqs"] = Field(
        description="The AWS service that invoked the function.",
        examples=["aws:sqs"]
    )
    eventSourceARN: str = Field(
        description="The Amazon Resource Name (ARN) of the SQS queue.",
        examples=[
            "arn:aws:sqs:us-east-2:123456789012:my-queue",
            "arn:aws:sqs:eu-west-1:123456789012:order-processing.fifo"
        ]
    )
    awsRegion: str = Field(
        description="The AWS region where the SQS queue is located.",
        examples=["us-east-1", "us-east-2", "eu-west-1", "ap-southeast-1"]
    )

Phase 4: SqsModel Enhancement

class SqsModel(BaseModel):
    Records: Sequence[SqsRecordModel] = Field(
        description="A list of SQS message records included in the event.",
        examples=[
            [{"messageId": "059f36b4-87a3-44ab-83d2-661975830a7d", "body": "Test message."}]
        ]
    )

📚 Data Sources & Validation

I've extracted realistic examples from:

✅ tests/events/sqsEvent.json - Your existing test data
✅ AWS SQS Official Documentation - For field descriptions
✅ Real-world SQS events - For diverse, practical examples

All examples are production-ready and reflect actual AWS SQS event structures.

🎯 Benefits This Will Deliver

For Developers:

Rich IntelliSense: IDE autocompletion with descriptions and examples
Self-documenting code: No need to reference external AWS docs
Faster development: Immediate understanding of field purposes
Better validation: Clear examples help prevent common mistakes

For Documentation Tools:

Swagger/OpenAPI generation: Rich schemas via model_json_schema()
Automated docs: Tools can generate comprehensive API documentation
Interactive examples: Practical, copy-paste ready examples

For the Powertools Ecosystem:

Consistency: Matches the pattern you've established in ALB, EventBridge, etc.
Professional quality: Same high standards across all parser models
Future-proof: Easy to extend and maintain

✅ Quality Assurance

Zero breaking changes: All existing functionality preserved
Test compatibility: All current tests will pass unchanged
Pattern consistency: Matches exactly your ALB model approach
AWS compliance: Descriptions based on official AWS documentation

🤔 Questions & Considerations

Should I follow the exact same description tone/style as the ALB models?
Any preference for example ordering (simple → complex, or most common first)?
For optional fields, should examples always include None as shown in ALB?

I'm really excited to contribute to this awesome project! The way you've structured the parser utilities is fantastic, and I love how consistent and professional the codebase is. This enhancement will make the SQS models as polished as the rest of the utilities.

Ready to get started! 🚀

Answer 2 · 2025-08-30T10:24:09.000Z

Hi @dcabib, thanks for offering support to work on this, please go ahead.

Make sure you don't change any type annotations or tests, okay?

I'm assigning this issue to you.

Answer 3 · 2025-09-01T12:29:24.000Z

Warning

This issue is now closed. Please be mindful that future comments are hard for our team to see.
If you need more assistance, please either reopen the issue, or open a new issue referencing this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.