Feature Request: support IP address with port in `requestContext.http.sourceIp` when parsing requests via TLS

Question

Feature Request: support IP address with port in `requestContext.http.sourceIp` when parsing requests via TLS

Closed this issue 2 months ago · 16 comments

Expected Behaviour

Model validates without error.

Current Behaviour

[ERROR] ValidationError: 1 validation error for APIGatewayProxyEventV2Model
requestContext.http.sourceIp
  value is not a valid IPv4 or IPv6 network [type=ip_any_network, input_value='10.1.15.242:39870', input_type=str]
Traceback (most recent call last):
  File "/var/task/main.py", line 63, in lambda_main
    return apigateway_main(event, context)
  File "/opt/python/aws_lambda_powertools/middleware_factory/factory.py", line 140, in wrapper
    response = middleware()
  File "/opt/python/aws_lambda_powertools/utilities/parser/parser.py", line 116, in event_parser
    parsed_event = parse(event=event, model=model)
  File "/opt/python/aws_lambda_powertools/utilities/parser/parser.py", line 200, in parse
    return _parse_and_validate_event(data=event, adapter=adapter)
  File "/opt/python/aws_lambda_powertools/utilities/parser/functions.py", line 84, in _parse_and_validate_event
    return adapter.validate_python(data)
  File "/opt/python/pydantic/type_adapter.py", line 421, in validate_python
    return self.validator.validate_python(

To point out: sourceIp is expecting an IP or CIDR, not an IP-with-port.

Code snippet

from aws_lambda_powertools.utilities.parser import event_parser
from aws_lambda_powertools.utilities.parser.models import APIGatewayProxyEventV2Model
from aws_lambda_powertools.utilities.typing import LambdaContext


@event_parser
def lambda_main(event: APIGatewayProxyEventV2Model, context: LambdaContext):
    return {
        'statusCode': 200,
        'headers': {
            'Content-Type': "application/json",
        },
        'body': "OK",
        'isBase64Encoded': False,
    }

Possible Solution

No response

Steps to Reproduce

Add layer arn:aws:lambda:us-east-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python312-x86_64:22 and invoke this function through API Gateway.
Get a custom domain onto Cloudflare and construct a routing chain like this:
```
graph LR;
A(["client"]);
B(Cloudflare);
C(CloudFront);
D(API Gateway);
E(Lambda);
A --> B;
B -- "mTLS" --> C;
C --> D;
D --> E;
```
Loading
where "mTLS" represents Cloudflare's "Authenticated Origin Pull" option. Cloudflare's Origin CA needs to be uploaded to CloudFront.

Powertools for AWS Lambda (Python) version

latest

AWS Lambda function runtime

3.12

Packaging format used

Lambda Layers

Debugging logs

Answer 1 · 2025-08-31T06:58:27.000Z

Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

Answer 2 · 2025-08-31T09:32:06.000Z

Expected Behaviour

Model validates without error.

Current Behaviour

[ERROR] ValidationError: 1 validation error for APIGatewayProxyEventV2Model
requestContext.http.sourceIp
value is not a valid IPv4 or IPv6 network [type=ip_any_network, input_value='10.1.15.242:39870', input_type=str]
Traceback (most recent call last):
File "/var/task/main.py", line 63, in lambda_main
return apigateway_main(event, context)
File "/opt/python/aws_lambda_powertools/middleware_factory/factory.py", line 140, in wrapper
response = middleware()
File "/opt/python/aws_lambda_powertools/utilities/parser/parser.py", line 116, in event_parser
parsed_event = parse(event=event, model=model)
File "/opt/python/aws_lambda_powertools/utilities/parser/parser.py", line 200, in parse
return _parse_and_validate_event(data=event, adapter=adapter)
File "/opt/python/aws_lambda_powertools/utilities/parser/functions.py", line 84, in _parse_and_validate_event
return adapter.validate_python(data)
File "/opt/python/pydantic/type_adapter.py", line 421, in validate_python
return self.validator.validate_python(
To point out: sourceIp is expecting an IP or CIDR, not an IP-with-port.

Hey @iBug thanks for opening this issue. I'm curious about the use case where you can have IP and port in the APIGW HTTP. Are you using VPC Links to access your APIGW? Because in normal situations it should be only source IP but without port, thats why we only support IPv4/IPv6 and not CIDR.

I'm more than happy to fix this, I just want to understand a little bit more.

Answer 3 · 2025-08-31T15:12:23.000Z

@leandrodamascena No, I'm not having any special setup, just a regular Lambda function behind API Gateway, only serving / accessing the public internet. It's exactly what my code received: An RFC 1918 private address with port. I don't think I'd ever bother with such an address, but it's failing Pydantic model validation.

Answer 4 · 2025-09-01T08:34:17.000Z

@leandrodamascena No, I'm not having any special setup, just a regular Lambda function behind API Gateway, only serving / accessing the public internet. It's exactly what my code received: An RFC 1918 private address with port. I don't think I'd ever bother with such an address, but it's failing Pydantic model validation.

Thanks for the additional explanation. I definitely couldn't reproduce this situation using public or private DNS. I always get an IP format without a port or CIDR.

I've also read specific parts of RFC 2616, RFC 7231, and the most recent 9110, and I don't see any clear mention that the source IP must include the port. I'm contacting the APIGW team to understand in which situations this might happen.

Another important point is that Pydantic doesn't have a built-in type to validate IP addresses with ports -at least I don't know of - so if this is truly impacting your production systems, please let me know and I'll be happy to release a new version of Powertools with a field_validator for sourceIp field to handle these cases. It doesn't hurt.

@sthulb @svozza @dreamorosi do you have any clues about this?

Answer 5 · 2025-09-01T08:45:51.000Z

Hi, thanks for the tag.

I've never seen this happening with API Gateway, but I've checked the Zod implementation and our schemas in TypeScript would also fail if there was a port number.

@iBug any chance that you could share a SAM/CDK/CloudFormation minimal example that we can deploy to reproduce the value in that field?

Answer 6 · 2025-09-01T13:26:34.000Z

@dreamorosi Unfortunately I'm familiar with none of them. I only deployed some personal projects as Lambda functions.

In API Gateway, I added my Lambda functions as integrations with "Payload format version" set to 2.0 (interpreted response format), and this is the JSON my function received (as its first parameter event):

{
  "version": "2.0",
  "routeKey": "ANY /_debug",
  "rawPath": "/_debug",
  "rawQueryString": "",
  "headers": {
    "accept": "*/*",
    "accept-encoding": "gzip, br",
    "cdn-loop": "cloudflare; loops=1",
    "cf-connecting-ip": "2a09:bac5:4303:dc::16:202",
    "cf-ipcountry": "JP",
    "cf-ray": "977a977dc9e57341-NRT",
    "cf-visitor": "{\"scheme\":\"https\"}",
    "content-length": "0",
    "host": "redacted.example.com",
    "user-agent": "curl/8.5.0",
    "x-forwarded-for": "162.158.118.190",
    "x-forwarded-proto": "https"
  },
  "requestContext": {
    "accountId": "<redacted>",
    "apiId": "<redacted>",
    "authentication": {
      "clientCert": {
        "clientCertPem": "-----BEGIN CERTIFICATE-----\nMIIEjTCCAnWgAwIBAgIUXyI9NfFAPqEJ8ww2qW1LBYJO8jAwDQYJKoZIhvcNAQEN\nBQAwgZAxCzAJBgNVBAYTAlVTMRkwFwYDVQQKExBDbG91ZEZsYXJlLCBJbmMuMRQw\nEgYDVQQLEwtPcmlnaW4gUHVsbDEWMBQGA1UEBxMNU2FuIEZyYW5jaXNjbzETMBEG\nA1UECBMKQ2FsaWZvcm5pYTEjMCEGA1UEAxMab3JpZ2luLXB1bGwuY2xvdWRmbGFy\nZS5uZXQwHhcNMjUwODExMTkyNzAwWhcNMjYwODExMTkyNzAwWjCBkDELMAkGA1UE\nBhMCVVMxEzARBgNVBAgTCkNhbGlmb3JuaWExFjAUBgNVBAcTDVNhbiBGcmFuY2lz\nY28xGTAXBgNVBAoTEENsb3VkZmxhcmUsIEluYy4xFDASBgNVBAsTC09yaWdpbiBQ\ndWxsMSMwIQYDVQQDExpvcmlnaW4tcHVsbC5jbG91ZGZsYXJlLm5ldDBZMBMGByqG\nSM49AgEGCCqGSM49AwEHA0IABDOGHqOu6PCiFHp7vljiw9b1y+guV9TRpVoPgViE\nYgqGRFq5mnHZib9BrnqMa5/hcPJ/5/lB5b4pGAZMm6ieL1ujgacwgaQwDgYDVR0P\nAQH/BAQDAgWgMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAMBgNVHRMB\nAf8EAjAAMB0GA1UdDgQWBBRNCt+5U6/oNHalxasFjiiB2xh2ljAfBgNVHSMEGDAW\ngBRDWUsraYuA4REzalfNVzjann3F6zAlBgNVHREEHjAcghpvcmlnaW4tcHVsbC5j\nbG91ZGZsYXJlLm5ldDANBgkqhkiG9w0BAQ0FAAOCAgEAcj8FdJW9S8lamTD29fF3\n8yRe6Q7wxnqi67NKRcwsjYOKuvfvHzBeaj/yvTz6McEhIXB3nT6RMFzAs2hDIxOw\n/bMe4TF59ZAR2IYU3H7D/0LFWZrlBGNNUwcu0HgWa3W9T2+Kb1/oj+S/NtVVNkul\nZsqNIl9s7QRggkG5Zl+Y8YLoP2YgahOkUXjIFhfPUvCG+znXfuAbCPHxd+WaBbh5\nN73zkp0N97oGGIhVTknuMusCyJKGjfyyGt3WNO2htjvjrV+6MHnCuMEGbeK7sB7W\nDS9EbYn6ZSpI6qcsoUCAl8b7J6ZN0Xb5yPFyP260DP8ISM+OHrXmWEg7AVQbLQT4\nX2ySesNMyLveRxuOGITWEdYjYAsypSthJhdNZFG1ElHGPt43a4a8roHa8RTqkEAs\naGpCms3lkrCDoZx6cn0Bj93ZBDoFQkZzUBMiCuvUcDM1liTJImAzWEer8/PFUoIs\np+a2aTzvkw+Ufhq3bKdEwl+0AUjfu1GxLMw4uq1yrRxxZlfeqO63K8Zx1XCHe5+K\nJSoRHfWv0Ek4rDh10ZqEWokfcvpJ16hmiYHpwmjh7XDbMqjz/8rkK8Pl0zK6qElc\netxYSCUD2WU8usTUtEVZlKAr5N4ScorE0OFXt8cU9ER7idP8Lgdy6nNTKE3Iiipn\n/R8QUhoi7rRbvU1ysV2A3zE=\n-----END CERTIFICATE-----\n",
        "issuerDN": "C=US,O=CloudFlare, Inc.,OU=Origin Pull,L=San Francisco,ST=California,CN=origin-pull.cloudflare.net",
        "serialNumber": "543117680770099604425064627706259066651480617520",
        "subjectDN": "C=US,ST=California,L=San Francisco,O=Cloudflare, Inc.,OU=Origin Pull,CN=origin-pull.cloudflare.net",
        "validity": {
          "notAfter": "Aug 11 19:27:00 2026 GMT",
          "notBefore": "Aug 11 19:27:00 2025 GMT"
        }
      }
    },
    "domainName": "redacted.example.com",
    "domainPrefix": "redacted",
    "http": {
      "method": "GET",
      "path": "/_debug",
      "protocol": "HTTP/1.1",
      "sourceIp": "10.1.21.82:7660",
      "userAgent": "curl/8.5.0"
    },
    "requestId": "QKKI4g_5oAMEb0A=",
    "routeKey": "ANY /_debug",
    "stage": "$default",
    "time": "31/Aug/2025:06:50:06 +0000",
    "timeEpoch": 1756623006878
  },
  "isBase64Encoded": false
}

As you can see, requestContext.http.sourceIp is indeed an IP-with-port.

In case it matters, I have my own domain attached to API Gateway, and then put Cloudfront in Front of it. So the whole chain is:

client → Cloudflare → CloudFront → API Gateway → Lambda

Answer 7 · 2025-09-01T14:26:02.000Z

Thank you so much for explaining this @iBug. Cloudfront is acting as a proxy from Cloudflare to APIGW and forwarding all headers because you are probably using an end-to-end TLS connection between Cloudflare and Cloudfront, right?

I can confirm this is something we need to fix, I just don't know if we should add a new type annotation like str to this field or if we should to remove the port with field_validator functions. Let me think a little bit about this, but we will fix this before our next release.

Thanks for your patience and explanation.

Answer 8 · 2025-09-01T14:57:27.000Z

I don't necessarily agree about this being a bug, the model works as intended given that as you both said port numbers are not strictly part of the spec.

If we decide to support it, imo it should be treated as a new use case we support - especially considering that this is happening using a 3P end-to-end TLS connection. With this in mind, I'd be more inclined to consider it an enhancement.

Answer 9 · 2025-09-02T05:21:26.000Z

@leandrodamascena Interesting, you found the point. When I curl the function using alternative domain that is not routed through Cloudflare, then the sourceIp field contains no port. And indeed I have Cloudflare "Authenticated Origin Pull" enabled for the primary domain and uploaded the Origin CA to CloudFront.

To clarify:

Primary domain:

graph LR;
A(["client"]);
B(Cloudflare);
C(CloudFront);
D(API Gateway);
E(Lambda);
A --> B;
B -- "mTLS" --> C;
C --> D;
D --> E;

Result: "sourceIp": "10.1.22.195:32554" (port included)

Secondary domain:

graph LR;
A(["client"]);
C(CloudFront);
D(API Gateway);
E(Lambda);
A --> C;
C --> D;
D --> E;

Result: "sourceIp": "104.28.243.105" (no port)

I'll try later to see if disabling mTLS between Cloudflare and CloudFront makes any difference.

In either case, I don't understand why API Gateway would even produce an RFC 1918 IP address in the first place - that's not supposed to happen - let alone attaching a port number.

Answer 10 · 2025-09-03T09:03:14.000Z

@leandrodamascena Interesting, you found the point. When I curl the function using alternative domain that is not routed through Cloudflare, then the sourceIp field contains no port. And indeed I have Cloudflare "Authenticated Origin Pull" enabled for the primary domain and uploaded the Origin CA to CloudFront.

Yes, that's what I expected in this case. I don't know exactly what Cloudflare's architecture and network are, but I imagine they might have some AWS resources to route it internally, but I don't know, just guessing here.

To clarify:

Primary domain:

Result: "sourceIp": "10.1.22.195:32554" (port included)

Secondary domain:

Result: "sourceIp": "104.28.243.105" (no port)

I'll try later to see if disabling mTLS between Cloudflare and CloudFront makes any difference.

I think this a good test, but we need to cover cases like this.

In either case, I don't understand why API Gateway would even produce an RFC 1918 IP address in the first place - that's not supposed to happen - let alone attaching a port number.

I think it's because it's not routing/producing that IP in fact, it's just forwarding/propagating what it receives from the connection, which I imagine is happening internally using VPC or something.

I'll fix it by using a field_validator to deal with this cases.

Thanks a lot for all the explanations.

Answer 11 · 2025-09-03T09:04:35.000Z

I don't necessarily agree about this being a bug, the model works as intended given that as you both said port numbers are not strictly part of the spec.

If we decide to support it, imo it should be treated as a new use case we support - especially considering that this is happening using a 3P end-to-end TLS connection. With this in mind, I'd be more inclined to consider it an enhancement.

I don't have strong opinions here. As long as we support this use case, we can call it a bug or an enhancement. Please feel free to change the issue title @dreamorosi

Answer 12 · 2025-09-04T13:01:41.000Z

~~@dcabib take a look at this, if interested.~~ sorry, it's already being worked on!

Answer 13 · 2025-09-04T13:27:01.000Z

Warning

This issue is now closed. Please be mindful that future comments are hard for our team to see.
If you need more assistance, please either reopen the issue, or open a new issue referencing this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

Answer 14 · 2025-09-08T16:49:31.000Z

Reopening this because the IPv6 checks.

Answer 15 · 2025-09-09T09:04:37.000Z

Warning

This issue is now closed. Please be mindful that future comments are hard for our team to see.
If you need more assistance, please either reopen the issue, or open a new issue referencing this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

Answer 16 · 2025-09-09T10:26:41.000Z

This is now released under 3.20.0 version!