jwt.exceptions.InvalidKeyError: Could not parse the provided public key.

Question

jwt.exceptions.InvalidKeyError: Could not parse the provided public key.

Closed this issue a month ago · 3 comments

Hello DefectDojo team,

This is my first time reaching out to the support team. I wanted to start by mentioning that I'm not entirely sure if what I'm experiencing is an actual bug or if it might be due to a misconfiguration on my part.

Be informative

I'm trying to integrate DefectDojo's authentication with Keycloak following the official documentation at https://defectdojo.github.io/django-DefectDojo/integrations/social-authentication/#keycloak.

Bug description

When testing the integration, I'm receiving the following error in the browser:

Well...
...this was unexpected.
500 Internal Server Error

I have defined the environment variable DD_SOCIAL_AUTH_KEYCLOAK_PUBLIC_KEY using the step 6 of the documentation, but Keycloak don't add the header and footer.

DD_SOCIAL_AUTH_KEYCLOAK_PUBLIC_KEY=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAs0YjRFE5Rr31dKuMjwu7BSrk2o2QN8Yp9xLH9QHT23HyymwN5fcmTkSvCpcrJu3/GuBqP7LOcacX2eeBnA2fUmzR7lPOLJ1eGgnXntju00K2VXHeyC0K5DZistMvGvHJyJQpnXRxutbsOeW8tpKc0GPUF8xvucJevlU5YwNnEEn1+8SRgqA553GUaiX397QXWUyVOlK/rQB/UmoyQyMJZJJyELOO/Tzj1vRXBgfADD9ZWPhXHtUaO+JsjYn0xV4qZTh9Ax1KY9bdC8hRRs0dFziTRgBOovH555Yue1q4q63XqvI+DBizG/TmMsU6pegs74JP74u5KB6/WMYChxZMgwIDAQAB

Analysing the logs and reviewing the expected public key format in the cryptography documentation (https://cryptography.io/en/latest/faq/#why-can-t-i-import-my-pem-file), I've adjusted my Terraform configuration (redacted to contain a minimum of information):

terraform {
  required_providers {
    keycloak = {
      source  = "mrparkers/keycloak"
      version = "~> 4.4"
    }
  }
}

locals {
  raw_lines = trimspace(data.keycloak_realm_keys.realm_keys.keys[0].public_key)
  formatted_lines = regexall(".{1,64}", local.raw_lines)
  formatted_public_key = trimspace(join("\n", concat(
    ["-----BEGIN PUBLIC KEY-----"],
    local.formatted_lines,
    ["-----END PUBLIC KEY-----"]
  )))
}

data "keycloak_realm_keys" "realm_keys" {
  realm_id   = data.keycloak_realm.realm.id
  algorithms = ["RS256"]
  status     = ["ACTIVE"]
}

resource "kubectl_manifest" "defectdojo_configmap" {
  yaml_body = <<YAML
apiVersion: v1
kind: ConfigMap
metadata:
  name: defectdojo-configmap
  namespace: defectdojo
data:
  ...
  DD_SESSION_COOKIE_SECURE: "true"
  DD_CSRF_COOKIE_SECURE: "true"
  DD_SECURE_SSL_REDIRECT: "true"
  DD_SOCIAL_AUTH_KEYCLOAK_OAUTH2_ENABLED: "true"
  DD_SOCIAL_AUTH_KEYCLOAK_KEY: "${keycloak_openid_client.defectdojo_client.client_id}"
  DD_SOCIAL_AUTH_KEYCLOAK_PUBLIC_KEY: |-
    ${replace(local.formatted_public_key, "\n", "\n    ")}
  ...
YAML
}

The result of this configuration is

defectdojo-django-6b9547b696-tx4mk:/app$ printenv
...
DD_UWSGI_HOST=localhost
DD_SOCIAL_AUTH_KEYCLOAK_PUBLIC_KEY=-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAs0YjRFE5Rr31dKuMjwu7
BSrk2o2QN8Yp9xLH9QHT23HyymwN5fcmTkSvCpcrJu3/GuBqP7LOcacX2eeBnA2f
UmzR7lPOLJ1eGgnXntju00K2VXHeyC0K5DZistMvGvHJyJQpnXRxutbsOeW8tpKc
0GPUF8xvucJevlU5YwNnEEn1+8SRgqA553GUaiX397QXWUyVOlK/rQB/UmoyQyMJ
ZJJyELOO/Tzj1vRXBgfADD9ZWPhXHtUaO+JsjYn0xV4qZTh9Ax1KY9bdC8hRRs0d
FziTRgBOovH555Yue1q4q63XqvI+DBizG/TmMsU6pegs74JP74u5KB6/WMYChxZM
gwIDAQAB
-----END PUBLIC KEY-----
DD_SOCIAL_AUTH_KEYCLOAK_OAUTH2_ENABLED=true
...

But the issue persists. Trying to figure out if the public key is valid, I saved the environment variable to a file and execute the existent openssl command in the pod:

defectdojo-django-6b9547b696-tx4mk:/app$ echo -e "$DD_SOCIAL_AUTH_KEYCLOAK_PUBLIC_KEY" > /tmp/pub_key.pem
defectdojo-django-6b9547b696-tx4mk:/app$ openssl rsa -text -pubin -in /tmp/pub_key.pem
Public-Key: (2048 bit)
Modulus:
    00:b3:46:23:44:51:39:46:bd:f5:74:ab:8c:8f:0b:
    bb:05:2a:e4:da:8d:90:37:c6:29:f7:12:c7:f5:01:
    d3:db:71:f2:ca:6c:0d:e5:f7:26:4e:44:af:0a:97:
    2b:26:ed:ff:1a:e0:6a:3f:b2:ce:71:a7:17:d9:e7:
    81:9c:0d:9f:52:6c:d1:ee:53:ce:2c:9d:5e:1a:09:
    d7:9e:d8:ee:d3:42:b6:55:71:de:c8:2d:0a:e4:36:
    62:b2:d3:2f:1a:f1:c9:c8:94:29:9d:74:71:ba:d6:
    ec:39:e5:bc:b6:92:9c:d0:63:d4:17:cc:6f:b9:c2:
    5e:be:55:39:63:03:67:10:49:f5:fb:c4:91:82:a0:
    39:e7:71:94:6a:25:f7:f7:b4:17:59:4c:95:3a:52:
    bf:ad:00:7f:52:6a:32:43:23:09:64:92:72:10:b3:
    8e:fd:3c:e3:d6:f4:57:06:07:c0:0c:3f:59:58:f8:
    57:1e:d5:1a:3b:e2:6c:8d:89:f4:c5:5e:2a:65:38:
    7d:03:1d:4a:63:d6:dd:0b:c8:51:46:cd:1d:17:38:
    93:46:00:4e:a2:f1:f9:e7:96:2e:7b:5a:b8:ab:ad:
    d7:aa:f2:3e:0c:18:b3:1b:f4:e6:32:c5:3a:a5:e8:
    2c:ef:82:4f:ef:8b:b9:28:1e:bf:58:c6:02:87:16:
    4c:83
Exponent: 65537 (0x10001)
writing RSA key
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAs0YjRFE5Rr31dKuMjwu7
BSrk2o2QN8Yp9xLH9QHT23HyymwN5fcmTkSvCpcrJu3/GuBqP7LOcacX2eeBnA2f
UmzR7lPOLJ1eGgnXntju00K2VXHeyC0K5DZistMvGvHJyJQpnXRxutbsOeW8tpKc
0GPUF8xvucJevlU5YwNnEEn1+8SRgqA553GUaiX397QXWUyVOlK/rQB/UmoyQyMJ
ZJJyELOO/Tzj1vRXBgfADD9ZWPhXHtUaO+JsjYn0xV4qZTh9Ax1KY9bdC8hRRs0d
FziTRgBOovH555Yue1q4q63XqvI+DBizG/TmMsU6pegs74JP74u5KB6/WMYChxZM
gwIDAQAB
-----END PUBLIC KEY-----
defectdojo-django-6b9547b696-tx4mk:/app$

Knowing that the public key is valid, I thought about validating the python library's ability to handle this information (inside the pod). So I wrote little python script:

import os
from cryptography.hazmat.primitives import serialization
import jwt.algorithms

def analyze_public_key():
    # Get environment variable
    key_content = os.getenv('DD_SOCIAL_AUTH_KEYCLOAK_PUBLIC_KEY')
    
    if not key_content:
        print("❌ Environment variable DD_SOCIAL_AUTH_KEYCLOAK_PUBLIC_KEY not found")
        return
    
    print("=== Keycloak Public Key Analysis ===\n")
    print(f"Content length: {len(key_content)} characters\n")
    
    # Try to identify the format
    key_format = "Unknown"
    if key_content.startswith('-----BEGIN'):
        key_format = "PEM"
    elif key_content.startswith('MII'):
        key_format = "Base64 (possible DER)"
    
    print(f"Detected format: {key_format}\n")
    print("Attempting to parse using different methods...")
    
    # Try to load as public key
    try:
        key_bytes = key_content.encode()
        public_key = serialization.load_pem_public_key(key_bytes)
        print("✓ Successfully loaded as RSA/EC public key")
        
        # Get key details
        key_size = getattr(public_key, 'key_size', None)
        if key_size:
            print(f"Key size: {key_size} bits")
            
        key_type = public_key.__class__.__name__
        print(f"Key type: {key_type}")
        
        # Try to export the key in different formats
        try:
            pem = public_key.public_bytes(
                encoding=serialization.Encoding.PEM,
                format=serialization.PublicFormat.SubjectPublicKeyInfo
            )
            print("✓ Key can be exported as PEM")
        except Exception as e:
            print(f"❌ Error exporting as PEM: {str(e)}")
            
    except Exception as e:
        print(f"❌ Error loading as public key: {str(e)}")
    
    # Try to use with PyJWT
    try:
        rsa = jwt.algorithms.RSAAlgorithm(jwt.algorithms.RSAAlgorithm.SHA256)
        rsa.prepare_key(key_content)
        print("✓ Key is accepted by PyJWT")
    except Exception as e:
        print(f"❌ Error validating with PyJWT: {str(e)}")
    
    # Print raw content for inspection
    print("\nRaw key content:")
    print("-" * 50)
    print(key_content)
    print("-" * 50)

if __name__ == "__main__":
    analyze_public_key()

save it to /tmp/check.py and run it using python3 /tmp/check.py the result was:

=== Keycloak Public Key Analysis ===

Content length: 450 characters

Detected format: PEM

Attempting to parse using different methods...
✓ Successfully loaded as RSA/EC public key
Key size: 2048 bits
Key type: RSAPublicKey
✓ Key can be exported as PEM
✓ Key is accepted by PyJWT

Raw key content:
--------------------------------------------------
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAs0YjRFE5Rr31dKuMjwu7
BSrk2o2QN8Yp9xLH9QHT23HyymwN5fcmTkSvCpcrJu3/GuBqP7LOcacX2eeBnA2f
UmzR7lPOLJ1eGgnXntju00K2VXHeyC0K5DZistMvGvHJyJQpnXRxutbsOeW8tpKc
0GPUF8xvucJevlU5YwNnEEn1+8SRgqA553GUaiX397QXWUyVOlK/rQB/UmoyQyMJ
ZJJyELOO/Tzj1vRXBgfADD9ZWPhXHtUaO+JsjYn0xV4qZTh9Ax1KY9bdC8hRRs0d
FziTRgBOovH555Yue1q4q63XqvI+DBizG/TmMsU6pegs74JP74u5KB6/WMYChxZM
gwIDAQAB
-----END PUBLIC KEY-----
--------------------------------------------------

and it works too!

Steps to reproduce

Just follow the the official documentation at https://defectdojo.github.io/django-DefectDojo/integrations/social-authentication/#keycloak and take care about the step 6, because Keycloak don't add the header of footer.

Expected behavior

The DefectDojo can load the public key and finish the process of login with Keycloak.

Deployment method (select with an X)

Docker Compose
Kubernetes
GoDojo

Environment information

DefectDojo version: defectdojo/defectdojo-django:2.39.1-alpine
Infrastructure: AWS EKS 1.31
RDS Aurora Postgres: 15.4
Elastic Cache Redis: 7.1
Keycloak: 23.0.6

Logs

ERROR [django.request:241] Internal Server Error: /complete/keycloak/
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/jwt/algorithms.py", line 343, in prepare_key
    RSAPrivateKey, load_pem_private_key(key_bytes, password=None)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: ('Could not deserialize key data. The data may be in an incorrect format, the provided password may be incorrect, it may be encrypted with an unsupported algorithm, or it may be an unsupported key type (e.g. EC curves with explicit parameters).', [<OpenSSLError(code=503841036, lib=60, reason=524556, reason_text=unsupported)>])

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/jwt/algorithms.py", line 347, in prepare_key
    return cast(RSAPublicKey, load_pem_public_key(key_bytes))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: Unable to load PEM file. See https://cryptography.io/en/latest/faq/#why-can-t-i-import-my-pem-file for more details. InvalidData(InvalidByte(0, 45))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/views/decorators/cache.py", line 80, in _view_wrapper
    response = view_func(request, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/views/decorators/csrf.py", line 65, in _view_wrapper
    return view_func(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/social_django/utils.py", line 49, in wrapper
    return func(request, backend, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/social_django/views.py", line 31, in complete
    return do_complete(
           ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/social_core/actions.py", line 49, in do_complete
    user = backend.complete(user=user, redirect_name=redirect_name, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/social_core/backends/base.py", line 39, in complete
    return self.auth_complete(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/social_core/utils.py", line 253, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/social_core/backends/oauth.py", line 427, in auth_complete
    return self.do_auth(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/social_core/utils.py", line 253, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/social_core/backends/oauth.py", line 434, in do_auth
    data = self.user_data(access_token, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/social_core/backends/keycloak.py", line 134, in user_data
    return jwt.decode(
           ^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jwt/api_jwt.py", line 211, in decode
    decoded = self.decode_complete(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jwt/api_jwt.py", line 152, in decode_complete
    decoded = api_jws.decode_complete(
              ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jwt/api_jws.py", line 210, in decode_complete
    self._verify_signature(signing_input, header, signature, key, algorithms)
  File "/usr/local/lib/python3.11/site-packages/jwt/api_jws.py", line 314, in _verify_signature
    prepared_key = alg_obj.prepare_key(key)
                   ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jwt/algorithms.py", line 349, in prepare_key
    raise InvalidKeyError("Could not parse the provided public key.")
jwt.exceptions.InvalidKeyError: Could not parse the provided public key.

Sample scan files

NA

Screenshots

Answer 1 · 2024-10-26T23:52:59.000Z

@eduardovalentim I've never hooked up DefectDojo to Keycloak so don't have any advice to give beyond suggesting you ask this on the OWASP Slack instance as there's usually more traffic/conversations happening there so you're more likely to get an answer.

You can find info on getting to the OWASP Slack #defectdojo channel at https://github.com/DefectDojo/django-DefectDojo?tab=readme-ov-file#community-getting-involved-and-updates

Answer 2 · 2024-10-28T11:01:35.000Z

I checked lib documentation and it looks that social_core.backends.keycloak.KeycloakOAuth2 expect PubKey in the format without -----BEGIN/END PUBLIC KEY----- prefix/suffix. You need to start with MIIBIjANBx...
https://python-social-auth.readthedocs.io/en/latest/backends/keycloak.html

Answer 3 · 2024-11-08T19:34:29.000Z

Hey guys!

I apologize for the delay in testing the information shared. @kiblik, thank you so much for your help - your suggestion was spot-on and worked on the first try! @mtesauro, I appreciate you pointing me to the OWASP Slack channel.

The issue wasn't actually a bug, but rather my misinterpretation of how to properly configure the public key. As indicated in the documentation, the KeycloakOAuth2 backend expects the public key without the "-----BEGIN/END PUBLIC KEY-----" prefix/suffix, starting directly with "MIIBIjANBx...". Once I made this adjustment, everything worked perfectly.

Ironically, this was actually the first solution I tried, but for some reason (probably due to another unrelated issue), it didn't work at that time. That initial failure sent me down a rabbit hole of trial and error, leading me further away from the correct solution.

For anyone who might need it in the future, here's my working Terraform configuration using the mrparkers/keycloak provider:

provider "keycloak" {
    client_id     = var.kc_client_id
    client_secret = var.kc_client_secret
    url           = "https://${var.kc_hostname_host}"
}

data "keycloak_realm" "realm" {
    realm = var.kc_realm_name
}

data "keycloak_realm_keys" "realm_keys" {
  realm_id   = data.keycloak_realm.realm.id
  algorithms = ["RS256"]
  status     = ["ACTIVE"]
}

resource "keycloak_openid_client" "defectdojo_client" {
  realm_id              = data.keycloak_realm.realm.id
  enabled               = true
  client_id             = "defectdojo"
  name                  = "DefectDojo"
  access_type           = "CONFIDENTIAL"
  standard_flow_enabled = true
  full_scope_allowed    = false

  root_url    = "https://${var.defectdojo_host}"
  base_url    = "https://${var.defectdojo_host}"
  web_origins = ["+"]

  valid_redirect_uris = [
    "https://${var.defectdojo_host}/*"
  ]

  extra_config = {
    "user.info.response.signature.alg"    = "RS256"
    "request.object.signature.alg"        = "RS256"
  }
}

resource "keycloak_openid_audience_protocol_mapper" "aud_mapper" {
  realm_id                 = data.keycloak_realm.realm.id
  client_id                = keycloak_openid_client.defectdojo_client.id
  included_client_audience = keycloak_openid_client.defectdojo_client.client_id
  name                     = "aud"

  # Configurações adicionais
  add_to_id_token     = false
  add_to_access_token = true
}

resource "kubectl_manifest" "defectdojo_configmap" {
  yaml_body = <<YAML
apiVersion: v1
kind: ConfigMap
metadata:
  name: defectdojo-configmap
  namespace: defectdojo
data:
  ...
  DD_SESSION_COOKIE_SECURE: "true"
  DD_CSRF_COOKIE_SECURE: "true"
  DD_SECURE_SSL_REDIRECT: "true"
  DD_SOCIAL_AUTH_KEYCLOAK_OAUTH2_ENABLED: "true"
  DD_SOCIAL_AUTH_KEYCLOAK_KEY: "${keycloak_openid_client.defectdojo_client.client_id}"
  DD_SOCIAL_AUTH_KEYCLOAK_PUBLIC_KEY: "${data.keycloak_realm_keys.realm_keys.keys.0.public_key}"
  DD_SOCIAL_AUTH_KEYCLOAK_AUTHORIZATION_URL: "https://${var.kc_hostname_host}/realms/${var.kc_realm_name}/protocol/openid-connect/auth"
  DD_SOCIAL_AUTH_KEYCLOAK_ACCESS_TOKEN_URL: "https://${var.kc_hostname_host}/realms/${var.kc_realm_name}/protocol/openid-connect/token"
  DD_SOCIAL_AUTH_KEYCLOAK_USER_INFO_URL: "https://${var.kc_hostname_host}/realms/${var.kc_realm_name}/protocol/openid-connect/userinfo"
  DD_SOCIAL_AUTH_KEYCLOAK_LOGOUT_URL: "https://${var.kc_hostname_host}/realms/${var.kc_realm_name}/protocol/openid-connect/logout"
  DD_SOCIAL_AUTH_KEYCLOAK_LOGIN_BUTTON_TEXT: "Login with Microsoft"
YAML

}

resource "kubectl_manifest" "defectdojo_secret" {
  yaml_body = <<-YAML
    apiVersion: v1
    kind: Secret
    metadata:
      name: defectdojo-secret
      namespace: defectdojo
    type: Opaque
    data:
      ...
      DD_SOCIAL_AUTH_KEYCLOAK_SECRET: "${base64encode(keycloak_openid_client.defectdojo_client.client_secret)}"
  YAML
}