Error when trying to use private key from environment variable

[meibensteiner]

Im using a self hosted runner on kubernetes and inject the private key into the runner from my secret manager via an environment variable.
When echoing "$github_app_id" and "$github_app_private_key" from a worker everything is fine and properly formatted.
When using those variables in a workflow to get a token like this:

jobs:
  deploy:
    runs-on: [ self-hosted ]
    steps:
     - name: Generate short living token for GitOps update
        uses: actions/create-github-app-token@v1
        id: generate-token
        with:
          owner: meibensteiner
          repositories: test-repo
          app-id: "$github_app_id"
          private-key: "$github_app_private_key"

I encounter the following error:

Failed to create token for "test-repo" (attempt 1): Invalid keyData
Failed to create token for "test-repo" (attempt 2): Invalid keyData
Failed to create token for "test-repo" (attempt 3): Invalid keyData
Failed to create token for "test-repo" (attempt 4): Invalid keyData
DOMException [DataError]: Invalid keyData
    at Object.rsaImportKey (node:internal/crypto/rsa:235:15)
Error: Invalid keyData
    at SubtleCrypto.importKey (node:internal/crypto/webcrypto:615:10)
    ... 6 lines matching cause stack trace ...
    at /runner/_work/_actions/actions/create-github-app-token/v1/dist/main.cjs:39741:71
    at RetryOperation._fn (/runner/_work/_actions/actions/create-github-app-token/v1/dist/main.cjs:39667:30) {
  attemptNumber: 4,
  retriesLeft: 0,
  [cause]: Error: Failed to read private key
      at createPrivateKey (node:internal/crypto/keys:632:12)
      at Object.rsaImportKey (node:internal/crypto/rsa:229:21)
      at SubtleCrypto.importKey (node:internal/crypto/webcrypto:615:10)
      at getToken (/runner/_work/_actions/actions/create-github-app-token/v1/dist/main.cjs:37861:56)
      at githubAppJwt (/runner/_work/_actions/actions/create-github-app-token/v1/dist/main.cjs:37894:23)
      at getAppAuthentication (/runner/_work/_actions/actions/create-github-app-token/v1/dist/main.cjs:39188:37)
      at hook4 (/runner/_work/_actions/actions/create-github-app-token/v1/dist/main.cjs:39472:37)
      at newApi (/runner/_work/_actions/actions/create-github-app-token/v1/dist/main.cjs:37156:36)
      at getTokenFromRepository (/runner/_work/_actions/actions/create-github-app-token/v1/dist/main.cjs:39792:26)
      at /runner/_work/_actions/actions/create-github-app-token/v1/dist/main.cjs:39741:71
}

Is this somehow unsupported? Id prefer to use my own secrets manager instead of the one from github actions.