keycloak/keycloak

Import Realm in Keycloak 18.0.0 - failed

edwint88 opened this issue · 19 comments

edwint88 commented

Describe the bug

I’m using a docker image and I uploaded the realm.json in /opt/keycloak/data/import to get the nice auto import functionality.
When I add now to start --import-realm I get:

2022-04-26 08:28:56,195 ERROR [org.keycloak.quarkus.runtime.cli.ExecutionExceptionHandler] (main) ERROR: Failed to start server in (production) mode
2022-04-26 08:28:56,195 ERROR [org.keycloak.quarkus.runtime.cli.ExecutionExceptionHandler] (main) ERROR: Failed to import realm: myrealm
2022-04-26 08:28:56,195 ERROR [org.keycloak.quarkus.runtime.cli.ExecutionExceptionHandler] (main) ERROR: Script upload is disabled

which confuses me, because

Script Upload should be removed (why is import using that?) and
doesn’t work.
Without the --import-realm all works fine

Reference to documentation: https://www.keycloak.org/server/importExport

Version

18.0.0

Expected behavior

To import the realms automatically from the data/import location.

Actual behavior

Gives an error

How to Reproduce?

  1. create a dockerfile
  2. add a realm.json to the path
  3. build keycloak in docker file
  4. build dockerfile
  5. start container

Anything else?

No response

edwint88 commented

The same error I get also when I try to upload a realm from the interface/admin console:
Uncaught server error: java.lang.RuntimeException: Script upload is disabled

stefanmaric commented

Linking this discussion I opened a few ago: #11658

  • 👍1
pedroigor commented

@edwint88 Is your realm declaring providers that rely on scripts such as mappers, authenticators, or policies?

If so, the realm configuration is now invalid and you should not be able to import. See the note here about the removal of the upload-scripts feature.

  • 👍2
stefanmaric commented

@edwint88 Is your realm declaring providers that rely on scripts such as mappers, authenticators, or policies?

I had the same issue and based on your feedback I search and found I had these policies for one of the realm clients:

"policies": [
  {
    "id": "b56eebd7-8e73-4449-b110-30dfdbc77f03",
    "name": "Default Policy",
    "description": "A policy that grants access only for users within this realm",
    "type": "js",
    "logic": "POSITIVE",
    "decisionStrategy": "AFFIRMATIVE",
    "config": {
      "code": "// by default, grants any permission associated with this policy\n$evaluation.grant();\n"
    }
  },
  { ... },
  {
    "id": "1428ae4c-b767-41b9-aaf9-bd8b0d8497e2",
    "name": "Default Permission",
    "description": "A permission that applies to the default resource type",
    "type": "resource",
    "logic": "POSITIVE",
    "decisionStrategy": "UNANIMOUS",
    "config": {
      "defaultResourceType": "urn:api:resources:default",
      "applyPolicies": "[\"Default Policy\"]"
    }
  },
  {...}
]

I checked and these were not referenced anywhere else so I proceeded to delete them and it worked.

I guess it was created automatically by a previous version of KC.

For anyone else experiencing the same issue: search for "type": "js" to pin-point the root cause in the realm export file.

  • 👍52
  • ❤️26
edwint88 commented

``> @edwint88 Is your realm declaring providers that rely on scripts such as mappers, authenticators, or policies?

If so, the realm configuration is now invalid and you should not be able to import. See the note here about the removal of the upload-scripts feature.

We have an Authenticator, but not a JavaScript one. It's a simple Java SPI that we add through .jar in /opt/keycloak/providers and then, yes, configured in the realm.json.
Is this affected too?

LE: thanks for the "type": "js" hint!
we got a default policy - that we didn't create it, I'll check without that in the config.

{ "id": "cf9f7684-45c2-4bcd-89a5-38c2a41b55fd", "name": "Default Policy", "description": "A policy that grants access only for users within this realm", "type": "js", "logic": "POSITIVE", "decisionStrategy": "AFFIRMATIVE", "config": { "code": "// by default, grants any permission associated with this policy\n$evaluation.grant();\n" } }

That was the problem! So when I've created a dummy confidential client in 17.0.1 the Default JS policies were activated!

  • 👍4
ericwout-overheid commented

We would like to reopen this discussion. We are just starting with a project using Keycloak (18.0.2). We have create a realm via the user interface with some basic settings. We exported the realm using the Manage > Export > Partial Export feature in the user interface.

When we try to import this export using the --import-realm option in a clean container (also 18.0.2), we get the mentioned error. It seems like a bug when Keycloak does not generate a compatible export for the same version?

It works when we remove the policies element enterily (not when we only remove the policy with the "js"-type). But as new users we are not sure whether that's a safe thing to do.

See also:

edwint88 commented
ericwout-overheid commented

Ah, thanks for clearing that up. But shouldn't it be registered as a bug that when you create an export, it cannot be imported without removing this by hand (after googling for it?).

  • 👍7
edwint88 commented

Sure, I don't know if there is a ticket already or to be fixed in the next version.

yangboyd commented

Change the default policy type to the script-jsfilename:

      {
        "id": "834d0278-8eda-4533-911b-78a1d983ffc9",
        "name": "Default Policy",
        "description": "A policy that grants access only for users within this realm",
        "type": "script-default-policy.js",
        "logic": "POSITIVE",
        "decisionStrategy": "AFFIRMATIVE"
      },

put the jar to providers, then
kc.sh build
and
kc.sh start
.
The default-policy.js in one sample jar photoz-uma-js-policies.jar.
photoz-uma-js-policies.jar

  • 👍1
qhaas commented

shouldn't it be registered as a bug that when you create an export, it cannot be imported without removing this by hand

I agree, this issue should be re-opened, this problem still exists in keycloak 20.0 (and 25.0)

The workaround described above seems to still work (using the quay.io/keycloak/keycloak docker image), here is a minimal, reproducible example of the issue and the workaround:

  1. Start an interactive docker container: docker run --name auth1 -e KEYCLOAK_ADMIN=admin -e KEYCLOAK_ADMIN_PASSWORD=12345 -it --entrypoint=/bin/bash -p 8080:8080 quay.io/keycloak/keycloak:20.0
  2. Start the keycloak service: /opt/keycloak/bin/kc.sh start-dev
  3. From the host or another system, login to localhost:8080 after it spins up and create the realm, users, clients, etc. as one sees fit, for this example, the realm will be myrealm
  4. Stop the keycloak service (e.g. CTRL-C)
  5. Export the realm (for KC25+, remove --users same_file): /opt/keycloak/bin/kc.sh export --realm myrealm --users same_file --file /tmp/realm-export.json.tmp
  6. Exit the docker instance and let it stop exit
  7. Copy the exported realm out of the docker instance to a temporary file: docker cp auth1:/tmp/realm-export.json.tmp realm-export.json.tmp
  8. Copy said temporary file to an import folder: cp realm-export.json.tmp import/realm-export.json
  9. Verify import bug still exists by attempting to import said realm upon container instance spinup:
$ docker run --name auth2 -e KEYCLOAK_ADMIN=admin -e KEYCLOAK_ADMIN_PASSWORD=12345 -v $(pwd)/import/:/opt/keycloak/data/import/:ro --rm quay.io/keycloak/keycloak:20.0 start-dev --import-realm
...
Unable to import realm myrealm from file /opt/keycloak/bin/../data/import/realm-export.json.: java.lang.RuntimeException: Script upload is disabled
  1. Stop the service with CTRL-C and let the docker instance exit and remove itself
  2. Remove the policies using jq to avoid editing by hand: cat realm-export.json.tmp | jq 'del(.clients[].authorizationSettings.policies)' > import/realm-export.json
  3. Verify import now works: docker run --name auth2 -e KEYCLOAK_ADMIN=admin -e KEYCLOAK_ADMIN_PASSWORD=12345 -v $(pwd)/import/:/opt/keycloak/data/import/:ro --rm -p 8080:8080 quay.io/keycloak/keycloak:20.0 start-dev --import-realm

Updated for KC25

  • 👍4
herbertscruz commented

Replacing the Default Policy setting with the one below resolves the issue and maintains the expected behavior.

          {
            "id": "98bb844e-5524-4c42-914b-cc5121d3124d",
            "name": "Default Policy",
            "description": "A policy that grants access only for users within this realm",
            "type": "role",
            "logic": "POSITIVE",
            "decisionStrategy": "AFFIRMATIVE",
            "config": {
              "roles": "[{\"id\":\"default-roles-main\",\"required\":false}]"
            }
          }
  • 👍14
ajaiswal595 commented

script-default-policy.js

@yangboyd this works

put the jar to providers, then
kc.sh build
and
kc.sh start
.
The default-policy.js in one sample jar photoz-uma-js-policies.jar.
photoz-uma-js-policies.jar

But the problem is with upgrade from wildfly(15) distro to keycloak(20) , as in database have of type js entry for default policy.
image

And we can not ask end user/admin to change the type of Policy type.

What is the recommendation in the scenarios of migration?

pedroigor commented

@edwint88 @yangboyd I'm working on it. Can we discuss this issue at #16074?

  • 👍1
hantsy commented

I encountered the same issue in the newest 25.0.1.

  • 👍1
hantsy commented

@qhaas 's solution resolved my problem.

goekhantopcu commented

The solution of @qhaas worked for me as well, thanks a lot men <3 (using keycloak 24.0.4)

qhaas commented

IMO, given this is still an issue with 25.0.x, likely deserves a new issue.

I'd be interested in someone with more keycloak expertise than I to explain the consequences of stefanmaric's workaround vs herbertscruz's workaround given defaults / behaviors have likely changed since they were first proposed as a workaround.