Feature request: add bootstrapping mechanism that can be automated instead of having to fetch generated password from the logs
Closed this issue · 27 comments
It would be great if rauthy bootstrap could be automated with software like Terraform/OpenTofu, e.g. by passing the initial admin credentials (email, password, possibly an admin API token) as environment variables the way other software does for bootstrap.
That way installing and then configuring rauthy via something like a Terraform provider could be fully automated without any manual steps.
I thought about adding something like this as well, yes.
Edit:
How I did such things until now is by prodiving an existing sqlite db file and using MIGRATE_DB_FROM. This works fine for CI and testing.
Apart from testing, where this is fine, why would you automate deployments? You usually set this up once for your environment and that's it.
It might be a bit tricky to generate an sqlite DB file with a rauthy specific schema as part of the deployment process.
It might be a bit tricky to generate an sqlite DB file with a rauthy specific schema as part of the deployment process.
No actually really easy.
I set up my test instance via the UI, just like I need it, and then only provide this single tiny file for my pipelines. Or you could just apply any sqlite backup from S3 storage inside your pipelines which would not even require you to mount a volume.
Preparing the file manually might be fine for testing but for something like an instance accessible from the public internet it would be useful if each deployed instance had a distinct, auto-generated password (and API token if that is applicable).
...it would be useful if each deployed instance had a distinct, auto-generated password...
This is the case. That's why you see the password in the logs at the very first startup.
The only use case I see for something like this is when providing an OIDC SaaS.
Where else would you need all of this to be fully automatic and externally handled?
Such a feature will probably come when I start developing the K8s operator at some point.
Not really as SaaS. I am currently experimenting with Kubernetes clusters after years of using Puppet for Infrastructure as Code and I am trying to get clusters set up in a fully (apart from specifying some access tokens and names as parameters) automated and repeated way and want to use some OIDC provider in the cluster to protect observability tools like Grafana and Prometheus, ideally something with low resource usage so the VPS used as basis for the Kubernetes cluster do not need to be too large just for that kind of basic cluster infrastructure that every cluster might need.
Ahh I see, okay... yes in that case it makes sense.
I can add something to achieve this before starting with the k8s operator.
Done.
I added a few new config vars that can be either set in the rauthy.cfg, .env or simply be env vars:
#####################################
############# BOOSTRAP ##############
#####################################
# If set, the email of the default admin will be changed
# during the initialization of an empty production database.
BOOTSTRAP_ADMIN_EMAIL="alfred@batcave.io"
# If set, this plain text password will be used for the
# initial admin password instead of generating a random
# password.
#BOOTSTRAP_ADMIN_PASSWORD_PLAIN="123SuperSafe"
# If set, this will take the argon2id hashed password
# during the initialization of an empty production database.
# If both BOOTSTRAP_ADMIN_PASSWORD_PLAIN and
# BOOTSTRAP_ADMIN_PASSWORD_ARGON2ID are set, the hashed version
# will always be prioritized.
BOOTSTRAP_ADMIN_PASSWORD_ARGON2ID='$argon2id$v=19$m=32768,t=3,p=2$mK+3taI5mnA+Gx8OjjKn5Q$XsOmyvt9fr0V7Dghhv3D0aTe/FjF36BfNS5QlxOPep0'
During prod db init, you will not see the given password, only the information that a given bootstrap password has been chosen:
And when you log in to the UI, you can do so with the given email and password, so you will end up with only that user as the only admin:
You can provide an argon2id hashed password as well, if you want to check in some stuff into git.
The only thing you need to be careful about is to set it in single '' because of the $.
Providing an API Key via bootstrap would require a not that small JSON to be serialized into b64, so it can be parsed correctly, and so on. There are quite a few footguns with these and it would probably be way faster to do this via the UI later.
Did you have something like this in mind? I can push a new nightly image for testing, if you like.
You can test with
ghcr.io/sebadob/rauthy:0.22.2-20240424-2-lite
if you like.
I will need to test it tomorrow but it sounds like exactly the kind of thing I had in mind. Thanks for the quick response.
I did test this today and at least the _PLAIN variant seems to work as expected, I haven't been able to find a way to generate the password hash yet so I didn't test the other one.
Now I just need to figure out a way to bootstrap my way from there to an API key automatically to configure the new instance.
Now I just need to figure out a way to bootstrap my way from there to an API key automatically to configure the new instance.
I could implement another env var which does this. But then you would have to carefully build the JSON object and serialize it into base64, if this would be an option for you. On my side this is not too much work. I just figured that people would not want to do that. :D
I would not mind carefully building a JSON object. Would that JSON object include just the permissions or would the actual secret key part of the API key be part of that as well? It might make sense to split them up into two environment variables if that JSON does not already exist on your end since for any given bootstrap you probably want a fixed set of permissions (fixed permission JSON object) but a different secret for each deploy?
I think they even must be separated, because they are handled differently inside.
I need to take a quick look at the code am will get back at you.
Edit:
All structs do already exist and I think I could re-use the JSON that would come via API otherwise.
At least for my Terraform use case building the JSON shouldn't be too hard since it has a jsonencode function even if they aren't separated but it might be cleaner for tools that do not have an equivalent functionality but just basic templating to build the environment variable values (e.g. Helm).
Yes I mean, you could even do this manually. Because I guess you would the same access rights each time anyway, like you mentioned.
So, I could either accept it as bas64 which would make setting the env var easier, or accept a json directly, but then you have to carefully quote it and make sure there are no shell escapes or anything like that.
The Json I would try do deserialize into would be:
#[derive(Debug, Serialize, Deserialize, Validate, ToSchema)]
pub struct ApiKeyRequest {
/// Validation: `^[a-zA-Z0-9_-/]{2,24}$`
#[validate(regex(path = "RE_API_KEY", code = "^[a-zA-Z0-9_-/]{2,24}$"))]
pub name: String,
/// Unix timestamp in seconds in the future (max year 2099)
#[validate(range(min = 1672527600, max = 4070905200))]
pub exp: Option<i64>,
pub access: Vec<ApiKeyAccess>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, ToSchema)]
pub struct ApiKeyAccess {
pub group: AccessGroup,
pub access_rights: Vec<AccessRights>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, ToSchema)]
pub enum AccessGroup {
Blacklist,
Clients,
Events,
Generic,
Groups,
Roles,
Secrets,
Sessions,
Scopes,
UserAttributes,
Users,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, ToSchema)]
#[serde(rename_all = "lowercase")]
pub enum AccessRights {
Read,
Create,
Update,
Delete,
}While the secret would need to be 64 random alnum characters.
Giving the secret can be done as well, but you need to append the key id then manually. It is not just simply the secret, but it will contain the name as well in the format
let secret_fmt = format!("{}${}", name, secret_plain);I think setting it as base64 encoded JSON would make things easier as you say. Quoting can be a bit of a nightmare when you have to use different quoting systems layered on top of each other (e.g. one in your templating, one in your config file format, one in shell code,...).
What would the name be in that case? And is there a required format for the secret?
New config vars:
# You can provide an API Key during the initial prod database
# bootstrap. This key must match the format and pass validation.
# You need to provide it as a base64 encoded JSON in the format:
#
# ```
# struct ApiKeyRequest {
# /// Validation: `^[a-zA-Z0-9_-/]{2,24}$`
# name: String,
# /// Unix timestamp in seconds in the future (max year 2099)
# exp: Option<i64>,
# access: Vec<ApiKeyAccess>,
# }
#
# struct ApiKeyAccess {
# group: AccessGroup,
# access_rights: Vec<AccessRights>,
# }
#
# enum AccessGroup {
# Blacklist,
# Clients,
# Events,
# Generic,
# Groups,
# Roles,
# Secrets,
# Sessions,
# Scopes,
# UserAttributes,
# Users,
# }
#
# #[serde(rename_all = "lowercase")]
# enum AccessRights {
# Read,
# Create,
# Update,
# Delete,
# }
# ```
#
# You can use the `api_key_example.json` from `/` as
# an example. Afterwards, just `base64 api_key_example.json | tr -d '\n'`
BOOTSTRAP_API_KEY="ewogICJuYW1lIjogImJvb3RzdHJhcCIsCiAgImV4cCI6IDE3MzU1OTk2MDAsCiAgImFjY2VzcyI6IFsKICAgIHsKICAgICAgImdyb3VwIjogIkNsaWVudHMiLAogICAgICAiYWNjZXNzX3JpZ2h0cyI6IFsKICAgICAgICAicmVhZCIsCiAgICAgICAgImNyZWF0ZSIsCiAgICAgICAgInVwZGF0ZSIsCiAgICAgICAgImRlbGV0ZSIKICAgICAgXQogICAgfSwKICAgIHsKICAgICAgImdyb3VwIjogIlJvbGVzIiwKICAgICAgImFjY2Vzc19yaWdodHMiOiBbCiAgICAgICAgInJlYWQiLAogICAgICAgICJjcmVhdGUiLAogICAgICAgICJ1cGRhdGUiLAogICAgICAgICJkZWxldGUiCiAgICAgIF0KICAgIH0sCiAgICB7CiAgICAgICJncm91cCI6ICJHcm91cHMiLAogICAgICAiYWNjZXNzX3JpZ2h0cyI6IFsKICAgICAgICAicmVhZCIsCiAgICAgICAgImNyZWF0ZSIsCiAgICAgICAgInVwZGF0ZSIsCiAgICAgICAgImRlbGV0ZSIKICAgICAgXQogICAgfQogIF0KfQ=="
# The secret for the above defined bootstrap API Key.
# This must be at least 64 alphanumeric characters long.
# You will be able to use that key afterwards with setting
# the `Authorization` header:
#
# `Authorization: API-Key <your_key_name_from_above>$<this_secret>`
BOOTSTRAP_API_KEY_SECRET=twUA2M7RZ8H3FyJHbti2AcMADPDCxDqUKbvi8FDnm3nYidwQx57Wfv6iaVTQynMh
The api_key_example.json:
{
"name": "bootstrap",
"exp": 1735599600,
"access": [
{
"group": "Clients",
"access_rights": [
"read",
"create",
"update",
"delete"
]
},
{
"group": "Roles",
"access_rights": [
"read",
"create",
"update",
"delete"
]
},
{
"group": "Groups",
"access_rights": [
"read",
"create",
"update",
"delete"
]
}
]
}When bootstrapped:
I guess that solves your problems. :)
I will test it tomorrow but it looks like what I need. Did you create another new image for testing or reuse the existing name you gave in the comment above?
Thank you for all the work you put into my use case.
I have some resources free on a host, so I can build a new nightly image. It will just take ~1 hour for all images and tests to run through :)
This should work out for you:
ghcr.io/sebadob/rauthy:0.22.2-20240424-3-lite
Thank you, I will give it a try tomorrow and let you know how it goes.
I got it working now up to the point where I tested the API-Key with a curl command similar to yours above. Everything worked perfectly. Thank you again very much.
Thanks for the feedback