Facing permission denied
Opened this issue · 3 comments
cloudcafetech commented
First of all, Thanks for nice project.
My vault running with backend consul with HA and I expose Vault using ingress. export VAULT_ADDR=http://vault-internal.172.31.14.138.nip.io
- Error
2021-07-30 02:35:37.711 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Refresh of secret internal-new-config in namespace kube-vault failed with exception
de.koudingspawn.vault.vault.communication.SecretNotAccessibleException: Couldn't load secret from vault path internal/new/config
at de.koudingspawn.vault.vault.VaultCommunication.getVersionedSecret(VaultCommunication.java:136) ~[classes!/:0.0.1-SNAPSHOT]
at de.koudingspawn.vault.vault.VaultCommunication.getVersionedSecret(VaultCommunication.java:111) ~[classes!/:0.0.1-SNAPSHOT]
at de.koudingspawn.vault.vault.impl.KeyValueV2Generator.getHash(KeyValueV2Generator.java:34) ~[classes!/:0.0.1-SNAPSHOT]
at de.koudingspawn.vault.kubernetes.scheduler.impl.KeyValueV2Refresh.hashHasChanged(KeyValueV2Refresh.java:33) ~[classes!/:0.0.1-SNAPSHOT]
at de.koudingspawn.vault.kubernetes.scheduler.impl.KeyValueV2Refresh.refreshIsNeeded(KeyValueV2Refresh.java:28) ~[classes!/:0.0.1-SNAPSHOT]
at de.koudingspawn.vault.kubernetes.scheduler.ScheduledRefresh.refreshCertificates(ScheduledRefresh.java:43) ~[classes!/:0.0.1-SNAPSHOT]
at sun.reflect.GeneratedMethodAccessor47.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_275]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_275]
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) [spring-context-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_275]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_275]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_275]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_275]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_275]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_275]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_275]
Caused by: org.springframework.vault.VaultException: Status 403 Forbidden [new/config]: 1 error occurred:
* permission denied
; nested exception is org.springframework.web.client.HttpClientErrorException$Forbidden: 403 Forbidden: [{"errors":["1 error occurred:\n\t* permission denied\n\n"]}
]
at org.springframework.vault.client.VaultResponses.buildException(VaultResponses.java:86) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at org.springframework.vault.core.VaultVersionedKeyValueTemplate.lambda$doRead$0(VaultVersionedKeyValueTemplate.java:118) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at org.springframework.vault.core.VaultTemplate.doWithSession(VaultTemplate.java:466) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at org.springframework.vault.core.VaultVersionedKeyValueTemplate.doRead(VaultVersionedKeyValueTemplate.java:100) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at org.springframework.vault.core.VaultVersionedKeyValueTemplate.get(VaultVersionedKeyValueTemplate.java:89) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at de.koudingspawn.vault.vault.VaultCommunication.getVersionedSecret(VaultCommunication.java:123) ~[classes!/:0.0.1-SNAPSHOT]
... 17 common frames omitted
Caused by: org.springframework.web.client.HttpClientErrorException$Forbidden: 403 Forbidden: [{"errors":["1 error occurred:\n\t* permission denied\n\n"]}
]
at org.springframework.web.client.HttpClientErrorException.create(HttpClientErrorException.java:109) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:170) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:112) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:785) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:743) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:677) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:586) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.vault.core.VaultVersionedKeyValueTemplate.lambda$doRead$0(VaultVersionedKeyValueTemplate.java:103) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
... 21 common frames omitted
2021-07-30 02:35:37.713 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
Steps I followed
- Created Policy
cat <<EOF > policy.hcl
path "testpki/issue/testrole" {
capabilities = ["create", "read", "update"]
}
path "secret/*" {
capabilities = ["read"]
}
EOF
vault write sys/policy/testpolicy policy=@policy.hcl
vault token create -policy=testpolicy -display-name=testtoken
- RBAC
apiVersion: v1
kind: ServiceAccount
metadata:
name: vault-crd-serviceaccount
namespace: kube-vault
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: vault-crd-clusterrole
rules:
- apiGroups:
- apiextensions.k8s.io
resources:
- customresourcedefinitions
verbs:
- get
- apiGroups:
- "koudingspawn.de"
resources:
- vault
verbs:
- list
- watch
- get
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
- create
- patch
- update
- delete
- apiGroups:
- extensions
- apps
resources:
- deployments
verbs:
- update
- get
- patch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: vault-crd-clusterrole-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: vault-crd-clusterrole
subjects:
- kind: ServiceAccount
name: vault-crd-serviceaccount
namespace: kube-vault
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: vault.koudingspawn.de
spec:
group: koudingspawn.de
scope: Namespaced
names:
plural: vault
singular: vault
kind: Vault
shortNames:
- vt
versions:
- name: v1
served: true
storage: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
path:
type: string
pattern: '^.*?\/.*?(\/.*?)?$'
type:
type: string
enum:
- PKI
- PKIJKS
- CERT
- CERTJKS
- DOCKERCFG
- KEYVALUE
- KEYVALUEV2
- PROPERTIES
pkiConfiguration:
type: object
properties:
commonName:
type: string
altNames:
type: string
ipSans:
type: string
ttl:
type: string
pattern: '^[0-9]{1,}[hm]$'
jksConfiguration:
type: object
properties:
password:
type: string
alias:
type: string
keyName:
type: string
caAlias:
type: string
versionConfiguration:
type: object
properties:
version:
type: integer
propertiesConfiguration:
type: object
properties:
context:
type: object
x-kubernetes-preserve-unknown-fields: true
files:
type: object
x-kubernetes-preserve-unknown-fields: true
dockerCfgConfiguration:
type: object
properties:
type:
type: string
enum:
- KEYVALUE
- KEYVALUEV2
version:
type: integer
changeAdjustmentCallback:
type: object
properties:
type:
type: string
name:
type: string
required:
- type
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: vault-crd
name: vault-crd
namespace: kube-vault
spec:
selector:
matchLabels:
app: vault-crd
replicas: 1
template:
metadata:
labels:
app: vault-crd
spec:
serviceAccountName: vault-crd-serviceaccount
containers:
- name: vault-crd
image: daspawnw/vault-crd:1.6.4
env:
- name: KUBERNETES_VAULT_URL
value: "http://vault-internal.172.31.14.138.nip.io/v1/"
- name: KUBERNETES_VAULT_TOKEN
value: "s.sOARfxb4nDTsUCKmAVc379ru"
ports:
- containerPort: 8080
livenessProbe:
httpGet:
port: 8080
path: "/actuator/health"
initialDelaySeconds: 30
failureThreshold: 3
periodSeconds: 30
successThreshold: 1
timeoutSeconds: 5
restartPolicy: Always
---
- Create secrets
vault kv put internal/new/config username="db-readonly-username"
cat <<EOF > vault-crd-secrets-new.yaml
apiVersion: "koudingspawn.de/v1"
kind: Vault
metadata:
name: internal-new-config
spec:
path: "internal/new/config"
type: "KEYVALUEV2"
versionConfiguration:
version: 1
EOF
kubectl create -f vault-crd-secrets-new.yaml
Please let me know if I missing!!
DaspawnW commented
Hi @cloudcafetech,
Maybe its simply a copy paste issue, but in your permissions you grant access to secret path secret:
path "secret/*" {
capabilities = ["read"]
}
But it should be:
path "internal/*" {
capabilities = ["read"]
}
cloudcafetech commented
Thanks, Will check & update.
Just one more question if I have multiple path (one for internal and other for external) for two Kubernetes.
Will it be as follows ... ?
path "internal/*" {
capabilities = ["read"]
}path "external/*" {
capabilities = ["read"]
}
cloudcafetech commented
It's working, but update not happening.
Holding old value also even I delete & recreate vault (kind) :(