On cluster building stuck in pending state
Closed this issue · 1 comments
Hi,
I'm trying to build a knative function with --remote
, however the pipeline stucks in pending state, what should I do to fix it?
$ func deploy --remote --service-account agent-less-builder
Warning: namespace chosen is 'agent-less', but currently active namespace is 'default'. Continuing with deployment to 'agent-less'.
Creating Pipeline resourcesRunning Pipeline Task: Fetching git repository with the function source code
The pipeline and taskun pods are created successfully, and it says that "0/1 nodes are available: 1 node(s) didn''t match pod affinity rules.".
conditions:
- lastTransitionTime: '2024-03-27T09:17:59Z'
message: 'pod status "PodScheduled":"False"; message: "0/1 nodes are available: 1 node(s) didn''t match pod affinity rules. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling.."'
reason: Pending
status: Unknown
type: Succeeded
podName: bing-search-tool-pack-git-pipeline-run-9x49m-fetch-sources-pod
After digging into it, I find that the issue might be caused by the workspaces that used in the pipelinerun:
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
annotations:
manifestival: new
generateName: helloworld-pack-git-pipeline-run-
labels:
boson.dev/function: 'true'
boson.dev/runtime: node
function.knative.dev: 'true'
function.knative.dev/name: helloworld
function.knative.dev/runtime: node
namespace: default
spec:
params:
- name: gitRepository
value: https://github.com/drriguz/helloworld.git
- name: gitRevision
value: master
- name: contextDir
value: ''
- name: imageName
value: localhost:5000/helloworld/helloworld:latest
- name: registry
value: localhost:5000/helloworld
- name: builderImage
value: ghcr.io/knative/builder-jammy-base:latest
- name: buildEnvs
value:
- '='
pipelineRef:
name: helloworld-pack-git-pipeline
taskRunTemplate:
serviceAccountName: default
timeouts:
pipeline: 1h0m0s
workspaces:
- name: source-workspace
persistentVolumeClaim:
claimName: helloworld-pack-git-pipeline-pvc
subPath: source
- name: cache-workspace
persistentVolumeClaim:
claimName: helloworld-pack-git-pipeline-pvc
subPath: cache
- name: dockerconfig-workspace
secret:
secretName: helloworld-pack-git-pipeline-secret
However, because I'm running it using minikube, pvc-04c96a5c-2916-49b5-9909-34b7b7998838
is use hostpath storageclass and it's RWO
, however it's shared by both source-workspace
and cache-workspace
, which then cause the pod getting stuck waiting for the PVC to become available.
$ kubectl get pvc
helloworld-pack-git-pipeline-pvc Bound pvc-04c96a5c-2916-49b5-9909-34b7b7998838 256Mi RWO standard
It seems that this could be solved by disable affinity assistant:
kubectl edit configmap feature-flags -n tekton-pipelines
# set disable-affinity-assistant to true