--remote builds cli hangs when using openshift pipelines

Question

--remote builds cli hangs when using openshift pipelines

Opened this issue a year ago · 13 comments

Having success with OpenShift Serverless 1.29.0 using the kn func deploy --remote except my cli seems to hang. It'll never finish waiting and I have to manually cancel it.

When its running (in the cli), I can see the pipeline history in my openshift cluster

However, the moment I ctrl-c out of the cli to get my cursor back all pipeline history is lost.

lance commented a year ago

/kind bug

Answer 1 · 2023-07-12T10:50:08.000Z

Let it run for longer and got some errors

❯ kn func deploy --remote -p ./func-helloworld
🕔 Creating Pipeline resources
E0712 18:43:48.263546   42989 reflector.go:140] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: Failed to watch *v1.PipelineRun: the server has asked for the client to provide credentials (get pi🕔 ineruns.tekton.dev)
W0712 18:43:49.440375   42989 reflector.go:424] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: failed to list *v1.PipelineRun: Unauthorized
🕘 12 18:43:49.440528   42989 reflector.go:140] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: Failed to watch *v1.PipelineRun: failed to list *v1.PipelineRun: Unauthorized
W0712 18:43:52.202352   42989 reflector.go:424] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: failed to list *v1.PipelineRun: Unauthorized
🕛 12 18:43:52.202418   42989 reflector.go:140] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: Failed to watch *v1.PipelineRun: failed to list *v1.PipelineRun: Unauthorized
W0712 18:43:57.367452   42989 reflector.go:424] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: failed to list *v1.PipelineRun: Unauthorized
🕔 12 18:43:57.367628   42989 reflector.go:140] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: Failed to watch *v1.PipelineRun: failed to list *v1.PipelineRun: Unauthorized
W0712 18:44:08.688306   42989 reflector.go:424] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: failed to list *v1.PipelineRun: Unauthorized
🕗 12 18:44:08.688376   42989 reflector.go:140] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: Failed to watch *v1.PipelineRun: failed to list *v1.PipelineRun: Unauthorized
W0712 18:44:24.549190   42989 reflector.go:424] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: failed to list *v1.PipelineRun: Unauthorized
🕔 12 18:44:24.549456   42989 reflector.go:140] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: Failed to watch *v1.PipelineRun: failed to list *v1.PipelineRun: Unauthorized
W0712 18:45:07.466489   42989 reflector.go:424] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: failed to list *v1.PipelineRun: Unauthorized
🕙 12 18:45:07.466676   42989 reflector.go:140] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: Failed to watch *v1.PipelineRun: failed to list *v1.PipelineRun: Unauthorized
W0712 18:46:00.783052   42989 reflector.go:424] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: failed to list *v1.PipelineRun: Unauthorized
🕓 12 18:46:00.783235   42989 reflector.go:140] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: Failed to watch *v1.PipelineRun: failed to list *v1.PipelineRun: Unauthorized
W0712 18:46:42.175003   42989 reflector.go:424] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: failed to list *v1.PipelineRun: Unauthorized
🕚 12 18:46:42.175776   42989 reflector.go:140] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: Failed to watch *v1.PipelineRun: failed to list *v1.PipelineRun: Unauthorized
W0712 18:47:23.656139   42989 reflector.go:424] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: failed to list *v1.PipelineRun: Unauthorized
🕘 12 18:47:23.656213   42989 reflector.go:140] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: Failed to watch *v1.PipelineRun: failed to list *v1.PipelineRun: Unauthorized
W0712 18:48:13.813756   42989 reflector.go:424] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: failed to list *v1.PipelineRun: Unauthorized
🕖 12 18:48:13.813936   42989 reflector.go:140] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: Failed to watch *v1.PipelineRun: failed to list *v1.PipelineRun: Unauthorized
W0712 18:48:47.230727   42989 reflector.go:424] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: failed to list *v1.PipelineRun: Unauthorized
🕙 12 18:48:47.230871   42989 reflector.go:140] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: Failed to watch *v1.PipelineRun: failed to list *v1.PipelineRun: Unauthorized

Answer 2 · 2023-07-12T17:03:29.000Z

@MichaelMarkieta have you used Red Hat built kn?

Answer 3 · 2023-07-12T17:07:42.000Z

Yes, I am using the kn CLI supplied by Red Hat OpenShift through the (?) menu.

…

On Thu, Jul 13, 2023, 1:03 AM Matej Vasek ***@***.***> wrote: @MichaelMarkieta <https://github.com/MichaelMarkieta> have you used Red Hat built kn? — Reply to this email directly, view it on GitHub <#1861 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXXWSE737VZZBHTITEE5ULXP3KGXANCNFSM6AAAAAA2HHE5HY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

Answer 4 · 2023-07-12T17:11:06.000Z

@MichaelMarkieta OK, I just wanted to be sure. Upstream kn may miss some openshift specific patches.

Answer 5 · 2023-07-12T17:18:46.000Z

@MichaelMarkieta what is output of kn func --version ?

Answer 6 · 2023-07-12T17:22:21.000Z

@matejvasek

❯ kn version
Version:      v1.8.1
Build Date:   2023-05-31 04:20:23
Git Revision: 8cb7de1e
Supported APIs:
* Serving
  - serving.knative.dev/v1 (knative-serving v1.8.0)
* Eventing
  - sources.knative.dev/v1 (knative-eventing v1.8.1)
  - eventing.knative.dev/v1 (knative-eventing v1.8.1)
❯ kn func version
v1.10.0

Answer 7 · 2023-07-12T18:00:10.000Z

@MichaelMarkieta What are versions of components of Red Hat OpenShift Pipelines operator?

Answer 8 · 2023-07-12T18:00:56.000Z

Are you running commands are developer user or as kubeadmin user?

Answer 9 · 2023-07-13T01:27:51.000Z

Red Hat OpenPipelines version: 1.11.0
Executing all commands as cluster-admin

Answer 10 · 2023-07-15T06:53:28.000Z

Downloaded all task logs for a successful run

undefined

step-generate

Processing Build Environment Variables
Generated Build Env Var file
------------------------------

------------------------------
Application dockerfile generated in /gen-source/Dockerfile.gen
Preparing func.yaml for later deployment
Function image registry: default-route-openshift-image-registry.apps.kayatoast.bufj.p1.openshiftapps.com/serverless


step-build

STEP 1/8: FROM registry.access.redhat.com/ubi8/nodejs-16
Trying to pull registry.access.redhat.com/ubi8/nodejs-16:latest...
Getting image source signatures
Checking if image destination supports signatures
Copying blob sha256:6c53be4efe39270422e3e2f7ee1c15887955e3d5e378ea6577c622f358f87691
Copying blob sha256:46e6dded7bc8d63e55de90e47014ed4a2717670c158710d5e48d5f4dd291d2dc
Copying blob sha256:d96c2f4a5465b79c577fdecc27cd49a2c0f103fd2cfd372e667512be069e0143
Copying config sha256:6864215d3e3942c0551dbbffe35cd5be18676ac631543c4444dd9513561a0622
Writing manifest to image destination
Storing signatures
STEP 2/8: LABEL "io.openshift.s2i.build.image"="registry.access.redhat.com/ubi8/nodejs-16"       "io.openshift.s2i.build.source-location"="."
--> ee06c3d2604
STEP 3/8: USER root
--> 55bac15cfed
STEP 4/8: COPY upload/src /tmp/src
--> 5f40ffaac6e
STEP 5/8: RUN chown -R 1001:0 /tmp/src
--> 2084739cdd1
STEP 6/8: USER 1001
--> b68d1e5790c
STEP 7/8: RUN /usr/libexec/s2i/assemble
---> Installing application source ...
---> Installing all dependencies

added 186 packages, and audited 187 packages in 3s

50 packages are looking for funding
  run `npm fund` for details

3 moderate severity vulnerabilities

To address all issues, run:
  npm audit fix

Run `npm audit` for details.
npm notice
npm notice New major version of npm available! 8.19.3 -> 9.8.0
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v9.8.0>
npm notice Run `npm install -g npm@9.8.0` to update!
npm notice
---> Building in production mode
---> Pruning the development dependencies

up to date, audited 111 packages in 565ms

33 packages are looking for funding
  run `npm fund` for details

1 moderate severity vulnerability

To address all issues, run:
  npm audit fix

Run `npm audit` for details.
/tmp is not a mountpoint
---> Cleaning the /tmp/npm-*
/opt/app-root/src/.npm is not a mountpoint
---> Cleaning the npm cache /opt/app-root/src/.npm
--> c44582dde7a
STEP 8/8: CMD /usr/libexec/s2i/run
COMMIT default-route-openshift-image-registry.apps.kayatoast.bufj.p1.openshiftapps.com/serverless/function-helloworld:latest
--> ba6d9865ff7
Successfully tagged default-route-openshift-image-registry.apps.kayatoast.bufj.p1.openshiftapps.com/serverless/function-helloworld:latest
ba6d9865ff7b33acbc8a0259ffa0a52dee65ea5281340c4cbe6e7ada37706108
Getting image source signatures
Copying blob sha256:7cca5afbfa3202d39489382a309d5c1026dad9fafc6db518b462bde4e754c005
Copying blob sha256:b51194abfc91510e7cc523fd1735ea595e9ab7b8fbb8697e7e2ac6bb04b0f5e3
Copying blob sha256:11d6ed21791d9352526913f1b6e340a112732eb9982b3e69d3d837a36e2333a5
Copying blob sha256:fb6a7cccdb8498194b19da874644bc52d0829c71647459de066ce00da5c97dd7
Copying blob sha256:0ae5199b61a3134c513d57eda9c7144e8cbbbc89b2f42405baadc1b323c56236
Copying blob sha256:800063d2e34132f2c0e199288fa507c2c4946373c16e130c1bf6b456609211ef
Copying config sha256:ba6d9865ff7b33acbc8a0259ffa0a52dee65ea5281340c4cbe6e7ada37706108
Writing manifest to image destination
Storing signatures
sha256:c464de9205021fa8f4bdfadc226afbd40f84ede025fc02ad4f99d677c3314f87

undefined

step-func-deploy

⬆️  Deploying function to the cluster
Environment:
	DEV_MODE=false
	NODE_ENV=production
	DEBUG_PORT=5858
Launching via npm...
npm info using npm@8.19.3
npm info using node@v16.19.1
npm timing npm:load:whichnode Completed in 0ms
npm timing config:load:defaults Completed in 2ms
npm timing config:load:file:/usr/lib/node_modules/npm/npmrc Completed in 1ms
npm timing config:load:builtin Completed in 1ms
npm timing config:load:cli Completed in 2ms
npm timing config:load:env Completed in 1ms
npm timing config:load:project Completed in 10ms
npm timing config:load:file:/opt/app-root/src/.npmrc Completed in 0ms
npm timing config:load:user Completed in 0ms
npm timing config:load:file:/opt/app-root/src/.npm-global/etc/npmrc Completed in 1ms
npm timing config:load:global Completed in 1ms
npm timing config:load:validate Completed in 0ms
npm timing config:load:credentials Completed in 5ms
npm timing config:load:setEnvs Completed in 1ms
npm timing config:load Completed in 23ms
npm timing npm:load:configload Completed in 23ms
npm timing npm:load:mkdirpcache Completed in 1ms
npm timing npm:load:mkdirplogs Completed in 1ms
npm timing npm:load:setTitle Completed in 1ms
npm timing config:load:flatten Completed in 3ms
npm timing npm:load:display Completed in 5ms
npm timing npm:load:logFile Completed in 4ms
npm timing npm:load:timers Completed in 0ms
npm timing npm:load:configScope Completed in 0ms
npm timing npm:load Completed in 37ms

> http-handler@0.1.0 start
> FUNC_LOG_LEVEL=info faas-js-runtime ./index.js

npm http fetch GET 200 https://registry.npmjs.org/npm 197ms (cache miss)
{"level":30,"time":1689403771197,"pid":23,"hostname":"function-helloworld-00003-deployment-6f5f6686c9-kzfxr","node_version":"v16.19.1","msg":"Server listening at http://[::]:8080"}
{"level":30,"time":1689403771249,"pid":23,"hostname":"function-helloworld-00003-deployment-6f5f6686c9-kzfxr","node_version":"v16.19.1","reqId":"req-1","req":{"method":"GET","url":"/health/readiness","hostname":"127.0.0.1:8080","remoteAddress":"::ffff:127.0.0.1","remotePort":49408},"msg":"incoming request"}
{"level":30,"time":1689403771254,"pid":23,"hostname":"function-helloworld-00003-deployment-6f5f6686c9-kzfxr","node_version":"v16.19.1","reqId":"req-1","res":{"statusCode":200},"responseTime":4.9982679998502135,"msg":"request completed"}
✅ Function updated in namespace "serverless" and exposed at URL:
   https://function-helloworld-serverless.apps.kayatoast.bufj.p1.openshiftapps.com

Answer 11 · 2023-07-25T16:09:50.000Z

I have the exact same issue.

OpenShift Serverless 1.29
OpenShift Pipelines v1.11.0
kn v1.10.0
kn-func v0.37.0-6-gf38444bf

The PipelineRun succeeds and the function is correctly deployed, but as soon as I terminate the func deploy --remote process with CTRL-C, the PipelineRun (which has already succeeded) disappears from the history.

Note that in my case I keep getting this warning:

🕘 eline/pkg/client/clientset/versioned/scheme/register.go:32\"") has prevented the request from succeeding
W0725 17:46:03.497396   62896 reflector.go:347] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: watch of *v1.PipelineRun ended with: an error on the server ("unable to decode an event from the watch stream: unable to decode watch event: no kind \"PipelineRun\" is registered for version \"tekton.dev/v1\" in scheme \"github.com/tektoncd/

(BTW the first letters of each line are not displayed, or hidden because of the emoji, and there are also display issues when requesting the registry username and password, but this is unrelated.)

Answer 12 · 2023-11-22T01:26:42.000Z

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.