TektonHub CR not able to clone

Question

TektonHub CR not able to clone

Elyytscha opened this issue 6 months ago · 16 comments

Elyytscha commented 6 months ago

Expected Behavior

Creates a working Tektonhub with the default tekton catalog

Actual Behavior

Fails with cloning the default tekton catalog repo

Steps to Reproduce the Problem

kubectl apply -f tektonhub.yml

apiVersion: operator.tekton.dev/v1alpha1
kind: TektonHub
metadata:
  name: hub
spec:
  api:
    catalogRefreshInterval: 30m
    secret: tekton-hub-api
  customLogo: {}
  db: {}
  default: {}
  options:
    disabled: false
  targetNamespace: tekton-pipelines

this will create the following configmap

# Secret: name: tekton-hub-api
kind: ConfigMap
apiVersion: v1
metadata:
  name: tekton-hub-api
  namespace: tekton-pipelines
  labels:
    app: tekton-hub-api
    operator.tekton.dev/operand-name: tektoncd-hub
data:
  CATALOGS: |
    - name: tekton
      org: tektoncd
      type: community
      provider: github
      url: https://github.com/tektoncd/catalog
      revision: main
  CATALOG_REFRESH_INTERVAL: 30m
  CATEGORIES: |
    - Automation
    - Build Tools
    - CLI
    - Cloud
    - Code Quality
    - Continuous Integration
    - Deployment
    - Developer Tools
    - Image Build
    - Integration & Delivery
    - Git
    - Kubernetes
    - Messaging
    - Monitoring
    - Networking
    - Openshift
    - Publishing
    - Security
    - Storage
    - Testing
  DEFAULT: |
    scopes:
      - rating:read
      - rating:write
  SCOPES: ''

which will result in the following error

{"level":"info","ts":1711167219.9637597,"caller":"git/git.go:49","msg":"cloning","service":"git","name":"git","path":"/tmp/catalog/tekton"}
{"level":"error","ts":1711167220.0836065,"caller":"git/git.go:149","msg":"git fetch --recurse-submodules=yes origin main : error exit status 128 ;output: ","service":"git","name":"git","stacktrace":"github.com/tektoncd/hub/api/pkg/git.git\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:149\ngithub.com/tektoncd/hub/api/pkg/git.(*client).Fetch\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:59\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Process\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:235\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Run.func1\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:105"}
{"level":"error","ts":1711167220.0947776,"caller":"git/git.go:149","msg":"git pull --recurse-submodules=yes origin : error exit status 1 ;output: ","service":"git","name":"git","stacktrace":"github.com/tektoncd/hub/api/pkg/git.git\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:149\ngithub.com/tektoncd/hub/api/pkg/git.(*client).Fetch\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:62\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Process\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:235\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Run.func1\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:105"}
{"level":"info","ts":1711167220.0948365,"caller":"git/git.go:63","msg":"Failed to pull originerrexit status 1","service":"git","name":"git"}
{"level":"error","ts":1711167220.0962138,"caller":"git/git.go:149","msg":"git checkout main : error exit status 1 ;output: ","service":"git","name":"git","stacktrace":"github.com/tektoncd/hub/api/pkg/git.git\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:149\ngithub.com/tektoncd/hub/api/pkg/git.(*client).Fetch\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:65\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Process\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:235\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Run.func1\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:105"}
{"level":"error","ts":1711167220.0962665,"caller":"catalog/syncer.go:237","msg":"exit status 1clone failed","service":"syncer","action":"process","stacktrace":"github.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Process\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:237\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Run.func1\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:105"}
{"level":"info","ts":1711167220.1104074,"caller":"app/gorm_logger.go:76","msg":"UPDATE \"sync_jobs\" SET \"status\"='error',\"updated_at\"='2024-03-23 04:13:40.096' WHERE \"sync_jobs\".\"deleted_at\" IS NULL AND \"id\" = 1","service":"syncer","file":"/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:214","duration":"[14.027ms]","rows":1}

Additional Info

Kubernetes version:

Output of kubectl version:

$ oc version
Client Version: v4.2.0-alpha.0-2008-g286cfa5
Kustomize Version: v5.0.1
Server Version: 4.15.0-0.okd-2024-03-10-010116
Kubernetes Version: v1.28.2-3598+6e2789bbd58938-dirty

Tekton Pipeline version:

v0.53.3
Tekton Operator version:

v0.69.1

installed via OLM and operatorhub

Answer 1 · 2024-04-25T12:15:29.000Z

@PuneetPunamiya please take a look at this

Answer 2 · 2024-05-16T11:12:55.000Z

@Elyytscha I tried to reproduce this issue, but was not able to reproduce it. All pods were up and running.

Also seeing your error it seems like it was a network issue error. Can you try it once if you can still reproduce it ?

Answer 3 · 2024-05-27T19:01:26.000Z

I can still reproduce it:

apiVersion: operator.tekton.dev/v1alpha1
kind: TektonHub
metadata:
  name: hub
spec:
  api:
    catalogRefreshInterval: 30m
    secret: tekton-hub-api
  customLogo: {}
  db: {}
  default: {}
  options:
    disabled: false
  targetNamespace: tekton-pipelines

{"level":"info","ts":1716835976.1408136,"caller":"catalog/syncer.go:104","msg":"processing the queue","service":"syncer","action":"run"}
{"level":"info","ts":1716835976.142004,"caller":"app/gorm_logger.go:76","msg":"SELECT * FROM \"sync_jobs\" WHERE \"sync_jobs\".\"status\" = 'queued' AND \"sync_jobs\".\"deleted_at\" IS NULL ORDER BY created_at,\"sync_jobs\".\"id\" LIMIT 1","service":"syncer","file":"/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:217","duration":"[1.089ms]","rows":1}
{"level":"info","ts":1716835976.154482,"caller":"app/gorm_logger.go:76","msg":"UPDATE \"sync_jobs\" SET \"status\"='running',\"updated_at\"='2024-05-27 18:52:56.142' WHERE \"sync_jobs\".\"deleted_at\" IS NULL AND \"id\" = 1","service":"syncer","file":"/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:214","duration":"[12.366ms]","rows":1}
{"level":"info","ts":1716835976.1553483,"caller":"app/gorm_logger.go:76","msg":"SELECT * FROM \"catalogs\" WHERE \"catalogs\".\"id\" = 1 AND \"catalogs\".\"deleted_at\" IS NULL","service":"syncer","file":"/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:229","duration":"[0.716ms]","rows":1}
{"level":"info","ts":1716835976.155447,"caller":"git/git.go:49","msg":"cloning","service":"git","name":"git","path":"/tmp/catalog/tekton"}
{"level":"error","ts":1716835976.2589736,"caller":"git/git.go:149","msg":"git fetch --recurse-submodules=yes origin main : error exit status 128 ;output: ","service":"git","name":"git","stacktrace":"github.com/tektoncd/hub/api/pkg/git.git\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:149\ngithub.com/tektoncd/hub/api/pkg/git.(*client).Fetch\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:59\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Process\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:235\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Run.func1\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:105"}
{"level":"error","ts":1716835976.2688093,"caller":"git/git.go:149","msg":"git pull --recurse-submodules=yes origin : error exit status 1 ;output: ","service":"git","name":"git","stacktrace":"github.com/tektoncd/hub/api/pkg/git.git\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:149\ngithub.com/tektoncd/hub/api/pkg/git.(*client).Fetch\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:62\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Process\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:235\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Run.func1\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:105"}
{"level":"info","ts":1716835976.2688737,"caller":"git/git.go:63","msg":"Failed to pull originerrexit status 1","service":"git","name":"git"}
{"level":"error","ts":1716835976.2704506,"caller":"git/git.go:149","msg":"git checkout main : error exit status 1 ;output: ","service":"git","name":"git","stacktrace":"github.com/tektoncd/hub/api/pkg/git.git\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:149\ngithub.com/tektoncd/hub/api/pkg/git.(*client).Fetch\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:65\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Process\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:235\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Run.func1\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:105"}
{"level":"error","ts":1716835976.2705216,"caller":"catalog/syncer.go:237","msg":"exit status 1clone failed","service":"syncer","action":"process","stacktrace":"github.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Process\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:237\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Run.func1\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:105"}
{"level":"info","ts":1716835976.2864943,"caller":"app/gorm_logger.go:76","msg":"UPDATE \"sync_jobs\" SET \"status\"='error',\"updated_at\"='2024-05-27 18:52:56.27' WHERE \"sync_jobs\".\"deleted_at\" IS NULL AND \"id\" = 1","service":"syncer","file":"/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:214","duration":"[15.865ms]","rows":1}
{"level":"info","ts":1716835976.2876918,"caller":"app/gorm_logger.go:76","msg":"SELECT count(*) FROM \"sync_jobs\" WHERE \"sync_jobs\".\"status\" = 'queued' AND \"sync_jobs\".\"deleted_at\" IS NULL","service":"syncer","file":"/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:186","duration":"[1.065ms]","rows":1}
{"level":"info","ts":1716835976.2877243,"caller":"catalog/syncer.go:191","msg":"queued job count: 0","service":"syncer","action":"next"}

Answer 4 · 2024-06-25T06:53:42.000Z

Have you upgraded Hub from previous version or did you fresh install Hub ?

Answer 5 · 2024-06-25T15:32:11.000Z

i freshly installed the hub, it happens everyime i want to install the hub from scratch

Answer 6 · 2024-06-26T08:19:15.000Z

I tried this again by fresh installing hub, i don't see any issue
I think you have upgraded from a previous version of operator

Answer 7 · 2024-06-26T13:25:46.000Z

I can confirm with Tekton Operator v0.70.0 its now getting past this step, now it syncs the task repo successfully as I interpret the logs. but it still shows:

should I open a new issue regarding this?

i attached the logs for now over here
tekton-hub-api-7c8f95779-np5h2-tekton-hub-api.log
tekton-hub-ui-5fb499c944-5g7rk-tekton-hub-ui.log

Answer 8 · 2024-06-26T13:41:50.000Z

Can you hit this url and check if you can see the resources in the api
<API_URL>/v1/resources

Answer 9 · 2024-06-26T15:37:34.000Z

yes I can, but I think due to your last comment I found the issue: API_URL is empty

Modified ENV
window.config = {
  API_URL: '',
  API_VERSION: 'v1',
  AUTH_BASE_URL: '',
  REDIRECT_URI: '',
  CUSTOM_LOGO_MEDIA_TYPE: '',
  CUSTOM_LOGO_BASE64_DATA: '',
};

https://github.com/tektoncd/hub/blob/main/docs/DEPLOYMENT.md#update-ui-configmap
Could you point me to an example or the docs how its possible to set the API_URL in the TektonHub CR? I think then we can close the issue.

Answer 10 · 2024-07-01T08:38:31.000Z

If you are installing Hub on openshift in that case the API_URL, AUTH_BASE_URL and REDIRECT_URL should be set by the operator only, can you see the values in configMap i.e. tekton-hub-ui configMap

And if in case you are using it on other k8s cluster then for that we might have to add a support for that where you can provide the url and UI will be served on that url

Answer 11 · 2024-07-01T08:43:41.000Z

As of today you can add these fields only in the Hub CR https://github.com/tektoncd/operator/blob/main/docs/TektonHub.md