Invalid credentials on Clone
0x01F4 opened this issue · 7 comments
Jenkins and plugins versions report
Environment
Jenkins: 2.446
OS: Linux - 5.4.17-2136.300.7.el8uek.aarch64
Java: 11.0.20 - Red Hat, Inc. (OpenJDK 64-Bit Server VM)
ansicolor:1.0.4
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
authentication-tokens:1.53.v1c90fd9191a_b_
bitbucket-oauth:0.13
blueocean-autofavorite:1.2.5
blueocean-commons:1.27.11
blueocean-config:1.27.11
blueocean-core-js:1.27.11
blueocean-dashboard:1.27.11
blueocean-display-url:2.4.2
blueocean-events:1.27.11
blueocean-git-pipeline:1.27.11
blueocean-github-pipeline:1.27.11
blueocean-i18n:1.27.11
blueocean-jwt:1.27.11
blueocean-personalization:1.27.11
blueocean-pipeline-api-impl:1.27.11
blueocean-pipeline-editor:1.27.11
blueocean-pipeline-scm-api:1.27.11
blueocean-rest:1.27.11
blueocean-rest-impl:1.27.11
blueocean-web:1.27.11
bootstrap5-api:5.3.2-4
bouncycastle-api:2.30.1.77-225.v26ea_c9455fd9
branch-api:2.1152.v6f101e97dd77
build-timeout:1.32
build-user-vars-plugin:1.9
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.0.2
cloudbees-bitbucket-branch-source:876.v857269a_5f439
cloudbees-folder:6.921.vfb_b_224371fb_4
command-launcher:107.v773860566e2e
commons-lang3-api:3.13.0-62.v7d18e55f51e2
commons-text-api:1.11.0-95.v22a_d30ee5d36
credentials:1319.v7eb_51b_3a_c97b_
credentials-binding:657.v2b_19db_7d6e6d
data-tables-api:1.13.8-4
display-url-api:2.200.vb_9327d658781
durable-task:550.v0930093c4b_a_6
echarts-api:5.4.3-4
email-ext:2.104
favorite:2.208.v91d65b_7792a_c
font-awesome-api:6.5.1-3
git:5.2.1
git-client:4.6.0
git-server:114.v068a_c7cc2574
github:1.38.0
github-api:1.318-461.v7a_c09c9fa_d63
github-branch-source:1772.va_69eda_d018d4
gradle:2.10
gson-api:2.10.1-15.v0d99f670e0a_7
handy-uri-templates-2-api:2.1.8-30.v7e777411b_148
hidden-parameter:237.v4b_df26c7a_f0e
htmlpublisher:1.32
instance-identity:185.v303dc7c645f9
ionicons-api:56.v1b_1c8c49374e
jackson2-api:2.16.1-373.ve709c6871598
jakarta-activation-api:2.0.1-3
jakarta-mail-api:2.0.1-3
javax-activation-api:1.2.0-6
javax-mail-api:1.6.2-9
jaxb:2.3.9-1
jdk-tool:73.vddf737284550
jenkins-design-language:1.27.11
jersey2-api:2.41-133.va_03323b_a_1396
jjwt-api:0.11.5-77.v646c772fddb_0
joda-time-api:2.12.7-29.v5a_b_e3a_82269a_
jquery3-api:3.7.1-2
jsch:0.2.16-86.v42e010d9484b_
json-path-api:2.9.0-33.v2527142f2e1d
junit:1259.v65ffcef24a_88
lockable-resources:1243.v346d600eea_24
mailer:463.vedf8358e006b_
matrix-auth:3.2.1
matrix-project:822.824.v14451b_c0fd42
mina-sshd-api-common:2.12.0-90.v9f7fb_9fa_3d3b_
mina-sshd-api-core:2.12.0-90.v9f7fb_9fa_3d3b_
okhttp-api:4.11.0-172.vda_da_1feeb_c6e
oracle-cloud-infrastructure-compute:1.0.17
pam-auth:1.10
pipeline-build-step:540.vb_e8849e1a_b_d8
pipeline-github-lib:42.v0739460cda_c4
pipeline-graph-analysis:202.va_d268e64deb_3
pipeline-groovy-lib:704.vc58b_8890a_384
pipeline-input-step:491.vb_07d21da_1a_fb_
pipeline-milestone-step:111.v449306f708b_7
pipeline-model-api:2.2175.v76a_fff0a_2618
pipeline-model-definition:2.2175.v76a_fff0a_2618
pipeline-model-extensions:2.2175.v76a_fff0a_2618
pipeline-rest-api:2.34
pipeline-stage-step:305.ve96d0205c1c6
pipeline-stage-tags-metadata:2.2175.v76a_fff0a_2618
pipeline-stage-view:2.34
pipeline-utility-steps:2.16.2
plain-credentials:143.v1b_df8b_d3b_e48
plugin-util-api:4.1.0
pubsub-light:1.18
resource-disposer:0.23
scm-api:683.vb_16722fb_b_80b_
script-security:1326.vdb_c154de8669
slack:684.v833089650554
snakeyaml-api:2.2-111.vc6598e30cc65
sse-gateway:1.26
ssh-credentials:308.ve4497b_ccd8f4
ssh-slaves:2.948.vb_8050d697fec
sshd:3.322.v159e91f6a_550
structs:337.v1b_04ea_4df7c8
timestamper:1.26
token-macro:400.v35420b_922dcb_
trilead-api:2.133.vfb_8a_7b_9c5dd1
variant:60.v7290fc0eb_b_cd
workflow-aggregator:596.v8c21c963d92d
workflow-api:1291.v51fd2a_625da_7
workflow-basic-steps:1042.ve7b_140c4a_e0c
workflow-cps:3867.v535458ce43fd
workflow-durable-task-step:1331.vc8c2fed35334
workflow-job:1400.v7fd111b_ec82f
workflow-multibranch:783.va_6eb_ef636fb_d
workflow-scm-step:415.v434365564324
workflow-step-api:657.v03b_e8115821b_
workflow-support:865.v43e78cc44e0d
ws-cleanup:0.45
What Operating System are you using (both controller, and any agents involved in the problem)?
Oracle linux
Reproduction steps
Create a job with cron
when the cron run it fails to clone with invalid credentials
`#!/bin/groovy
pipeline {
agent {
node {
label 'build'
}
}
options {
disableConcurrentBuilds()
}
triggers {
cron('0 9 * * *')
}
stages {
stage('CleanUp Container Images') {
steps {
sh "sudo make run"
}
}
}`
Expected Results
Clone happens successfully
Actual Results
Fails to clone when job is triggered by cron but works after Jenkins restart and triggered manually
`Fetching upstream changes from https://x-token-auth@bitbucket.org/xxxxxxx/xxxxxxx.git
git --version # timeout=10
git --version # 'git version 2.31.1'
git fetch --no-tags --force --progress -- https://x-token-auth:{xxxxxxx}@bitbucket.org/xxxxxxx/xxxxxxx.git +refs/heads/master:refs/remotes/origin/master # timeout=10
ERROR: Error cloning remote repo 'origin'
hudson.plugins.git.GitException: Command "git fetch --no-tags --force --progress -- https://x-token-auth:{xxxxxxx}@bitbucket.org/xxxxxxx/xxxxxxx.git +refs/heads/master:refs/remotes/origin/master" returned status code 128:
stdout:
stderr: remote: Invalid credentials
fatal: Authentication failed for 'https://bitbucket.org/xxxxxxx/xxxxxxx.git/'
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2842)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2185)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:635)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:871)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:170)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161)
at hudson.remoting.UserRequest.perform(UserRequest.java:211)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:377)
at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)`
Anything else?
No response
Are you interested in contributing a fix?
No response
We experience the same issue very frequently for many of our "normal" pipelines. The last version that properly worked was the 866 one.
The only way we can run those jobs is to trigger a manual pipeline scan and then it works for some time for that specific multibranch pipeline.
Much like @cshabee, we've started to experience this issue recently and it is not limited to runs triggered by cron. The workaround we found was to save the configuration for the multibranch pipeline project (without doing any changes), which in turn triggers a pipeline scan. So next time we encounter the problem, we will try with @cshabee's suggestion and only trigger the scan 🙏
I can file a separate issue for the "general issue" (not tied to cron as the trigger) or do you think this issue can be used for tracking both cases, @giri-vsr ? 🙂
We have that option already set (to 1 day), it seems to not solve the issue, however setting it to a more frequent option might do the trick, I'm waiting for your findings, too.
I see this issue on my Jenkins as well.
I suspect Bitbucket has made some changes.
On 15th February (Hong Kong time), ssh bitbucket.org works, but git pull does not work. Same "auth failed".
I have generated new SSH keys, added to bitbucket.org, tried again, failed.
I have suspected if it is due to ED25519 key format. Tried RSA still did not work.
Yet, using Gitkraken Bitbucket integration works.
On 16th, pulling with SSH still fails 99% of the time. One or two pull succeeded.
Pulling on Jenkins failed.
On 21st, Bitbucket announced there was incident.
Some days later, whenever Jenkins fails to pull, I create a new app-password or oauth-token.
And the issue persists till now.
I hope this provide some clues what Bitbucket might have changed.
We have that option already set (to 1 day), it seems to not solve the issue, however setting it to a more frequent option might do the trick, I'm waiting for your findings, too.
Workaround setting it to 1 hour seems to be working.Haven't go any failure yet.