Authentication fails when using OAuth Credentials
rgrizzell opened this issue · 13 comments
Jenkins and plugins versions report
Environment
Jenkins: 2.444
OS: Linux - 6.2.0-1018-aws
Java: 11.0.21 - Ubuntu (OpenJDK 64-Bit Server VM)
---
adoptopenjdk:1.5
ansicolor:1.0.4
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
atlassian-jira-software-cloud:2.0.12
authentication-tokens:1.53.v1c90fd9191a_b_
authorize-project:1.7.1
aws-credentials:218.v1b_e9466ec5da_
aws-java-sdk:1.12.633-430.vf9a_e567a_244f
aws-java-sdk-cloudformation:1.12.633-430.vf9a_e567a_244f
aws-java-sdk-codebuild:1.12.633-430.vf9a_e567a_244f
aws-java-sdk-ec2:1.12.633-430.vf9a_e567a_244f
aws-java-sdk-ecr:1.12.633-430.vf9a_e567a_244f
aws-java-sdk-ecs:1.12.633-430.vf9a_e567a_244f
aws-java-sdk-efs:1.12.633-430.vf9a_e567a_244f
aws-java-sdk-elasticbeanstalk:1.12.633-430.vf9a_e567a_244f
aws-java-sdk-iam:1.12.633-430.vf9a_e567a_244f
aws-java-sdk-kinesis:1.12.633-430.vf9a_e567a_244f
aws-java-sdk-logs:1.12.633-430.vf9a_e567a_244f
aws-java-sdk-minimal:1.12.633-430.vf9a_e567a_244f
aws-java-sdk-secretsmanager:1.12.633-430.vf9a_e567a_244f
aws-java-sdk-sns:1.12.633-430.vf9a_e567a_244f
aws-java-sdk-sqs:1.12.633-430.vf9a_e567a_244f
aws-java-sdk-ssm:1.12.633-430.vf9a_e567a_244f
basic-branch-build-strategies:81.v05e333931c7d
blueocean:1.27.10
blueocean-autofavorite:1.2.5
blueocean-bitbucket-pipeline:1.27.10
blueocean-commons:1.27.10
blueocean-config:1.27.10
blueocean-core-js:1.27.10
blueocean-dashboard:1.27.10
blueocean-display-url:2.4.2
blueocean-events:1.27.10
blueocean-git-pipeline:1.27.10
blueocean-github-pipeline:1.27.10
blueocean-i18n:1.27.10
blueocean-jira:1.27.10
blueocean-jwt:1.27.10
blueocean-personalization:1.27.10
blueocean-pipeline-api-impl:1.27.10
blueocean-pipeline-editor:1.27.10
blueocean-pipeline-scm-api:1.27.10
blueocean-rest:1.27.10
blueocean-rest-impl:1.27.10
blueocean-web:1.27.10
bootstrap5-api:5.3.2-3
bouncycastle-api:2.30.1.77-225.v26ea_c9455fd9
branch-api:2.1148.vce12cfcdf090
build-timeout:1.32
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.0.2
cloudbees-bitbucket-branch-source:872.vb_8fa_89198661
cloudbees-disk-usage-simple:203.v3f46a_7462b_1a_
cloudbees-folder:6.921.vfb_b_224371fb_4
command-launcher:107.v773860566e2e
commons-httpclient3-api:3.1-3
commons-lang3-api:3.13.0-62.v7d18e55f51e2
commons-text-api:1.11.0-95.v22a_d30ee5d36
config-file-provider:968.ve1ca_eb_913f8c
copyartifact:722.v0662a_9b_e22a_c
cors-filter:1.1
coverage:1.10.0
credentials:1319.v7eb_51b_3a_c97b_
credentials-binding:657.v2b_19db_7d6e6d
dark-theme:416.v535839b_c4e88
dashboard-view:2.508.va_74654f026d1
data-tables-api:1.13.8-2
display-url-api:2.200.vb_9327d658781
docker-commons:439.va_3cb_0a_6a_fb_29
docker-workflow:572.v950f58993843
durable-task:550.v0930093c4b_a_6
ec2-fleet:3.2.0
echarts-api:5.4.3-2
editable-choice:71.v02a291ebbe45
envinject:2.908.v66a_774b_31d93
envinject-api:1.199.v3ce31253ed13
extended-read-permission:53.v6499940139e5
extensible-choice-parameter:1.8.1
external-monitor-job:215.v2e88e894db_f8
favorite:2.208.v91d65b_7792a_c
font-awesome-api:6.5.1-2
forensics-api:2.3.0
git:5.2.1
git-client:4.6.0
git-forensics:2.0.0
git-parameter:0.9.19
git-server:114.v068a_c7cc2574
github:1.38.0
github-api:1.318-461.v7a_c09c9fa_d63
github-branch-source:1772.va_69eda_d018d4
golang:1.4
gradle:2.10
gson-api:2.10.1-15.v0d99f670e0a_7
handy-uri-templates-2-api:2.1.8-30.v7e777411b_148
htmlpublisher:1.32
instance-identity:185.v303dc7c645f9
ionicons-api:56.v1b_1c8c49374e
jackson2-api:2.16.1-373.ve709c6871598
jacoco:3.3.5
jakarta-activation-api:2.0.1-3
jakarta-mail-api:2.0.1-3
javadoc:243.vb_b_503b_b_45537
javax-activation-api:1.2.0-6
javax-mail-api:1.6.2-9
jaxb:2.3.9-1
jdk-tool:73.vddf737284550
jenkins-design-language:1.27.10
jersey2-api:2.41-133.va_03323b_a_1396
jira:3.12
jjwt-api:0.11.5-77.v646c772fddb_0
jnr-posix-api:3.1.18-1
job-dsl:1.87
jobConfigHistory:1229.v3039470161a_d
joda-time-api:2.12.7-29.v5a_b_e3a_82269a_
jquery:1.12.4-1
jquery3-api:3.7.1-1
jsch:0.2.16-86.v42e010d9484b_
json-path-api:2.9.0-33.v2527142f2e1d
junit:1259.v65ffcef24a_88
ldap:711.vb_d1a_491714dc
lockable-resources:1232.v512d6c434eb_d
mailer:463.vedf8358e006b_
mapdb-api:1.0.9-28.vf251ce40855d
matrix-auth:3.2.1
matrix-project:822.824.v14451b_c0fd42
maven-plugin:3.23
metrics:4.2.21-449.v6960d7c54c69
mina-sshd-api-common:2.12.0-90.v9f7fb_9fa_3d3b_
mina-sshd-api-core:2.12.0-90.v9f7fb_9fa_3d3b_
nexus-artifact-uploader:2.14
nodejs:1.6.1
okhttp-api:4.11.0-172.vda_da_1feeb_c6e
openJDK-native-plugin:1.8
pam-auth:1.10
parameterized-scheduler:262.v00f3d90585cc
periodicbackup:2.0
pipeline-aws:1.43
pipeline-build-step:540.vb_e8849e1a_b_d8
pipeline-graph-analysis:202.va_d268e64deb_3
pipeline-groovy-lib:704.vc58b_8890a_384
pipeline-input-step:491.vb_07d21da_1a_fb_
pipeline-milestone-step:111.v449306f708b_7
pipeline-model-api:2.2175.v76a_fff0a_2618
pipeline-model-definition:2.2175.v76a_fff0a_2618
pipeline-model-extensions:2.2175.v76a_fff0a_2618
pipeline-npm:204.v4dc4c2202625
pipeline-rest-api:2.34
pipeline-stage-step:305.ve96d0205c1c6
pipeline-stage-tags-metadata:2.2175.v76a_fff0a_2618
pipeline-stage-view:2.34
pipeline-utility-steps:2.16.2
plain-credentials:143.v1b_df8b_d3b_e48
plugin-util-api:3.8.0
prism-api:1.29.0-10
prometheus:2.5.1
publish-over:0.22
pubsub-light:1.18
resource-disposer:0.23
role-strategy:689.v731678c3e0eb_
scm-api:683.vb_16722fb_b_80b_
script-security:1321.va_73c0795b_923
shelve-project-plugin:3.2
slack:684.v833089650554
snakeyaml-api:2.2-111.vc6598e30cc65
sse-gateway:1.26
ssh-agent:346.vda_a_c4f2c8e50
ssh-credentials:308.ve4497b_ccd8f4
ssh-slaves:2.948.vb_8050d697fec
sshd:3.322.v159e91f6a_550
structs:337.v1b_04ea_4df7c8
terraform:1.0.10
testng-plugin:835.v51ed3da_fcc35
theme-manager:215.vc1ff18d67920
throttle-concurrents:2.14
token-macro:400.v35420b_922dcb_
trilead-api:2.133.vfb_8a_7b_9c5dd1
uno-choice:2.8.1
variant:60.v7290fc0eb_b_cd
workflow-aggregator:596.v8c21c963d92d
workflow-api:1291.v51fd2a_625da_7
workflow-basic-steps:1042.ve7b_140c4a_e0c
workflow-cps:3853.vb_a_490d892963
workflow-durable-task-step:1331.vc8c2fed35334
workflow-job:1400.v7fd111b_ec82f
workflow-multibranch:783.va_6eb_ef636fb_d
workflow-scm-step:415.v434365564324
workflow-step-api:657.v03b_e8115821b_
workflow-support:865.v43e78cc44e0d
ws-cleanup:0.45
What Operating System are you using (both controller, and any agents involved in the problem)?
Ubuntu 22.04.3
Linux 6.2.0-1018-aws
Reproduction steps
- Configure the Bitbucket Branch Source to authenticate with OAuth credentials
- Run any job that requires cloning the repository via Git.
Expected Results
Authentication should succeed and the Git repository checked out in the workspace.
Actual Results
The recommended git tool is: NONE
using credential *****
> git rev-parse --resolve-git-dir /var/lib/jenkins/workspace/*****/.git # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://*****@bitbucket.org/*****/*****.git # timeout=10
Fetching without tags
Fetching upstream changes from https://*****@bitbucket.org/*****/*****.git
> git --version # timeout=10
> git --version # 'git version 2.34.1'
using GIT_ASKPASS to set credentials Bitbucket OAuth Credentials
> git fetch --no-tags --force --progress -- https://*****@bitbucket.org/*****/*****.git +refs/heads/develop:refs/remotes/origin/develop # timeout=10
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from https://*****@bitbucket.org/*****/*****.git
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:999)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1241)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1305)
at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:129)
at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:97)
at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:84)
at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: hudson.plugins.git.GitException: Command "git fetch --no-tags --force --progress -- https://*****@bitbucket.org/*****/*****.git +refs/heads/develop:refs/remotes/origin/develop" returned status code 128:
stdout:
stderr: remote: Invalid credentials
fatal: Authentication failed for 'https://bitbucket.org/*****/*****.git/'
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2842)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2185)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:635)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:997)
... 11 more
Anything else?
Version 866.vdea_7dcd3008e still works as expected.
Are you interested in contributing a fix?
No response
The fix for #806 likely needs to be expanded upon to address this issue as well.
I'll check
@andrey-fomin We're on the latest version of the plugin and we're still experiencing this exact issue, however it's not on every build, seems to only be on random branch builds, sometimes rerunning fixes the issue, other times it doesn't. Any ideas?
@TomTucka @andrey-fomin We are experiencing exactly the same behavior, I can't pinpoint a pattern. Like token was expired? Some pipelines are running still successfully while some other are failing until i.e. restart Jenkins.
We are experiencing the same issue as well. Please can this be reopened?
@TomTucka @ElectricWeasel @hoo29
Please can you compare the clone links in successful and failed builds? Oauth token is directly encoded into clone link as user info and visible in console logs.
For me, the common characteristic of all the failed jobs is that they try to reuse a token that was already (successfully) used in a previous build. In older versions this wasn't a problem because a new token was generated every time (?).
We had issues trying to downgrade just the plugin so did a full Jenkins restore from a backup that included version 866.vdea_7dcd3008e which is stable for us. Unfortunately we have now lost the logs for the builds that were failing so cannot look.
Thanks @mfrodl. I've got the problem
We've been using oAuth for years and all of a sudden last night our builds started to fail with this error. I think I may have triggered it because I was starting to evaluate Bitbucket Runners and I think this all started after I created one. I have no clue why this would be but that's the only thing that changed as far as I can recall. I don't know if this kicked off a process on our account or what. I even upgraded us to the next tier to be able to use access tokens but using an access token fails with the same error.
Seems I can "refresh" the oAuth token if I go into the configuration for the multi-branch pipeline, select a different credential, then back to the correct credential, then save. This works for a short period of time.
Actually now that I think about it I worked on so much yesterday I completely forgot that one thing I did was upgrade our Jenkins plugins. I'm sure that's when this plugin was upgraded to 877.vb_b_d5243f6794 and our builds all broke. It doesn't look like the Jenkins UI will let me downgrade either 😠.
Another thing I noticed: when I switch to the new workspace access token credential, Jenkins pulls up the list of repos in the configuration page, then when I save it runs the multi-branch pipeline scan successfully (in other words, the new credential appears to work). However, when I then manually run a build it immediately fails. I can see in the logs that it's using some seemingly random string. I've checked and rechecked the credential and it's correct, but for some reason this plugin isn't using the correct credential in the build.
As far as using the oAuth credential, it appears that the only workaround so far is to go into the configuration, select a different cred, then back to the oAuth cred to refresh the token. That fixes builds for period of time anyway. Good thing it's the weekend.
Hello! I have just upgraded Jenkins to version 2.461.2 and the plugin to the latest version (888.v8e6d479a_1730) and we found the exact same issue described in this ticket. Could you please help?
We have not found any workaround to avoid this issue in the meantime
Update
I have found that the issue #862 is affecting the Bitbucket authentication, so I guess this issue was solved. Excuse me for the inconvenience