Deadlock on Jenkins controller while using lockable resources

Question

Deadlock on Jenkins controller while using lockable resources

Closed this issue a year ago · 13 comments

Jenkins and plugins versions report

Environment

Jenkins: 2.414.3
OS: Linux - 5.4.0-169-generic
Java: 17.0.9 (OpenJDK 64-Bit Server VM)
---
Parameterized-Remote-Trigger:3.2.0
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
authentication-tokens:1.53.v1c90fd9191a_b_
authorize-project:1.7.1
blueocean:1.27.9
blueocean-bitbucket-pipeline:1.27.9
blueocean-commons:1.27.9
blueocean-config:1.27.9
blueocean-core-js:1.27.9
blueocean-dashboard:1.27.9
blueocean-display-url:2.4.2
blueocean-events:1.27.9
blueocean-git-pipeline:1.27.9
blueocean-github-pipeline:1.27.9
blueocean-i18n:1.27.9
blueocean-jwt:1.27.9
blueocean-personalization:1.27.9
blueocean-pipeline-api-impl:1.27.9
blueocean-pipeline-editor:1.27.9
blueocean-pipeline-scm-api:1.27.9
blueocean-rest:1.27.9
blueocean-rest-impl:1.27.9
blueocean-web:1.27.9
bootstrap5-api:5.3.2-3
bouncycastle-api:2.30.1.77-225.v26ea_c9455fd9
branch-api:2.1128.v717130d4f816
build-user-vars-plugin:1.9
build-with-parameters:76.v9382db_f78962
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.0.2
cloudbees-bitbucket-branch-source:856.v04c46c86f911
cloudbees-folder:6.858.v898218f3609d
command-launcher:107.v773860566e2e
commons-lang3-api:3.13.0-62.v7d18e55f51e2
commons-text-api:1.11.0-95.v22a_d30ee5d36
config-file-provider:959.vcff671a_4518b_
copyartifact:722.v0662a_9b_e22a_c
credentials:1311.vcf0a_900b_37c2
credentials-binding:642.v737c34dea_6c2
data-tables-api:1.13.6-5
display-url-api:2.200.vb_9327d658781
docker-commons:439.va_3cb_0a_6a_fb_29
docker-workflow:572.v950f58993843
durable-task:543.v262f6a_803410
echarts-api:5.4.0-7
extended-read-permission:53.v6499940139e5
external-monitor-job:215.v2e88e894db_f8
favorite:2.208.v91d65b_7792a_c
flaky-test-handler:1.2.3
font-awesome-api:6.5.1-1
gerrit-trigger:2.39.3
gerrit-trigger-manual-patch-set:0.6
git:5.2.1
git-client:4.6.0
github:1.37.3.1
github-api:1.318-461.v7a_c09c9fa_d63
github-branch-source:1767.va_7d01ea_c7256
gson-api:2.10.1-15.v0d99f670e0a_7
h2-api:11.1.4.199-12.v9f4244395f7a_
handy-uri-templates-2-api:2.1.8-30.v7e777411b_148
htmlpublisher:1.32
http_request:1.18
influxdb:3.6
instance-identity:185.v303dc7c645f9
ionicons-api:56.v1b_1c8c49374e
jackson2-api:2.16.1-373.ve709c6871598
jakarta-activation-api:2.0.1-3
jakarta-mail-api:2.0.1-3
javadoc:243.vb_b_503b_b_45537
javax-activation-api:1.2.0-6
javax-mail-api:1.6.2-9
jaxb:2.3.9-1
jdk-tool:73.vddf737284550
jenkins-design-language:1.27.9
jjwt-api:0.11.5-77.v646c772fddb_0
joda-time-api:2.12.5-5.v5495a_235fedf
jquery3-api:3.7.1-1
jsch:0.2.16-86.v42e010d9484b_
json-path-api:2.8.0-21.v8b_7dc8b_1037b_
junit:1252.vfc2e5efa_294f
kubernetes:4054.v2da_8e2794884
kubernetes-client-api:6.8.1-224.vd388fca_4db_3b_
kubernetes-credentials:0.11
lockable-resources:1222.v3d55a_36d63e4
mailer:463.vedf8358e006b_
mapdb-api:1.0.9-28.vf251ce40855d
matrix-project:822.v01b_8c85d16d2
maven-plugin:3.23
metrics:4.2.18-442.v02e107157925
mina-sshd-api-common:2.11.0-86.v836f585d47fa_
mina-sshd-api-core:2.11.0-86.v836f585d47fa_
okhttp-api:4.11.0-157.v6852a_a_fa_ec11
pipeline-build-step:540.vb_e8849e1a_b_d8
pipeline-graph-analysis:202.va_d268e64deb_3
pipeline-groovy-lib:689.veec561a_dee13
pipeline-input-step:477.v339683a_8d55e
pipeline-maven:1368.vfb_8509d7b_869
pipeline-maven-api:1368.vfb_8509d7b_869
pipeline-milestone-step:111.v449306f708b_7
pipeline-model-api:2.2150.v4cfd8916915c
pipeline-model-definition:2.2150.v4cfd8916915c
pipeline-model-extensions:2.2150.v4cfd8916915c
pipeline-rest-api:2.34
pipeline-stage-step:305.ve96d0205c1c6
pipeline-stage-tags-metadata:2.2150.v4cfd8916915c
pipeline-stage-view:2.34
pipeline-utility-steps:2.16.0
plain-credentials:143.v1b_df8b_d3b_e48
plugin-util-api:3.8.0
pubsub-light:1.18
rebuild:330.v645b_7df10e2a_
resource-disposer:0.23
role-strategy:689.v731678c3e0eb_
saml:4.429.v9a_781a_61f1da_
scm-api:676.v886669a_199a_a_
scm-sync-configuration:0.0.10
script-security:1310.vf24a_dfce068b_
slack:684.v833089650554
snakeyaml-api:2.2-111.vc6598e30cc65
sonar:2.16.1
sse-gateway:1.26
ssh-agent:346.vda_a_c4f2c8e50
ssh-credentials:308.ve4497b_ccd8f4
sshd:3.312.v1c601b_c83b_0e
structs:325.vcb_307d2a_2782
subversion:2.17.3
timestamper:1.26
token-macro:400.v35420b_922dcb_
trilead-api:2.133.vfb_8a_7b_9c5dd1
variant:60.v7290fc0eb_b_cd
workflow-aggregator:596.v8c21c963d92d
workflow-api:1283.v99c10937efcb_
workflow-basic-steps:1042.ve7b_140c4a_e0c
workflow-cps:3832.vc43e04d6d68c
workflow-durable-task-step:1313.vcb_970b_d2a_fb_3
workflow-job:1385.vb_58b_86ea_fff1
workflow-multibranch:770.v1a_d0708dd1f6
workflow-scm-step:415.v434365564324
workflow-step-api:639.v6eca_cd8c04a_a_
workflow-support:865.v43e78cc44e0d
ws-cleanup:0.45

What Operating System are you using (both controller, and any agents involved in the problem)?

Debian 12 (in Docker)

Reproduction steps

Not sure how exactly it could be reproduced, but issue seems to be happening when we have ~10 pipelines running concurrently and utilizing lockable resources (not necessarily the same resources).

Expected Results

No deadlocks.

Actual Results

Running pipelines freeze and never finish. Triggering a thread dump of the Jenkins controller process outputs information that a deadlock was found.

Anything else?

Issue seems to be caused by the changes introduced in #586. The thread dump prints the following information (pipeline names are redacted):

Found one Java-level deadlock:
=============================
"Running CpsFlowExecution[Owner[tests.integration.0/34719:tests.integration.0 #34719]]":
  waiting to lock monitor 0x00007f19201bcf40 (object 0x000000060f6dbe10, a org.jenkins.plugins.lockableresources.LockableResourcesManager),
  which is held by "Running CpsFlowExecution[Owner[tests.integration.1/15001:tests.integration.1 #15001]]"

"Running CpsFlowExecution[Owner[tests.integration.1/15001:tests.integration.1 #15001]]":
  waiting to lock monitor 0x00007f1a3c3c4f40 (object 0x000000060d81d1d0, a java.lang.Object),
  which is held by "Running CpsFlowExecution[Owner[tests.integration.1/15002:tests.integration.1 #15002]]"

"Running CpsFlowExecution[Owner[tests.integration.1/15002:tests.integration.1 #15002]]":
  waiting to lock monitor 0x00007f19201bcf40 (object 0x000000060f6dbe10, a org.jenkins.plugins.lockableresources.LockableResourcesManager),
  which is held by "Running CpsFlowExecution[Owner[tests.integration.1/15001:tests.integration.1 #15001]]"

Java stack information for the threads listed above:
===================================================
"Running CpsFlowExecution[Owner[tests.integration.0/34719:tests.integration.0 #34719]]":
	at org.jenkins.plugins.lockableresources.LockableResourcesManager.unlockNames(LockableResourcesManager.java:567)
	- waiting to lock <0x000000060f6dbe10> (a org.jenkins.plugins.lockableresources.LockableResourcesManager)
	at org.jenkins.plugins.lockableresources.LockStepExecution$Callback.finished(LockStepExecution.java:187)
	at org.jenkinsci.plugins.workflow.steps.BodyExecutionCallback$TailCall.onSuccess(BodyExecutionCallback.java:118)
	at org.jenkinsci.plugins.workflow.cps.CpsBodyExecution$SuccessAdapter.receive(CpsBodyExecution.java:375)
	at com.cloudbees.groovy.cps.Outcome.resumeFrom(Outcome.java:70)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:149)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:146)
	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:136)
	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:275)
	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:146)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:187)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:423)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:331)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:295)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:97)
	at java.util.concurrent.FutureTask.run(java.base@17.0.9/FutureTask.java:264)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
	at jenkins.util.ErrorLoggingExecutorService.lambda$wrap$0(ErrorLoggingExecutorService.java:51)
	at jenkins.util.ErrorLoggingExecutorService$$Lambda$723/0x00007f19d0c75b88.run(Unknown Source)
	at java.util.concurrent.Executors$RunnableAdapter.call(java.base@17.0.9/Executors.java:539)
	at java.util.concurrent.FutureTask.run(java.base@17.0.9/FutureTask.java:264)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@17.0.9/ThreadPoolExecutor.java:1136)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@17.0.9/ThreadPoolExecutor.java:635)
	at java.lang.Thread.run(java.base@17.0.9/Thread.java:840)
"Running CpsFlowExecution[Owner[tests.integration.1/15001:tests.integration.1 #15001]]":
	at org.jenkins.plugins.lockableresources.LockableResourcesManager.getResourcesWithLabel(LockableResourcesManager.java:227)
	- waiting to lock <0x000000060d81d1d0> (a java.lang.Object)
	at org.jenkins.plugins.lockableresources.LockableResourcesManager.getResourcesWithLabel(LockableResourcesManager.java:216)
	at org.jenkins.plugins.lockableresources.LockableResourcesManager.checkResourcesAvailability(LockableResourcesManager.java:1109)
	- locked <0x000000060f6dbe10> (a org.jenkins.plugins.lockableresources.LockableResourcesManager)
	at org.jenkins.plugins.lockableresources.LockableResourcesManager.checkResourcesAvailability(LockableResourcesManager.java:1077)
	- locked <0x000000060f6dbe10> (a org.jenkins.plugins.lockableresources.LockableResourcesManager)
	at org.jenkins.plugins.lockableresources.LockableResourcesManager.getNextQueuedContext(LockableResourcesManager.java:707)
	at org.jenkins.plugins.lockableresources.LockableResourcesManager.getNextQueuedContext(LockableResourcesManager.java:682)
	at org.jenkins.plugins.lockableresources.LockableResourcesManager.unlockNames(LockableResourcesManager.java:577)
	- locked <0x000000060f6dbe10> (a org.jenkins.plugins.lockableresources.LockableResourcesManager)
	at org.jenkins.plugins.lockableresources.LockStepExecution$Callback.finished(LockStepExecution.java:187)
	at org.jenkinsci.plugins.workflow.steps.BodyExecutionCallback$TailCall.onSuccess(BodyExecutionCallback.java:118)
	at org.jenkinsci.plugins.workflow.cps.CpsBodyExecution$SuccessAdapter.receive(CpsBodyExecution.java:375)
	at com.cloudbees.groovy.cps.Outcome.resumeFrom(Outcome.java:70)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:149)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:146)
	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:136)
	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:275)
	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:146)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:187)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:423)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:331)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:295)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:97)
	at java.util.concurrent.FutureTask.run(java.base@17.0.9/FutureTask.java:264)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
	at jenkins.util.ErrorLoggingExecutorService.lambda$wrap$0(ErrorLoggingExecutorService.java:51)
	at jenkins.util.ErrorLoggingExecutorService$$Lambda$723/0x00007f19d0c75b88.run(Unknown Source)
	at java.util.concurrent.Executors$RunnableAdapter.call(java.base@17.0.9/Executors.java:539)
	at java.util.concurrent.FutureTask.run(java.base@17.0.9/FutureTask.java:264)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@17.0.9/ThreadPoolExecutor.java:1136)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@17.0.9/ThreadPoolExecutor.java:635)
	at java.lang.Thread.run(java.base@17.0.9/Thread.java:840)
"Running CpsFlowExecution[Owner[tests.integration.1/15002:tests.integration.1 #15002]]":
	at org.jenkins.plugins.lockableresources.LockableResourcesManager.fromName(LockableResourcesManager.java:270)
	- waiting to lock <0x000000060f6dbe10> (a org.jenkins.plugins.lockableresources.LockableResourcesManager)
	at org.jenkins.plugins.lockableresources.LockableResourcesManager.resourceExist(LockableResourcesManager.java:292)
	at org.jenkins.plugins.lockableresources.LockableResourcesManager.addResource(LockableResourcesManager.java:803)
	- locked <0x000000060d81d1d0> (a java.lang.Object)
	at org.jenkins.plugins.lockableresources.LockableResourcesManager.createResource(LockableResourcesManager.java:753)
	at org.jenkins.plugins.lockableresources.LockStepExecution.start(LockStepExecution.java:55)
	at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:323)
	at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:196)
	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:124)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:47)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
	at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20)
	at org.jenkinsci.plugins.workflow.cps.LoggingInvoker.methodCall(LoggingInvoker.java:105)
	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:90)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:116)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:85)
	at jdk.internal.reflect.GeneratedMethodAccessor289.invoke(Unknown Source)
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@17.0.9/DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(java.base@17.0.9/Method.java:568)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.ClosureBlock.eval(ClosureBlock.java:46)
	at com.cloudbees.groovy.cps.Next.step(Next.java:83)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:152)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:146)
	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:136)
	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:275)
	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:146)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:187)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:423)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:331)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:295)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:97)
	at java.util.concurrent.FutureTask.run(java.base@17.0.9/FutureTask.java:264)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
	at jenkins.util.ErrorLoggingExecutorService.lambda$wrap$0(ErrorLoggingExecutorService.java:51)
	at jenkins.util.ErrorLoggingExecutorService$$Lambda$723/0x00007f19d0c75b88.run(Unknown Source)
	at java.util.concurrent.Executors$RunnableAdapter.call(java.base@17.0.9/Executors.java:539)
	at java.util.concurrent.FutureTask.run(java.base@17.0.9/FutureTask.java:264)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@17.0.9/ThreadPoolExecutor.java:1136)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@17.0.9/ThreadPoolExecutor.java:635)
	at java.lang.Thread.run(java.base@17.0.9/Thread.java:840)

Are you interested in contributing a fix?

No response

Answer 1 · 2024-01-10T00:00:38.000Z

Duplicate #503

Answer 2 · 2024-01-10T00:03:53.000Z

Thx for reporting issues with helpful stack. This is because, the synchronized are used in wrong way.

Answer 3 · 2024-01-10T00:08:34.000Z

I need to merge my changes used by my own instance #558 into this master. But who will review 17 changed files? That the reason, why I split it in few smaller PRs.
Let me check how can be merged it effectively, because there are many other things done.

Answer 4 · 2024-01-10T11:33:58.000Z

In case it helps, we've experienced this quite a bit today after upgrading the plugin, so we've gone back to our previously installed version.

We've got a much more succinct deadlock with just two threads, where they are each waiting on each other, having apparently tried to take the same two locks in different orders (stacktraces truncated for clarity):

Thread 1:

        at org.jenkins.plugins.lockableresources.LockableResourcesManager.fromName(LockableResourcesManager.java:321)
	-  blocked on org.jenkins.plugins.lockableresources.LockableResourcesManager@c5ac193
	at org.jenkins.plugins.lockableresources.LockableResourcesManager.resourceExist(LockableResourcesManager.java:343)
	at org.jenkins.plugins.lockableresources.LockableResourcesManager.addResource(LockableResourcesManager.java:866)
	-  locked java.lang.Object@56c05066
	at org.jenkins.plugins.lockableresources.LockableResourcesManager.createResource(LockableResourcesManager.java:816)

Thread 2:

	at org.jenkins.plugins.lockableresources.LockableResourcesManager.save(LockableResourcesManager.java:1451)
	-  blocked on java.lang.Object@56c05066
	at org.jenkins.plugins.lockableresources.LockableResourcesManager.lock(LockableResourcesManager.java:567)
	-  locked org.jenkins.plugins.lockableresources.LockableResourcesManager@c5ac193

Hopefully that is useful in validating your proposed changes.

Answer 5 · 2024-01-10T13:53:14.000Z

We have also been experiencing this issue over the last couple of days, which we incorrectly attributed to GitHub API issues. We have been experiencing deadlocks when acquiring locks and on restarting the Jenkins controller we get concurrent modification exceptions (solved by removing the plugin and reinstalling currently), which should have been fixed by 1218.va_3dd45e2b_fa_7?.

I can see that there have been a few releases recently which seem to be related to this issue. Is there a version which is safe to roll back to?

Answer 6 · 2024-01-10T14:27:54.000Z

Official probably not.
We use version builded in #558
Probably good candidate will be
https://ci.jenkins.io/job/Plugins/job/lockable-resources-plugin/job/PR-558/40/artifact/

In that case you need to install plugin manually (just upload in the plugin page and reboot jenkins)

With that versions we runs over mounths without all the stucks and concurent modifications issues.

When somebody else try that, and confirm that it works, I will merge it as is. Otherwise it will takes time (currently I am only one maintaner here, and provide so deep and many changes without review, secondar test and discussions ist not the best idea)

Answer 7 · 2024-01-10T16:33:54.000Z

Thanks, I will update to that version overnight. Haven't seen the issue with https://github.com/jenkinsci/lockable-resources-plugin/releases/tag/1212.v1a_d65ecea_115 in the past 2 hours, previously had it occur twice in the last 6 hours with the latest release.

I had a brief look at the code (but unfortunately I would be unable to help with the workflow, I'm not experienced with Jenkins plugins) to determine if it was some kind of cache or database which could be cleared out to reduce the issue, as we have a lot of short lived locks, and it appears to be only storing locks which are in use. The https://Jenkins_URL/lockable-resources/ is rather helpful too

Answer 8 · 2024-01-15T09:23:51.000Z

Last Version where still works is
1222.v3d55a_36d63e4
we had Deadlocks for the next two actual Versions

Answer 9 · 2024-01-17T14:09:48.000Z

We have run into a very similar deadlock, where one thread runs freeResources (which is synchronized on the LockableResourcesManager instance), and calls save (which tries to also synchronized on syncResources), while another thread is running addResource and (while being synchronized on syncResources) calls fromName (which is synchronized on the LockableResourcesManager instance).

Answer 10 · 2024-01-17T16:19:35.000Z

I'm also experiencing this issue. Rolled back to 1218.va_3dd45e2b_fa_7 (which had been the last version we had been using)

Answer 11 · 2024-01-17T18:50:03.000Z

As well experiencing the same lockups since last update on Saturday. Went back to [1222.v3d55a_36d63e4] as stated above and its working again!

Answer 12 · 2024-01-17T22:28:25.000Z

Ok, finnaly i merged #558 and this issue shauld be fixed in the next release. I will keep it now opne, until somebody confirm, that the next release works as well. Thx

Answer 13 · 2024-01-19T04:42:48.000Z

looks good to me: lockable resources-view works while resource is locked with 1232.v512d6c434eb_d