Can't avoid send file that haven't change
Opened this issue · 2 comments
Jenkins and plugins versions report
Jenkins: 2.483
OS: Linux - 6.1.0-26-cloud-amd64
Java: 17.0.13 - Eclipse Adoptium (OpenJDK 64-Bit Server VM)
apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
apache-httpcomponents-client-5-api:5.4-124.v31e2987e48f4
asm-api:9.7.1-97.v4cc844130d97
authentication-tokens:1.119.v50285141b_7e1
bootstrap5-api:5.3.3-1
bouncycastle-api:2.30.1.78.1-248.ve27176eb_46cb_
branch-api:2.1197.vfa_d0c47c267d
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.2.1
cloud-stats:336.v788e4055508b_
cloudbees-folder:6.955.v81e2a_35c08d3
command-launcher:115.vd8b_301cc15d0
commons-lang3-api:3.17.0-84.vb_b_938040b_078
commons-text-api:1.12.0-129.v99a_50df237f7
conditional-buildstep:1.4.3
config-file-provider:980.v88956a_a_5d6a_d
credentials:1389.vd7a_b_f5fa_50a_2
credentials-binding:687.v619cb_15e923f
display-url-api:2.209.v582ed814ff2f
docker-commons:443.v921729d5611d
docker-java-api:3.4.0-94.v65ced49b_a_7d5
docker-plugin:1.7.0
docker-workflow:580.vc0c340686b_54
dtkit-api:3.0.2
durable-task:577.v2a_8a_4b_7c0247
echarts-api:5.5.1-4
eddsa-api:0.3.0-4.v84c6f0f4969e
envinject:2.919.v009a_a_1067cd0
envinject-api:1.199.v3ce31253ed13
font-awesome-api:6.6.0-2
git:5.6.0
git-client:6.1.0
gitlab-api:5.6.0-97.v6603a_83f8690
gitlab-branch-source:711.v41fb_c8b_2b_7a_e
gitlab-plugin:1.9.5
google-oauth-plugin:1.330.vf5e86021cb_ec
gson-api:2.11.0-85.v1f4e87273c33
handy-uri-templates-2-api:2.1.8-30.v7e777411b_148
instance-identity:201.vd2a_b_5a_468a_a_6
ionicons-api:74.v93d5eb_813d5f
jackson2-api:2.17.0-379.v02de8ec9f64c
jakarta-activation-api:2.1.3-1
jakarta-mail-api:2.1.3-1
javax-activation-api:1.2.0-7
javax-mail-api:1.6.2-10
jaxb:2.3.9-1
jdk-tool:80.v8a_dee33ed6f0
jersey2-api:2.44-151.v6df377fff741
joda-time-api:2.13.0-93.v9934da_29b_a_e9
jquery3-api:3.7.1-2
jsch:0.2.16-86.v42e010d9484b_
json-api:20240303-101.v7a_8666713110
json-path-api:2.9.0-118.v7f23ed82a_8b_8
junit:1307.vdd5b_2646279e
junit-realtime-test-reporter:149.v05a_d403e2f48
mailer:489.vd4b_25144138f
matrix-project:840.v812f627cb_578
mina-sshd-api-common:2.14.0-133.vcc091215a_358
mina-sshd-api-core:2.14.0-133.vcc091215a_358
nodejs:1.6.2
oauth-credentials:0.653.v14cf2088e950
people-view:1.2
pipeline-graph-analysis:216.vfd8b_ece330ca_
pipeline-groovy-lib:744.v5b_556ee7c253
pipeline-input-step:495.ve9c153f6067b_
pipeline-model-api:2.2214.vb_b_34b_2ea_9b_83
pipeline-model-definition:2.2214.vb_b_34b_2ea_9b_83
pipeline-model-extensions:2.2214.vb_b_34b_2ea_9b_83
pipeline-rest-api:2.34
pipeline-stage-step:312.v8cd10304c27a_
pipeline-stage-tags-metadata:2.2214.vb_b_34b_2ea_9b_83
plain-credentials:183.va_de8f1dd5a_2b_
plugin-util-api:5.1.0
publish-over:0.22
publish-over-ssh:1.25
resource-disposer:0.25
role-strategy:743.v142ea_b_d5f1d3
run-condition:1.7
scm-api:698.v8e3b_c788f0a_6
script-security:1367.vdf2fc45f229c
snakeyaml-api:2.3-123.v13484c65210a_
ssh-credentials:343.v884f71d78167
ssh-slaves:2.973.v0fa_8c0dea_f9f
sshd:3.330.vc866a_8389b_58
structs:338.v848422169819
thinBackup:2.1.1
token-macro:400.v35420b_922dcb_
trilead-api:2.147.vb_73cc728a_32e
variant:60.v7290fc0eb_b_cd
workflow-api:1336.vee415d95c521
workflow-basic-steps:1058.vcb_fc1e3a_21a_9
workflow-cps:3990.vd281dd77a_388
workflow-durable-task-step:1371.vb_7cec8f3b_95e
workflow-job:1460.v28178c1ef6e6
workflow-multibranch:795.ve0cb_1f45ca_9a_
workflow-scm-step:427.v4ca_6512e7df1
workflow-step-api:678.v3ee58b_469476
workflow-support:930.vf51d22b_ce488
ws-cleanup:0.48
What Operating System are you using (both controller, and any agents involved in the problem)?
debian 12
Reproduction steps
enable option ( avoid send not changed file )
Expected Results
send only file that have change
Actual Results
output is null
Anything else?
that feature is extremly important in our society, we have more than 3k file that his send each time and made loose arround 20 minute each time because we gote 7 dedicated server to update...
that issue is strong one, please fix it or explain us how to contrib to fix it because it's was no more acceptable to send the 3k file on 7 server each time
Are you interested in contributing a fix?
OFC YES
Any UP ?
How to Fix This Issue
Step 1:
In your Jenkins jobs, implement a way to create a tar.gz archive of your source files to upload (in my case, more than 3,000 files). Be sure to exclude any files that you don't want to include in the archive.
Step 2:
Use the (poorly optimized) plugin to send the tar.gz file to your servers.
Step 3 (Option 1):
Extract the tar.gz file into a separate folder (e.g., an uploaded folder). Then, use an internal rsync command to sync your source files. This approach optimizes file transfer by minimizing unnecessary file movements.
Step 3 (Option 2):
Alternatively, extract the tar.gz file directly into your source folder.
Using this method, you'll avoid sending 3,000 individual files via the plugin, which takes a very long time because of a bug that prevents skipping unchanged files.
Instead of sending 3,000 files, you'll only send one large file, which can be transferred at the full speed of your network.
In my case:
Before the change: Deployment took more than 15 minutes.
After the change: Deployment now takes only 3 minutes.
Thanks to the amazing contributor team that abandoned the project and left us to deal with these issues. We love you all—keep up the great work! ❤️