g10k does not deploy module correctly when -retrygitcommands is set
Closed this issue · 2 comments
We have a big environment with 380 modules.
When deploying a new environment with -maxworker 2 -maxextractworker 2 -retrygitcommands
on 2 nodes I don't get consistent, neither satisfying results :
- on first node : 371 out of 380 have been deployed
- on second node : 367 :/
When digging, it seems that when we use -retrygitcommands
g10k will retry once, and then ignore silently if the second try fails too :
2019/06/06 10:27:13 Executing git --git-dir /var/cache/g10k/modules/git@git.company.net-PP-Puppet_common_dnsmasq.git remote update --prune took 0.27376s
WARN: git command failed: git --git-dir /var/cache/g10k/modules/git@git.company.net-PP-Puppet_common_dnsmasq.git remote update --prune deleting local cached repository and retrying...
2019/06/06 10:27:13 DEBUG purgeDir(): Trying to remove: /var/cache/g10k/modules/git@git.company.net-PP-Puppet_common_dnsmasq.git called from doMirrorOrUpdate, because git command failed, retrying
2019/06/06 10:27:13 DEBUG executeCommand(): Executing git clone --mirror git@git.company.net:PP-Puppet/common_dnsmasq.git /var/cache/g10k/modules/git@git.company.net-PP-Puppet_common_dnsmasq.git
2019/06/06 10:27:14 Executing git clone --mirror git@git.company.net:PP-Puppet/common_dnsmasq.git /var/cache/g10k/modules/git@git.company.net-PP-Puppet_common_dnsmasq.git took 0.88200s
WARN: git repository git@git.company.net:PP-Puppet/common_dnsmasq.git does not exist or is unreachable at this moment!
2019/06/06 10:27:22 DEBUG executeCommand(): Executing git --git-dir /var/cache/g10k/modules/git@git.company.net-PP-Puppet_common_dnsmasq.git rev-parse --verify 'production^{object}'
2019/06/06 10:27:22 Executing git --git-dir /var/cache/g10k/modules/git@git.company.net-PP-Puppet_common_dnsmasq.git rev-parse --verify 'production^{object}' took 0.00156s
But at the end the run does not raise any error :
Synced /etc/g10k.yaml with branch test_g10k_deploy with 313 git repositories and 68 Forge modules in 17.4s with git (6.9s sync, I/O 10.1s) and Forge (2.0s query+download, I/O 0.4s) using 2 resolv and 2 extract workers
but some modules are missing
So, checked the code, it seems the behavior is not directly related to the retrygitcommands option.
As I understand it, issue is that
Line 81 in d9d50df
doMirrorOrUpdate
So we have no way to detect that environment has deployed correctly (unless we grep in logs)
Please check if v0.6.1 works for you: https://github.com/xorpaul/g10k/releases/tag/v0.6.1