xorpaul/g10k

Removed dirs are not purged after update

Closed this issue · 8 comments

cedriclaudrel commented

Hi,

We happily use g10k with -branch to deploy environments one branch at a time (after each push), and everything is fine besides one issue :

when a commit in the branch renames a directory ( == moves all files from a directory to a new one), deletes a directory, or remove all files from a directory (so the path doesn't exist anymore in git), this directory that was previously deployed is not removed and still exists next to the new one.

It breaks Puppet autoload module layout by holding the (old) module name in the namespace.

Workflow example :

  • git add mymodule/ && git commit && git push
  • g10k -config r10k.yaml -branch <mybranch>
ls -1 <mybranch_dir>/<module_dir>
mymodule
  • git mv mymodule mymodule2
  • git commit && git push
  • g10k -config r10k.yaml -branch <mybranch>
ls -1 <mybranch_dir>/<module_dir>
mymodule
mymodule2

In this example, <mybranch_dir>/<module_dir> should only contain mymodule2.

Thanks !

cedriclaudrel commented

This is with

deploy:
  purge_levels: [ 'deployment', 'environment', 'puppetfile' ]

In my r10k.yaml

xorpaul commented

Hmm, with the most recent g10k version https://github.com/xorpaul/g10k/releases/tag/v0.7.2

I can't even create a directory inside the control git repository under the <module_dir> with g10k purging it as unmodified content.

Example control git branch singe_git https://github.com/xorpaul/g10k-environment/tree/single_git

The <module_dir> is the default modules and in there I've created a folder called foo
Now when I run g10k it removes this folder completely:

$ g10k -config test.yaml -branch single_git -info
Need to sync /tmp/example/example_single_git/
Removing unmanaged path /tmp/example/example_single_git/modules/foo
Synced test.yaml with branch single_git with 2 git repositories and 0 Forge modules in 1.5s with git (0.7s sync, I/O 0.0s) and Forge (0.0s query+download, I/O 0.0s) using 50 resolv and 20 extract workers

Are you sure you're using v0.7.2 ?

cedriclaudrel commented

I'm using v0.7.0, I did not see a fix about that in more recent releases so I haven't tried them, but I will do now :)

cedriclaudrel commented

It's taking a bit more time than I expected, because -branch seems to have a different behavior since 143ea1e, and it breaks our deployment workflow starting from v0.7.1.

I'll keep you updated.

cedriclaudrel commented

I opened another issue describing the problems I encountered with g10 > v0.7.0 : #147

cedriclaudrel commented

OK, I've done some tests with v0.7.3, and I've managed to reproduce it :

Test g10k config :

---
:cachedir: '/tmp/g10k_cache'
purge_levels: ['deployment', 'puppetfile', 'environment']

sources:
  foo:
    remote: 'https://github.com/xorpaul/g10k-environment.git'
    basedir: /tmp/deploy
    prefix: true
  bar:
    remote: 'https://github.com/xorpaul/g10k-environment.git'
    basedir: /tmp/deploy
    prefix: true

/tmp/deploy is empty, as well as /tmp/g10k_cache.

Deploy :

# ./g10k -config /root/g10k_test.yaml -environment bar_single_git
Resolving Git modules (1/1)    0s [====================================================================] 100%
Synced /root/g10k_test.yaml with 2 git repositories and 0 Forge modules in 1.1s with git (0.6s sync, I/O 0.0s) and Forge (0.0s query+download, I/O 0.0s) using 50 resolv and 20 extract workers

Dummy test files and directories creation :

# touch /tmp/deploy/bar_single_git/file1
# touch /tmp/deploy/bar_single_git/mymodule2/file2
# mkdir /tmp/deploy/bar_single_git/mymodule2/dir1
# touch /tmp/deploy/bar_single_git/mymodule2/dir1/file3

Re-deployment :

./g10k -config /root/g10k_test.yaml -environment bar_single_git
Resolving Git modules (1/1)    0s [====================================================================] 100%
Synced /root/g10k_test.yaml with 2 git repositories and 0 Forge modules in 1.1s with git (0.6s sync, I/O 0.0s) and Forge (0.0s query+download, I/O 0.0s) using 50 resolv and 20 extract workers

Results :

# ls -l /tmp/deploy/bar_single_git/file1
ls: cannot access '/tmp/deploy/bar_single_git/file1': No such file or directory

OK for this one.

# ls -l /tmp/deploy/bar_single_git/mymodule2/file2
-rw-r--r-- 1 root root 0 Sep 13 16:00 /tmp/deploy/bar_single_git/mymodule2/file2
# ls -l /tmp/deploy/bar_single_git/mymodule2/dir1/file3 
-rw-r--r-- 1 root root 0 Sep 13 16:01 /tmp/deploy/bar_single_git/mymodule2/dir1/file3

But these 2 files that are under a git managed directory are not purged.

It seems that g10k only purges files at git rootdir, that is consistent with the behavior I had when I created the issue, some files under a module directory persisted.

It also seems that other directories in basedir not described in the config file are not purged, despite the purge_levels: ['deployment', 'puppetfile', 'environment'] line in my config file.

Thanks for your help :)

xorpaul commented

Please check out https://github.com/xorpaul/g10k/releases/tag/v0.8.2

I've added your example as a test.

cedriclaudrel commented

I'll try this one :)