Bladrak/capistrano-rsync

Bypass clone stage?

Closed this issue · 35 comments

csdougliss commented

I use jenkins to deploy and it fetches the repo already down to /var/lib/jenkins/workspace/my git repo/

Is there anyway to to bypass clone stage and use the files already there?

Bladrak commented

You might override the rsync:create_stage and rsync:stage tasks, it should allow you to remove the git operations

csdougliss commented

@Bladrak Could you make this a config option?

Bladrak commented

We could, but I'm not sure this is mandatory, given you can override those tasks quite easily. However, we could add some documentation to explain how to do it. Would you mind trying this solution, so we can proof-test it before documenting it?

csdougliss commented

@Bladrak yes! do you have some instructions?

Bladrak commented

You could try putting the following code in your deploy.rb:

namespace :rsync do
  task :create_stage do
    next
  end
  task :stage do
    next
  end
end

And set the :rsync_stage option to your git directory.

csdougliss commented

@Bladrak can the rsync_stage option be set to relative e.g. ./

Bladrak commented

Yes, it can :)

csdougliss commented

@Bladrak it worked! how can I omit .git and .idea files from being uploaded?

Bladrak commented

set :rsync_options, %w[--recursive --delete --delete-excluded --exclude .git* --exclude .idea*]

Also, see https://github.com/Bladrak/capistrano-rsync#exclude-files-from-being-deployed :)

csdougliss commented

@Bladrak works thank you 👍

csdougliss commented

One more for you:

      01 rsync --recursive --delete --delete-excluded --exclude .git* --exclude .idea* . x@xx:/var/www/vhosts/xx/shared/deploy
xx@xx's password: 
      01 skipping non-regular file "vendor/bin/composer"
      01 skipping non-regular file "vendor/bin/jsonlint"
      01 skipping non-regular file "vendor/bin/lessc"
      01 skipping non-regular file "vendor/bin/pdepend"
      01 skipping non-regular file "vendor/bin/php-cs-fixer"
      01 skipping non-regular file "vendor/bin/phpcs"
      01 skipping non-regular file "vendor/bin/phpmd"
      01 skipping non-regular file "vendor/bin/phpunit"
      01 skipping non-regular file "vendor/bin/static-review.php"
      01 skipping non-regular file "vendor/bin/validate-json"

Why is it skipping these files?

Bladrak commented

Because they are binaries. Check the manual page for rsync if you want to include them nonetheless

csdougliss commented

@Bladrak It seems to ignore the ssh key specified in deploy.rb to do the rsync and therefore fails on my server:

set :ssh_options, {:keys => ["#{ENV['HOME']}/.ssh/xxx"] }
Bladrak commented

I think you need to put this as an rsync option (you can specify the ssh command through the rsync options).

csdougliss commented

@Bladrak

It does:

      01 git fetch --quiet --all --prune --depth=1
    ✔ 01 jenkins@localhost 1.016s
      02 git reset --quiet --hard origin/master

Which is an issue as it would overwrite your branch with master.

Bladrak commented

Weird, it shouldn't if it's overridden. I'll check

Bladrak commented

Ok, try adding :

Rake::Task["rsync:create_stage"].clear
Rake::Task["rsync:stage"].clear

Before re-defining the tasks in your deploy.rb file.

csdougliss commented

@Bladrak

Still did it for me:

00:00 rsync:stage
      01 git fetch --quiet --all --prune --depth=1
    ✔ 01 jenkins@localhost 1.134s
      02 git reset --quiet --hard origin/master
    ✔ 02 jenkins@localhost 0.029s
Bladrak commented

Could you copy/paste your deploy.rb?

csdougliss commented

See below:

# config valid only for current version of Capistrano
lock '3.7.1'

set :application, 'xx'
set :repo_url, 'git@github.com:xx/xx.git'
set :deploy_to, '/var/www/vhosts/xx'
set :scm, :rsync

# bypass rsync to local
Rake::Task["rsync:create_stage"].clear
Rake::Task["rsync:stage"].clear

namespace :rsync do
  task :stage do
    next
  end
  task :stage do
    next
  end
end

set :rsync_stage, '.'
set :rsync_options, %w[-e 'ssh -i /var/lib/jenkins/.ssh/xx' --recursive --delete --delete-excluded --exclude .git* --exclude .idea*]

# Default value for :linked_files is [
#  									   'app/etc/env.php',
#									   'var/.setup_cronjob_status',
#									   'var/.update_cronjob_status',
#									   'pub/sitemap.xml'
# 									 ]
# set :linked_files, fetch(:linked_files, []).push('config/database.yml', 'config/secrets.yml')

# Default value for linked_dirs is [
#                                    'pub/media',
#                                    'var/backups',
#                                    'var/composer_home',
#                                    'var/importexport',
#                                    'var/import_history',
#                                    'var/log',
#                                    'var/session',
#                                    'var/tmp'
#                                  ]
# set :linked_dirs, fetch(:linked_dirs, []).push('log', 'tmp/pids', 'tmp/cache', 'tmp/sockets', 'public/system')

# Default value for keep_releases is 5
set :keep_releases, 3

set :ssh_options, {:keys => ["#{ENV['HOME']}/.ssh/xx"] }
Bladrak commented

Try clear_actions instead of clear and maybe add some debugging (puts) in the re-defined tasks?

csdougliss commented

@Bladrak Still did the same I'm afraid

Bladrak commented

Ok, then I guess we'll need the option you suggested. I won't have the time to do that in the near future, however if you can't wait you can always submit a pull request with it, I'll take time to review, merge and publish.

csdougliss commented

@Bladrak #27

I don't program in ruby - does it make sense to you? 👍

Bladrak commented

Seems good, thanks :)

Bladrak commented

I've pushed 1.4 to release this feature, tell me if that's ok for you?

csdougliss commented

@Bladrak That's great thank you. When will 1.4 be released?

Bladrak commented
csdougliss commented

@Bladrak Excellent thanks. Probably should have called it rsync_bypass_git_clone now that I think about it :)

csdougliss commented

Disregard, I had 1.3.8 in my Gemfile

csdougliss commented

@Bladrak

Something didn't work:

(Backtrace restricted to imported tasks)
cap aborted!
NameError: undefined local variable or method `bypass_git_clone' for main:Object

Tasks: TOP => rsync:create_release => rsync:release => rsync => rsync:stage_done => rsync:stage
(See full trace by running task with --trace)
The deploy has failed with an error: undefined local variable or method `bypass_git_clone' for main:Object


** DEPLOY FAILED
** Refer to log/capistrano.log for details. Here are the last 20 lines:

This is my deploy.rb addition:

# bypass rsync to local
set :bypass_git_clone, true
Bladrak commented

I've release 1.4.1 which should fix that

csdougliss commented

@Bladrak Thank you that worked.

However it is still doing it, even in 1.4.1?

00:00 rsync:stage
      01 git fetch --quiet --all --prune --depth=1
    ✔ 01 jenkins@localhost 1.362s
      02 git reset --quiet --hard origin/master
    ✔ 02 jenkins@localhost 0.595s
    ✔ 01 jenkins@localhost 1.361s
    ✔ 02 jenkins@localhost 0.314s
Bladrak commented

@craigcarnell sorry, I realized the if wasn't correct, fixed in 1.4.2

csdougliss commented

@Bladrak Thank you, it works 👍