can I simply copy to s3 using the tool without achiving?

[ethan-riskiq]

What went wrong?

I have some large backups that I would like to copy to an s3 bucket using this tool, however it doesn't appear as tho the "Storage:S3" class has a means to specify a directory without it being archived/tar'd/compressed first. The s3 Sync method won't work either since some of the solr files I'm backing up are > 5GB. Is this by design or is there something I'm missing if I simply want a group of directories backed up into s3 without achiving first?

What steps did you follow?

Change the steps below to match the commands that you ran

backup perform --trigger solr_s3

  ## How is your copy of backup configured?
  Model.new(:solr_crawl_s3, 'Description for solr_crawl_s3') do

  #  archive :app_archive do |archive|
   #   archive.add '/export/backup/solr/'
    #end

     ##
   # Amazon Simple Storage Service [Storage]
    #
  store_with S3 do |s3|
   # AWS Credentials
    s3.access_key_id     = "REDACTED"
     s3.secret_access_key = "REDACTED"
   # Or, to use a IAM Profile:
   # s3.use_iam_profile = true

s3.region            = "us-west-1"
s3.bucket            = "bucketname"
s3.path              = "/"
s3.keep              = 5
s3.chunk_size = 10
# s3.keep              = Time.now - 2592000 # Remove all backups older than 1 month.
    end

    ##
    # Gzip [Compressor]
    #
    compress_with Gzip

  Logger.configure do
    logfile.enabled   = true
    logfile.log_path  = '/var/log/backup/'
    logfile.max_bytes = 500_000
  end

  notify_by Slack do |slack|
    slack.on_success = true
    slack.on_warning = true
    slack.on_failure = true

     The integration token
    slack.webhook_url = 'https://hooks.slack.com/services/REDACTED/REDACTED/REDACTED'
   end
  end

Tell us about the computer that runs the backup gem

• Operating system: Centos 6.9
• Ruby version: 2.3.1p112

[tombruijn]

Hi @ethan-riskiq,

It's currently not possible to upload entire directories directly to AWS S3 using the backup gem. The archiving/packaging tool is a core part of the backup pipeline.

We don't have any plans to support this in the near future, so you would probably have to write something custom using the AWS S3 ruby gem directly.

[pwelch]

Closing issue as answered. Please feel free to re-open if there are any more questions.