s3_dir

This cookbook leverages the s3_file cookbook to recursively download all of the files in a specified S3 “directory.” It uses the et_fog cookbook cookbook to retrieve a listing of said “directories”.

Usage

Firstly, you will need to include et_fog in your recipe in order to install the Fog library in Chef, e.g.:

include_recipe 'et_fog'

`s3_dir` Resource

s3_dir '/local/path' do
  bucket            's3.bucket.name'
  dir               's3_directory'
  owner             'root'
  group             'root'
  mode              '0755'
  recursive         true
  region            'us-east-1'
  access_key_id     'ACCESS_KEY'
  secret_access_key 'SECRET_ACCESS_KEY'
end

name - The local path where the files should be downloaded to
bucket - The S3 bucket name you want to download from
dir - The directory within the bucket where the files will come from
owner - What you want the local file/directory owner to be set to. Defaults to root.
group - What you want the local file/directory group to be set to. Defaults to root.
mode - File permissions. Created directories will inherit these permissions plus executability. Defaults to '0755'
recursive - Whether parent directories of name should be created recursively
region - AWS Region. Defaults to us-east-1.
access_key_id - AWS Access Key ID
secret_access_key - AWS Secret Key

AWS Permissions

The following minimum AWS permissions are required for s3_dir to work:

"Action": [
  "s3:GetBucketLocation",
  "s3:GetObject",
  "s3:ListBucket"
],
"Resource": [
  "arn:aws:s3:::bucket-name",
  "arn:aws:s3:::bucket-name/*"
]

However it is recommended that you request a more permissive set of permissions so as to avoid unpredictable edge cases:

"Action": [
  "s3:Get*",,
  "s3:List*"
],
"Resource": [
  "arn:aws:s3:::bucket-name",
  "arn:aws:s3:::bucket-name/*"
]

Contributing

Fork the repository on Github
Create a named feature branch (i.e. add-new-recipe)
Write your change
Write tests for your change (if applicable)
Run the tests, ensuring they all pass
Submit a Pull Request

License and Authors

Author:: EverTrue, Inc. (devops@evertrue.com)