Amazon S3 driver for VFS (Apache Commons Virtual File System)

Latest branch 4.x.x

How to add as dependency into your Maven build

<dependency>
    <groupId>com.github.abashev</groupId>
    <artifactId>vfs-s3</artifactId>
    <version>4.2.0</version>
</dependency>

How to add as dependency into your Gradle build

implementation 'com.github.abashev:vfs-s3:4.2.0'

URL format

s3://[[access-key:secret-key]:sign-region]@endpoint-url/folder-or-bucket/

Access key and Secret key could come from default AWS SDK chain (environment, container etc)
Sign-region is useful for custom implementations
If endpoint URL from known providers then we will try to get bucket name from host, if not able to do it then bucket is first path segment

Supported providers

Provider	URL
Amazon S3	https://aws.amazon.com/s3/
Yandex Object Storage	https://cloud.yandex.ru/services/storage
Mail.ru Cloud Storage	https://mcs.mail.ru/storage/

Sample groovy scripts - https://github.com/abashev/vfs-s3/tree/branch-4.x.x/samples

s3-copy able to copy between clouds, via http url or between different accounts

s3-copy s3://access1:secret1@s3-tests.storage.yandexcloud.net/javadoc.jar s3://access2:secret2@s3.eu-central-1.amazonaws.com/s3-tests-2/javadoc.jar

s3-copy https://oss.sonatype.org/some-name/120133-1-javadoc.jar s3://s3.eu-central-1.amazonaws.com/s3-tests-2/javadoc.jar

Sample Java Code for AWS S3

// Create a folder
FileSystemManager fsManager = VFS.getManager();
FileObject dir = fsManager.resolveFile("s3://simple-bucket.s3-eu-west-1.amazonaws.com/test-folder/");
dir.createFolder();

// Upload file to S3
FileObject dest = fsManager.resolveFile("s3://s3-eu-west-1.amazonaws.com/test-bucket/backup.zip");
FileObject src = fsManager.resolveFile(new File("/path/to/local/file.zip").getAbsolutePath());
dest.copyFrom(src, Selectors.SELECT_SELF);

Sample Java Code for Yandex Cloud https://cloud.yandex.ru/

// Upload file
FileObject dest = fsManager.resolveFile("s3://s3-tests.storage.yandexcloud.net/backup.zip");
FileObject src = fsManager.resolveFile(new File("/path/to/local/file.zip").getAbsolutePath());
dest.copyFrom(src, Selectors.SELECT_SELF);

Running tests

For running tests you need active credentials for AWS. You can specify them as

Shell environment properties

export AWS_ACCESS_KEY=AAAAAAA
export AWS_SECRET_KEY=SSSSSSS
export BASE_URL=s3://<full url like simple-bucket.s3-eu-west-1.amazonaws.com or s3-eu-west-1.amazonaws.com/test-bucket>