[question] s3 storage backend
Opened this issue · 7 comments
Hello.
Interesting, how i can use s3 storage backend in my container?
S3 buckets, always have a region-dependent url they can be accessed at (e.g. us standard ones can be accessed at https://s3.amazonaws.com/{bucket-name}/). Because of that, you can create an ACI with a name similar to that, host a html page indicating the url to pubkeys and so on, and it should work just like it would on any regular http server.
For example, I've hosted a docker2aci copy of redis on an s3 bucket I control which can be accessed via rkt fetch --debug s3.amazonaws.com/s.euank.com/acis/redis. The url https://s3.amazonaws.com/s.euank.com/acis/redis points to an http document (with an appropriate content-type) that has meta-headers pointing to a pubkey url and aci hosted in the same bucket. I modified the ACI's manifest to have a name of the above discovery url.
Other options include using AWS's "static site" option to have a shorter non-s3-specific domain or using cloudfront to do the same.
You might also be interested in rkt's ability to do AWS authentication for s3 urls even if the objects aren't public (see the type: aws auth on the config docs.
Happy to clarify further if you have a more specific question or run into particular problems.
Sorry, maybe i incorrect writed. I do not want repository storage for APPC. I want S3 bucket as a volume in App Container. It is really? Where i can read about this detailed?
If I understand your question correctly you want to use an S3 bucket as a volume to attach to a mount point in a running AppC container.
I think there isn't explicit support for this, but you should be able to locally mount an s3 bucket via s3fs and then use the --volume flag on rkt to point a volume at the location s3fs is mounted.
So something vaguely like:
> s3fs -o use_cache=/tmp/cache mydbbackup /s3mnt
> rkt run myimage --volume data,kind=host,source=/s3mnt
Disclaimer: I've never done this
Yes correctly. Hm.. Maybe possible add the functional into pod file?
What types of volumes are supported isn't defined in the spec. I think what you're asking for is support for an s3 kind in rkt's volume flag, so one could do something like:
> rkt run myimage --volume data,kind=s3,mydbbackup
If that's the case you should open an issue against https://github.com/coreos/rkt so we can keep track of this request.
Yeah; it's up to an ACE or orchestration system (like K8s volume plugins) to manage such a volume kind.
However, using S3 as a volume (implied, as a posix filesystem) is fraught with peril. There's no official way to make such a mount and S3 does not offer "read-after-write" consistency for update operations, nor any common way to represent various other posix features (e.g. some people make a block-store on top, some make metadata files for things like acls, etc).
The recommended way of using S3 is with the client libraries as an object store (which will work just fine in rkt today since everything is just network requests in that model), not as a filesystem volume (which, as above, is messy).
Because of the above, I'm not sure if an ACE should reasonably consider including such a volume type as a first class citizen.
The easiest shot at this might be a wrapper that creates a host-path that is then referenced by rkt.. similar to how volume plugins operate in Kubernetes today.
Closing this out based on the comments - not something we'll pursue in the spec.