R&D Setup ASW S3 buckets so the users can pay for the reading from it
khorolets opened this issue · 1 comments
Given that we can setup S3 in a way that the users will pay for their read usage
We want to enable it to anyone who runs or will need to run their own indexer in order to remove the pain from the process.
We want to check how to set it up and how to use it so we can document it and recommend to use NEAR Lake Framework
-
In a Bucket Permission settings we need to unblock public access (Block public access (bucket settings))
-
Create or edit bucket policy (allows to read objects and list objects in the specified bucket
{ "Version": "2012-10-17", "Statement": [ { "Sid": "PublicRead", "Effect": "Allow", "Principal": "*", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::near-lake-data-tesnet/*" }, { "Sid": "PublicList", "Effect": "Allow", "Principal": "*", "Action": "s3:ListBucket", "Resource": "arn:aws:s3:::near-lake-data-testnet" } ] }
-
Access control list (ACL):
Grant Authenticated users group (anyone with an AWS account) List and Read
After these changes, all requests to our S3 have to be done by authenticated users (any AWS user) with request_payer
provided. The requester will be charged for the queries to AWS by AWS
I'm going to apply the changes listed above to buckets:
-
near-lake-data-testnet
-
near-lake-data-mainnet
I need to update the README about it and close this after that.