allenai/dolma

Fix Actions to skip S3 tests from external contributors

Closed this issue · 5 comments

soldni commented

This would make the failures for #23 and #24 go away.

Having external collaborators hit the test bucket is a security risk, so those tests should be skipped.

chris-ha458 commented

This would be helpful!
Some reproducible and accessible data, even if it requires some work on the external collaborators part, could be a reasonable alternative as well.

Also, If S3 access is crucial even for outside cotnributors there are some world-readable-free aws buckets such as PMC Open Access (OA) Subset on AWS s3://pmc-oa-opendata
I am not sure if it is acceptable access to use it for this purpose. (If I had to guess it would be no)

soldni commented
chris-ha458 commented

Also is this issue that is blocking the assessment and merging of those two PRs at this time?
I am planning further commits upon those changes, if they cannot be integrated before this is addressed, I would rather assist in developing or testing this issue to help move things along.

soldni commented

@chris-ha458 I think the fixed I just merged in main should do it. if you have time to rebase, it would be great!

  • 👍1
soldni commented

ok looks like it worked for #24 ! closing this issue.