kerberos-io/machinery

Question S3 Upload Invaild Request

fpulch opened this issue · 4 comments

Is your feature request related to a problem? Please describe.
I'm trying to add my own s3 bucket in eu-central-1 to the kerberos machinery using the web ui.
It isn't as easy as i expected it, thus it would be usefull to get a more precise error message with a bit more details by default.

Describe the solution you'd like

It would be nice to see the error code even though it's not 200 OK in the keberos log, which is nicely visible in the ui.

It just displays

S3: file was not uploaded, something went wrong. Please check if you internet connectivity works.

When the response code 403 is returned the error message is more precise.

Additional context

After digging into some S3 log files it seemed like s3 returned the response code 307 temporary redirect.
This might have been caused by the bucket location not being in us-east-1

See s3-http-307-response

After some hours this problem resolved itself but another one occured.
In the S3 logs it seems like request is invaild now and the response code 400 is returned.

Maybe it might be related to a dash being in my bucket name.

Steps to reproduce:

  • Create an Amazon S3 Bucket
  • Tap in the bucketname, folder and credentials in the cloud tab
  • Enable S3 logging in order to debug

Overall i really like Kerberos so far.
Hope to get this issue resolved in order to use it on a larger scale.
Please keep up the great work :)

Sry, for creating a second issue about the s3 topic.
#159 seems to be referring to a similar problem.

Somehow it fixed itself. For now it seems to be working with an bucket in us-east-1 (Nord-Virginia).
Here's the link to the amazon docs where it's mentioned that the url s3.amazonaws.com which is used in the code only belongs to the us-east-1 AWS region

Using a S3 Bucket
AWS Endpoints

Some Uploads are successful. I'll have to let it run for an while to check if it uploads all events because the uploading error messages are still appearing in the kerberos machinery log.

But for now it's possible to close this issue although it's not fully resolved.
In the future it would be nice to set specific aws region or maybe switch the cloud provider.

Overall i'm quite surprised by this project. It runs nicely in docker, has the ability to push the images and videos to the cloud and is quite good documented.
I hope i can get my development skills further improved in order to contribute more in the future to awesome projects like this.

hey @fpulch, we use the S3 upload primary for our cloud application. However as you noticed you could also connect it to your own S3 bucket, no problem with that.

It's important to note that your data will passthrough one of our proxy applications, before it is directed to AWS. The reason why it didn't worked this afternoon is because one of the proxies was down. We have resolved the issue approx. 1 hour ago.

Sounds like a major SPOF for people running the "CE" edition :/