Sitespeed.io compatibility to send html report to Ceph S3 Storage ?
Closed this issue · 18 comments
Your question
Hello,
We use a self-managed Ceph S3 cluster inside our company, so we want to store the sitespeed HTML reports there.
Unlike Amazon S3 there is no region configured for the bucket, so I tried to set --s3.region with an empty string ("").
S3 parameters in sitespeed.io command : --s3.endpoint "https://a-radosgw-endpoint" --s3.bucketname "s3://bucket-name" --s3.key "" --s3.secret "***" --s3.region ""
And I have this error :
ERROR: [sitespeedio.plugin.s3] Could not upload to S3 SignatureDoesNotMatch: null at Request.extractError (/usr/src/app/node_modules/aws-sdk/lib/services/s3.js:711:35) at Request.callListeners (/usr/src/app/node_modules/aws-sdk/lib/sequential_executor.js:106:20) at Request.emit (/usr/src/app/node_modules/aws-sdk/lib/sequential_executor.js:78:10) at Request.emit (/usr/src/app/node_modules/aws-sdk/lib/request.js:686:14) at Request.transition (/usr/src/app/node_modules/aws-sdk/lib/request.js:22:10) at AcceptorStateMachine.runTo (/usr/src/app/node_modules/aws-sdk/lib/state_machine.js:14:12) at /usr/src/app/node_modules/aws-sdk/lib/state_machine.js:26:10 at Request.<anonymous> (/usr/src/app/node_modules/aws-sdk/lib/request.js:38:9) at Request.<anonymous> (/usr/src/app/node_modules/aws-sdk/lib/request.js:688:12) at Request.callListeners (/usr/src/app/node_modules/aws-sdk/lib/sequential_executor.js:116:18) at Request.emit (/usr/src/app/node_modules/aws-sdk/lib/sequential_executor.js:78:10) at Request.emit (/usr/src/app/node_modules/aws-sdk/lib/request.js:686:14) at Request.transition (/usr/src/app/node_modules/aws-sdk/lib/request.js:22:10) at AcceptorStateMachine.runTo (/usr/src/app/node_modules/aws-sdk/lib/state_machine.js:14:12) at /usr/src/app/node_modules/aws-sdk/lib/state_machine.js:26:10 at Request.<anonymous> (/usr/src/app/node_modules/aws-sdk/lib/request.js:38:9) Errors while running: SignatureDoesNotMatch: null Error: Errors while running: SignatureDoesNotMatch: null at start (file:///usr/src/app/bin/sitespeed.js:178:15) at processTicksAndRejections (node:internal/process/task_queues:95:5) at runNextTicks (node:internal/process/task_queues:64:3) at process.processImmediate (node:internal/timers:449:9) at async file:///usr/src/app/bin/sitespeed.js:189:1
Using an other tool s3cmd everything works with the same configuration.
So, I wonder if there is an "issue" with Ceph S3 and the "aws-sdk" (v2) used by sitespeedio ?
Hi @valgurau thanks for creating the issue. I think this could be that we need to upgrade to v3? I did a test a year ago or something and if I remember there was a new parameter that we needed so it was not backward compatible so I didn't upgrade at that time. If I do some changes, are you able to test from a branch in the GitHub repo? If so I would first try to upgrade to latest v2 and you can try that, and then if that do not work, I can make a branch with v3 (I still need to upgrade that in the future).
Hi @soulgalore, thank you for responding so quickly. Yes I will be able to test from a branch.
Great! Can you try with https://github.com/sitespeedio/sitespeed.io/tree/aws-2.1582.0 ? That updates to latest v2. Hopefully tomorrow I can have branch with v3. I remember before the updates/upgrades doc for v3 wasn't super helpful but that was long time ago.
I checked the new code. In v2 you don't need to set region (it defaulted to one of the US regions, don't remember which). In v3 you need to set region, and that will be a breaking change for users. I can probably push a branch tonight or early tomorrow so you at least can try with v3.
In the meantime while trying with version 2 aws-sdk (current version used by the sitespeedio s3 plugin) I realized that there is an option to force the path-style for the bucket in the options --s3.options.s3ForcePathStyle=true and with that it works in my case (with Ceph S3). In addition I had an error in the bucket name, the use of the s3:// prefix also caused problems. And as you mentioned I don't need to set the region parameter.
So the right configuration that works for me: --s3.endpoint "https://a-radosgw-endpoint/" --s3.bucketname "bucket-name" --s3.key "xxxx" --s3.secret "xxxx" --s3.options.s3ForcePathStyle=true
I tested with the branch you created for the latest version 2 and it also works. Also I will try with the branch for version 3 of the aws sdk to see if it's ok.
Ok! Here's a version using v3: https://github.com/sitespeedio/sitespeed.io/tree/aws-s3-v3
I need to tune it but I could verify I could upload as long as I set the region.
Hi @soulgalore,
I tried with v3 and it seems that the parameters --s3.endpoint and --s3.options.forcePathStyle=true ( before in v2 it was called s3ForcePathStyle ) have no effect.
Hi @valgurau thank you for trying it out. Endpoint config was moved so I missed adding it. I updated the branch now so it can be set. For s3.options.X we pass on anything that is set on the options object. Did you find the name change in the documentation if so, can you share the link, I'm having a hard time navigating the documentation for AWS.
Aws documentation : https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/migrating/notable-changes/
v2: Whether to force path style URLs for S3 objects.
v3: Renamed to forcePathStyle
Ok cool, thank you. Were you able to retry with my endpoint fix? Hoping setting the endpoint fix forcePathStyle.
Yes, I retried yesterday, everything seems ok with the endpoint and the forcePathStyle option. But as you mentioned before there is a problem with the region on uploading files. Even if I set a region, it still says that the region is missing on each uploaded file.
Example:
ERROR: [sitespeedio.plugin.s3] Error uploading /sitespeed.io/sitespeed-result/www.sitespeed.io/{datetime}/chromedriver-1.log.gz to S3 bucket {bucket name}: Error: Region is missing at default (/usr/src/app/node_modules/@smithy/config-resolver/dist-cjs/index.js:117:11) at /usr/src/app/node_modules/@smithy/node-config-provider/dist-cjs/index.js:72:104 at /usr/src/app/node_modules/@smithy/property-provider/dist-cjs/index.js:79:33 at async coalesceProvider (/usr/src/app/node_modules/@smithy/property-provider/dist-cjs/index.js:106:18) at async /usr/src/app/node_modules/@smithy/property-provider/dist-cjs/index.js:117:20 at async region (/usr/src/app/node_modules/@smithy/config-resolver/dist-cjs/index.js:142:30) at async /usr/src/app/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:89:28 at async /usr/src/app/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:120:14 at async /usr/src/app/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22 at async uploadFile (file:///usr/src/app/lib/plugins/s3/index.js:33:5)
Ok, so for Ceph S3 using AWS v3 it do not work if you set the region to an empty string (like in the example here)?
Hello @soulgalore , with the release of the latest major version 34 of sitespeed.io, I noticed that the migration to the aws sdk v3 is effective.
So, I ran tests with this version using the default value for the ‘region’ field, as suggested in the changelog. However, I believe there might be something missing regarding the ‘s3.options.x’ field passed to the S3 plugin. Currently, it seems that no additional options are being considered.
To check that, I took the code from the latest sitespeedio version and modified the instantiation of the S3 client by adding the extra options alongside the region like this:
const s3ClientOptions = { region: this.s3Options.region, ...this.s3Options.options };
After this change, uploading files to S3 Ceph with the option --s3.options.forcePathStyle (and the default region set to us-east-1) works very well for me.
Could you verify if the s3.options are really missing on your end ?
I think I missed that @valgurau sorry, I can roll out a fix later today. The additional params are passed on to the parameters for the put should be for the client too.
When I tested with S3 and Minio it worked out of the box, do you know what's different between the Ceph implementation and Minio (I haven't used Ceph)?
Ok thank you.
I'm not very familiar with Ceph / S3 in general either, but one constraint that I have compared to other S3 storages is that the URL to access a bucket with Ceph requires the precision of the bucket after the endpoint url like https://my-endpoint-url.com/my-bucket-name, while on other solutions the bucket make part of the endpoint url like https://my-bucket-name.my-endpoint-url.com/. And so to be able to manage this particular case I need to specify the option forcePathStyle=true
Ok. I just pushed 34.0.3, hopefully that fixes the problem.
Hello,
Tested with 34.1.0 and it works! 🎉
So, this "issue" can be closed.
Thank you
Great thank you @valgurau for the feedback!