Version @spreadshirt/backstage-plugin-s3-viewer-backend@0.8.0 breaks the frontend
Closed this issue · 6 comments
Overview
The backend plugin in version 0.8.0 appears to be malfunctioning after the latest release. It can fetch the buckets as expected and shown in the logs, however, they are not being displayed in the frontend. Instead, a 404 error is displayed.
This error is not encountered in version 0.7.6, the same behavior of the backend finding the buckets and not displaying occurs in a production environment.
Environment
Locally with (fedora:39)
Production (Kubernetes:1.24.10 - @backstage/create-app@0.5.10)
Steps to Reproduce
app-config.yaml used
# Stub app config needed to run with yarn start
app:
title: backstage example app
baseUrl: http://localhost:3000
backend:
baseUrl: http://localhost:7007
listen:
port: 7007
permission:
enabled: true
s3:
bucketLocatorMethods:
- type: config
platforms:
- endpoint: http://127.0.0.1:8080
name: test
accessKeyId: 70VkRWd3IHDxEafKZFX9 # Has to match `CEPH_DEMO_ACCESS_KEY` when using docker compose
secretAccessKey: v0GerzwTw0cD2Dcq4m0aGeNzQVnpyzc0zW4Mc05A # Has to match `CEPH_DEMO_SECRET_KEY` when using docker compose
region: 'us-east-1'
allowedBuckets:
- platform: test
buckets:
- foobar
- git checkout @spreadshirt/backstage-plugin-s3-viewer-backend@0.8.0 / git checkout @spreadshirt/backstage-plugin-s3-viewer-backend@0.7.6
- Launch the bucket container / wait for it to be ready
- yarn start
Expected
The bucket should be seen in backstage frontend
Actual
Frontend returns a message with "Request failed for buckets/grouped, 404 Not Found"
Logs
Logs from the container with the bucket
Restarting RESTful API server...
c2752236-ef0d-44ef-92c1-f4efdb3f4190
changed ownership of '/etc/ceph/ceph.client.crash.keyring' from root:root to ceph:ceph
ownership of '/var/lib/ceph/crash/posted' retained as ceph:ceph
ownership of '/var/lib/ceph/crash' retained as ceph:ceph
nohup: ignoring input and appending output to 'nohup.out'
2024-04-05 16:31:00 /opt/ceph-container/bin/entrypoint.sh: SUCCESS
exec: PID 4078: spawning ceph --cluster ceph -w
exec: Waiting 4078 to quit
cluster:
id: 3915a179-4e6e-4511-9f25-b60b7b658423
health: HEALTH_OK
services:
mon: 1 daemons, quorum bac395a105be (age 2m)
mgr: bac395a105be(active, since 2m)
mds: 1/1 daemons up
osd: 2 osds: 2 up (since 2m), 2 in (since 2m); 1 remapped pgs
rbd-mirror: 1 daemon active (1 hosts)
rgw: 1 daemon active (1 hosts, 1 zones)
rgw-nfs: 1 daemon active (1 hosts, 1 zones)
data:
volumes: 1/1 healthy
pools: 9 pools, 233 pgs
objects: 238 objects, 579 KiB
usage: 53 MiB used, 200 GiB / 200 GiB avail
pgs: 233 active+clean
io:
recovery: 37 B/s, 1 objects/s
10.0.2.100 - - [05/Apr/2024 16:32:12] "GET /buckets.html HTTP/1.1" 200 -
10.0.2.100 - - [05/Apr/2024 16:32:12] "GET /css/bootstrap.min.css HTTP/1.1" 200 -
10.0.2.100 - - [05/Apr/2024 16:32:12] "GET /css/font-awesome.min.css HTTP/1.1" 200 -
10.0.2.100 - - [05/Apr/2024 16:32:12] "GET /css/style.css HTTP/1.1" 200 -
10.0.2.100 - - [05/Apr/2024 16:32:12] "GET /css/jquery.dataTables.min.css HTTP/1.1" 200 -
10.0.2.100 - - [05/Apr/2024 16:32:12] "GET /css/validationEngine.jquery.css HTTP/1.1" 200 -
10.0.2.100 - - [05/Apr/2024 16:32:12] "GET /image/ceph-nano-logo-horizontal.svg HTTP/1.1" 200 -
10.0.2.100 - - [05/Apr/2024 16:32:12] "GET /js/lib/require.js HTTP/1.1" 200 -
10.0.2.100 - - [05/Apr/2024 16:32:12] "GET /js/lib/require.config.js HTTP/1.1" 200 -
Logs from yarn start in branch @spreadshirt/backstage-plugin-s3-viewer-backend@0.8.0
xxx@fedora ~/Downloads/backstage-plugin-s3 ◈ @spreadshirt/backstage-plugin-s3-viewer-backend@0.8.0 yarn start
yarn run v1.22.22
$ concurrently "yarn start:backend" "yarn start:frontend"
$ yarn workspace @spreadshirt/backstage-plugin-s3-viewer-backend start
$ yarn workspace @spreadshirt/backstage-plugin-s3-viewer start
$ LEGACY_BACKEND_START=1 backstage-cli package start
$ backstage-cli package start
[0] Build succeeded
[1] Loaded config from app-config.yaml
[1] <i> [webpack-dev-server] Project is running at:
[1] <i> [webpack-dev-server] Loopback: http://localhost:3000/, http://[::1]:3000/
[1] <i> [webpack-dev-server] Content not from webpack is served from '/home/xxx/Downloads/backstage-plugin-s3/plugins/s3-viewer/public' directory
[1] <i> [webpack-dev-server] 404s will fallback to '/index.html'
[1] <i> [webpack-dev-middleware] wait until bundle finished: /
[0] Loaded config from app-config.yaml
[0] 2024-04-05T17:01:38.269Z s3-viewer-backend info Found 1 new secrets in config that will be redacted
[0] 2024-04-05T17:01:38.282Z s3-viewer-backend warn Generated a secret for service-to-service authentication: DEVELOPMENT USE ONLY.
[0] 2024-04-05T17:01:38.283Z s3-viewer-backend info Initializing S3 backend
[0] 2024-04-05T17:01:38.285Z s3-viewer-backend info Fetching S3 buckets...
[0] 2024-04-05T17:01:38.311Z s3-viewer-backend info CORS is disabled, allowing all origins
[0] 2024-04-05T17:01:38.325Z backstage info Listening on :7007
[0] 2024-04-05T17:01:38.443Z s3-viewer-backend info Fetched 1 S3 buckets
[1] webpack compiled successfully
[0] 2024-04-05T17:01:48.553Z backstage info ::ffff:127.0.0.1 - - [05/Apr/2024:17:01:48 +0000] "GET /api/s3-viewer/buckets/grouped? HTTP/1.1" 404 - "http://localhost:3000/" "Mozilla/5.0 (X11; Linux x86_64; rv:124.0) Gecko/20100101 Firefox/124.0" type=incomingRequest
[0] 2024-04-05T17:01:55.270Z backstage info ::ffff:127.0.0.1 - - [05/Apr/2024:17:01:55 +0000] "GET /api/s3-viewer/buckets/grouped? HTTP/1.1" 404 - "http://localhost:3000/" "Mozilla/5.0 (X11; Linux x86_64; rv:124.0) Gecko/20100101 Firefox/124.0" type=incomingRequest
Logs from yarn start in branch @spreadshirt/backstage-plugin-s3-viewer-backend@0.7.6
xxx@fedora ~/Downloads/backstage-plugin-s3 ◈ @spreadshirt/backstage-plugin-s3-viewer-backend@0.7.6 yarn start
yarn run v1.22.22
$ concurrently "yarn start:backend" "yarn start:frontend"
$ yarn workspace @spreadshirt/backstage-plugin-s3-viewer-backend start
$ yarn workspace @spreadshirt/backstage-plugin-s3-viewer start
$ LEGACY_BACKEND_START=1 backstage-cli package start
$ backstage-cli package start
[0] Build succeeded
[1] Loaded config from app-config.yaml
[1] <i> [webpack-dev-server] Project is running at:
[1] <i> [webpack-dev-server] Loopback: http://localhost:3000/, http://[::1]:3000/
[1] <i> [webpack-dev-server] Content not from webpack is served from '/home/xxx/Downloads/backstage-plugin-s3/plugins/s3-viewer/public' directory
[1] <i> [webpack-dev-server] 404s will fallback to '/index.html'
[1] <i> [webpack-dev-middleware] wait until bundle finished: /
[0] Loaded config from app-config.yaml
[0] 2024-04-05T16:33:30.677Z s3-viewer-backend info Found 1 new secrets in config that will be redacted
[0] 2024-04-05T16:33:30.688Z s3-viewer-backend warn Generated a secret for service-to-service authentication: DEVELOPMENT USE ONLY.
[0] 2024-04-05T16:33:30.689Z s3-viewer-backend info Initializing S3 backend
[0] 2024-04-05T16:33:30.690Z s3-viewer-backend info Fetching S3 buckets...
[0] 2024-04-05T16:33:30.704Z s3-viewer-backend info CORS is disabled, allowing all origins
[0] 2024-04-05T16:33:30.719Z backstage info Listening on :7007
[0] 2024-04-05T16:33:30.840Z s3-viewer-backend info Fetched 1 S3 buckets
[1] webpack compiled successfully
[0] 2024-04-05T16:33:42.037Z backstage info ::1 - - [05/Apr/2024:16:33:42 +0000] "POST /api/permission/authorize HTTP/1.1" 200 74 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)" type=incomingRequest
[0] 2024-04-05T16:33:42.044Z backstage info ::ffff:127.0.0.1 - - [05/Apr/2024:16:33:42 +0000] "GET /api/s3/buckets/grouped? HTTP/1.1" 304 - "http://localhost:3000/" "Mozilla/5.0 (X11; Linux x86_64; rv:124.0) Gecko/20100101 Firefox/124.0" type=incomingRequest
[0] 2024-04-05T16:33:59.251Z backstage info ::1 - - [05/Apr/2024:16:33:59 +0000] "POST /api/permission/authorize HTTP/1.1" 200 74 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)" type=incomingRequest
[0] 2024-04-05T16:33:59.256Z backstage info ::ffff:127.0.0.1 - - [05/Apr/2024:16:33:59 +0000] "GET /api/s3/buckets/grouped? HTTP/1.1" 304 - "http://localhost:3000/" "Mozilla/5.0 (X11; Linux x86_64; rv:124.0) Gecko/20100101 Firefox/124.0" type=incomingRequest
^C[0] 2024-04-05T16:35:06.221Z backstage info CTRL+C pressed; exiting.
[1] <i> [webpack-dev-server] Gracefully shutting down. To force exit, press ^C again. Please wait...
Aditional info
In version 0.7.6 in a production environment there seems to be another issue regarding permissions, could you provide a working "app-config.yaml" in a production environment.
Hi. Thanks for reporting!
Can you also provide the backstage version you're using? We are already aware that the latest release of Backstage if breaking this plugin due to all the permissions setup. This is currently in progress and under-review here: #90
However, the error described might be related to something else. I will test it first. However, I can say we are using the latest version for the backend plugin in our backstage instance in production and the listing is working fine. The main differences I can see is that you are using the config setup while we use the radosgw-admin setup, and that the permissionMiddleware is not enabled in your example.
And regarding the issue you're facing due to the permissions issue. If you're running the latest Backstage version (1.24 or 1.25), sadly you'll need to wait until we release the changes in the PR mentioned above. If you're running an older version, I believe you need to set the middleware properly. The documentation should explain how to do that (https://github.com/spreadshirt/backstage-plugin-s3/tree/main/plugins/s3-viewer-backend#permissions-setup)
Alright, I found the issue. But the problem should only be happening with the yarn start setup I think. The issue is that the standaloneServer is creating the wrong route: https://github.com/spreadshirt/backstage-plugin-s3/blob/main/plugins/s3-viewer-backend/src/service/standaloneServer.ts#L52. Instead, it should be "/s3-viewer". This is something that has already been tackled in the MR supporting the latest backstage version.
I was using the latest version of Backstage, I believe 1.25.0
"app@link:packages/app":
version "0.0.0"
dependencies:
"@backstage/app-defaults" "^1.5.3"
"@backstage/catalog-model" "^1.4.5"
"@backstage/cli" "^0.26.2"
"@backstage/core-app-api" "^1.12.3"
"@backstage/core-components" "^0.14.3"
"@backstage/core-plugin-api" "^1.9.1"
"@backstage/integration-react" "^1.1.25"
"@backstage/plugin-api-docs" "^0.11.3"
"@backstage/plugin-catalog" "^1.18.2"
"@backstage/plugin-catalog-common" "^1.0.22"
"@backstage/plugin-catalog-graph" "^0.4.3"
"@backstage/plugin-catalog-import" "^0.10.9"
"@backstage/plugin-catalog-react" "^1.11.2"
"@backstage/plugin-github-actions" "^0.6.14"
"@backstage/plugin-org" "^0.6.23"
"@backstage/plugin-permission-react" "^0.4.21"
"@backstage/plugin-scaffolder" "^1.19.2"
"@backstage/plugin-search" "^1.4.9"
"@backstage/plugin-search-react" "^1.7.9"
"@backstage/plugin-tech-radar" "^0.7.2"
"@backstage/plugin-techdocs" "^1.10.3"
"@backstage/plugin-techdocs-module-addons-contrib" "^1.1.8"
"@backstage/plugin-techdocs-react" "^1.2.2"
"@backstage/plugin-user-settings" "^0.8.4"
"@backstage/theme" "^0.5.2"
"@material-ui/core" "^4.12.2"
"@material-ui/icons" "^4.9.1"
"@spreadshirt/backstage-plugin-s3-viewer" "^0.5.0"
history "^5.0.0"
react "^18.0.2"
react-dom "^18.0.2"
react-router "^6.3.0"
react-router-dom "^6.3.0"
react-use "^17.2.4"
backstage created with 0.5.11 and the fix mentioned worked! thanks @ivangonzalezacuna
Alright. In that case, please bump the plugin to the latest version. We have just released it.
Also, please note that there are a couple possible breaking changes that you will need to apply in order to make it work. Basically those changes are affecting the permission setup: the middleware is not needed any longer and the way to set the cookie has changed a little. If you're using the new backend system all the other changes will be completely transparent for you, but you might need some more changes if you're using the old backend setup.
backstage created with 0.5.11 and the fix mentioned worked! thanks @ivangonzalezacuna
Awesome!
Alright, I found the issue. But the problem should only be happening with the
yarn startsetup I think. The issue is that the standaloneServer is creating the wrong route: https://github.com/spreadshirt/backstage-plugin-s3/blob/main/plugins/s3-viewer-backend/src/service/standaloneServer.ts#L52. Instead, it should be "/s3-viewer". This is something that has already been tackled in the MR supporting the latest backstage version.
Closing this as the solution solves the issue.