v0.5.0 - s3-endpoint breaking change

[grzegorzbialy]

Hi,

On version 0.5.0 when using AWS S3 and s3-endpoint (or S3_ENDPOINT env) is not defined there are errors on every function (GET / POST file) like:

"[problem getting object: operation error S3: GetObject, failed to resolve service endpoint, endpoint rule error, Custom endpoint `` was not a valid URI]"

The fix is to set S3_ENDPOINT to https://s3.[region].amazonaws.com along with S3_BUCKET, S3_REGION, etc.
Note that S3_ENDPOINT wasn't required in 0.4.1 and earlier. I think it should be noted as breaking change in changelog.

[dbarrosop]

Please, could you describe how to reproduce?

[grzegorzbialy]

Steps to reproduce:

1. Setup docker-compose.yaml like this:

version: "3.9"
services:
  storage:
    image: nhost/hasura-storage:0.5.0
    depends_on:
      - graphql-engine
    restart: always
    ports:
      - "8000:8000"
    environment:
      PUBLIC_URL: http://localhost:${PROXY_PORT:-1337}
      DEBUG: "true"
      HASURA_METADATA: 1
      HASURA_ENDPOINT: http://graphql-engine:8080/v1
      HASURA_GRAPHQL_ADMIN_SECRET: ${HASURA_GRAPHQL_ADMIN_SECRET}
      S3_ACCESS_KEY: ${S3_ACCESS_KEY}
      S3_SECRET_KEY: ${S3_SECRET_KEY}
      S3_BUCKET: ${S3_BUCKET}
      S3_REGION: ${S3_REGION}
      POSTGRES_MIGRATIONS: 1
      POSTGRES_MIGRATIONS_SOURCE: ${DB_URL}
    command: serve

2. Try to upload file, POST to /files/ and get response:

{
  "error": {
    "message": "an internal server error occurred"
  }
}

Error log:
time="2023-11-22T14:53:22Z" level=error msg="call completed with some errors" client_ip=10.0.10.77 errors="[problem processing request: problem uploading file to storage: problem putting object: operation error S3: PutObject, failed to resolve service endpoint, endpoint rule error, Custom endpoint `` was not a valid URI]" latency_time=2.88124317s method=POST status_code=500 url=/v1/storage/files

4. Same goes to open file, /files/[uuid]:

level=error msg="call completed with some errors" client_ip=10.0.10.77 errors="[problem getting object: operation error S3: GetObject, failed to resolve service endpoint, endpoint rule error, Custom endpoint `` was not a valid URI]" latency_time=11.225949ms method=GET status_code=500 url=/v1/storage/files/2c9f63e2-af92-4813-92ea-a75a3db9ff04 time="2023-11-22T14:56:04Z" level=error msg="call completed with some errors" client_ip=10.0.10.77 errors="[problem getting object: operation error S3: GetObject, failed to resolve service endpoint, endpoint rule error, Custom endpoint `` was not a valid URI]" latency_time=11.225949ms method=GET status_code=500 url=/v1/storage/files/2c9f63e2-af92-4813-92ea-a75a3db9ff04

Solution:

Add environmental variable S3_ENDPOINT. In above example docker-compose.yaml should look like this:

version: "3.9"
services:
  storage:
    image: nhost/hasura-storage:0.5.0
    depends_on:
      - graphql-engine
    restart: always
    ports:
      - "8000:8000"
    environment:
      PUBLIC_URL: http://localhost:${PROXY_PORT:-1337}
      DEBUG: "true"
      HASURA_METADATA: 1
      HASURA_ENDPOINT: http://graphql-engine:8080/v1
      HASURA_GRAPHQL_ADMIN_SECRET: ${HASURA_GRAPHQL_ADMIN_SECRET}
      S3_ACCESS_KEY: ${S3_ACCESS_KEY}
      S3_SECRET_KEY: ${S3_SECRET_KEY}
      S3_BUCKET: ${S3_BUCKET}
      S3_REGION: ${S3_REGION}
      S3_ENDPOINT: https://s3.${S3_REGION}.amazonaws.com
      POSTGRES_MIGRATIONS: 1
      POSTGRES_MIGRATIONS_SOURCE: ${DB_URL}
    command: serve

[dbarrosop]

Added a note to the release notes, thanks for reporting.