Problem Executing Query

Question

Problem Executing Query

FranciscoMendes10866 opened this issue 3 years ago · 13 comments

FranciscoMendes10866 commented 3 years ago

Hi!

I'm using this project for the first time, I followed this docker compose and I already have the container running. Migrations were done and the metadata was applied.

However I'm getting this error even after setting user permissions on storage.files table.

time="2022-04-28T08:28:37Z" level=error msg="call completed with some errors" client_ip=172.20.0.1 errors="[problem processing request: problem executing query: Message: field \"bucket\" not found in type: 'query_root', Locations: []]" latency_time=161.356167ms method=POST status_code=403 url=/v1/storage/files

In the http request I am sending the following headers:

x-hasura-admin-secret
authorization

I believe I'm failing to check permissions, what things should I be aware of?

Answer 1 · 2022-04-28T09:35:14.000Z

Which version of hasura-storage are you using? Make sure you are running 0.1.5 as there used to be a bug with permissions where you had to also give permissions to the bucket.

Answer 2 · 2022-04-28T09:41:24.000Z

@dbarrosop I'm using version 0.1.5. Do I have to set permissions on the storage.buckets? So that I can upload files?

Answer 3 · 2022-04-28T13:41:36.000Z

could you provide the exact docker-compose file and the steps to reproduce the issue?

Answer 4 · 2022-04-28T13:57:44.000Z

My docker compose file is this one.

The error happens even using an application like postman, where I send the x-hasura-admin-secret and authorization headers. With Content-Type multipart/form-data where I send the file[] (which in my case is an image) and the bucket_id with the value of default.

This on the endpoint http://localhost:8000/v1/storage/files.

Answer 5 · 2022-04-28T14:30:53.000Z

Why are you passing both x-hasura-admin-secret and authorization? You should pass only one of them, also, how are you generating the token for the authorization header?

I tried to replicate myself with your docker-compose and found it had a mistake on the minio definition, it should be:

  minio:
    container_name: minio
    image: minio/minio:RELEASE.2022-04-01T03-41-39Z
    restart: always
    environment:
      MINIO_ACCESS_KEY: 5a7bdb5f42c41e0622bf61d6e08d5537
      MINIO_SECRET_KEY: 9e1c40c65a615a5b52f52aeeaf549944ec53acb1dff4a0bf01fb58e969f915c8
    ports:
      - 9000:9000
      - 32765:32765
    expose:
      - 9000
      - 32765
    entrypoint: sh
    command: -c 'mkdir -p /data/default && /opt/bin/minio server --address 0.0.0.0:9000 --console-address 0.0.0.0:32765 /data'

Other than that, I could upload a file just fine without changing any permissions:

curl http://localhost:8000/v1/storage/files/ \
  -v \
  -H "Content-Type: multipart/form-data" \
  -H "x-hasura-admin-secret: NIWB24maLuDT5Iqa" \
  -F "file[]=@asd"
*   Trying 127.0.0.1:8000...
* Connected to localhost (127.0.0.1) port 8000 (#0)
> POST /v1/storage/files/ HTTP/1.1
> Host: localhost:8000
> User-Agent: curl/7.79.1
> Accept: */*
> x-hasura-admin-secret: NIWB24maLuDT5Iqa
> Content-Length: 47186117
> Content-Type: multipart/form-data; boundary=------------------------dfe65240f6f5d9e2
> Expect: 100-continue
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
* Mark bundle as not supporting multiuse
< HTTP/1.1 201 Created
< Content-Type: application/json; charset=utf-8
< Date: Thu, 28 Apr 2022 14:22:25 GMT
< Content-Length: 333
<
* Connection #0 to host localhost left intact
{"processedFiles":[{"id":"3ba4874c-7829-4a88-a67d-5dcdff68fbfa","name":"asd","size":47185920,"bucketId":"default","etag":"\"9737457d272bf7e8f1d10ebde39c95c9\"","createdAt":"2022-04-28T14:22:24.808558+00:00","updatedAt":"2022-04-28T14:22:25.05601+00:00","isUploaded":true,"mimeType":"application/octet-stream","uploadedByUserId":""}]}

so my bet is that the issue is that you are sending both headers and that, most likely, the authorization header is not correct, it should be generated using the JWT secrets and be of the form Authorization: Bearer this-is-my-generated-token. You have a small tool to generate tokens here.

$ ~/workspace/nhost/hasura-storage/build/dev/jwt-gen/jwt-gen.Darwin.arm64 -jwt-secret '{"type":"HS256", "key": "2C2FB4D61CD8C46218A83084FD49878C72FC0F2E4A8EC74C8B3CB42E1E1F4F18"}'
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NTExNTcwNzAsImh0dHBzOi8vaGFzdXJhLmlvL2p3dC9jbGFpbXMiOnsieC1oYXN1cmEtYWxsb3dlZC1yb2xlcyI6WyJhZG1pbiJdLCJ4LWhhc3VyYS1kZWZhdWx0LXJvbGUiOiJhZG1pbiIsIngtaGFzdXJhLXVzZXItaWQiOiJhYjViYTU4ZS05MzJhLTQwZGMtODdlOC03MzM5OTg3OTRlYzIiLCJ4LWhhc3VyYS11c2VyLWlzQW5vbnltb3VzIjoiZmFsc2UifSwiaWF0IjoxNjUxMTU2MTcwLCJpc3MiOiJoYXN1cmEtYXV0aCIsInN1YiI6ImFiNWJhNThlLTkzMmEtNDBkYy04N2U4LTczMzk5ODc5NGVjMiJ9.AzZ-6v61LfldDPCxpK7WJ2Dx_I8FlcMef_i_nOSangs%

$ curl http://localhost:8000/v1/storage/files/ \
  -v \
  -H "Content-Type: multipart/form-data" \
  -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NTExNTcwNzAsImh0dHBzOi8vaGFzdXJhLmlvL2p3dC9jbGFpbXMiOnsieC1oYXN1cmEtYWxsb3dlZC1yb2xlcyI6WyJhZG1pbiJdLCJ4LWhhc3VyYS1kZWZhdWx0LXJvbGUiOiJhZG1pbiIsIngtaGFzdXJhLXVzZXItaWQiOiJhYjViYTU4ZS05MzJhLTQwZGMtODdlOC03MzM5OTg3OTRlYzIiLCJ4LWhhc3VyYS11c2VyLWlzQW5vbnltb3VzIjoiZmFsc2UifSwiaWF0IjoxNjUxMTU2MTcwLCJpc3MiOiJoYXN1cmEtYXV0aCIsInN1YiI6ImFiNWJhNThlLTkzMmEtNDBkYy04N2U4LTczMzk5ODc5NGVjMiJ9.AzZ-6v61LfldDPCxpK7WJ2Dx_I8FlcMef_i_nOSangs" \
  -F "file[]=@asd"
*   Trying 127.0.0.1:8000...
* Connected to localhost (127.0.0.1) port 8000 (#0)
> POST /v1/storage/files/ HTTP/1.1
> Host: localhost:8000
> User-Agent: curl/7.79.1
> Accept: */*
> Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NTExNTcwNzAsImh0dHBzOi8vaGFzdXJhLmlvL2p3dC9jbGFpbXMiOnsieC1oYXN1cmEtYWxsb3dlZC1yb2xlcyI6WyJhZG1pbiJdLCJ4LWhhc3VyYS1kZWZhdWx0LXJvbGUiOiJhZG1pbiIsIngtaGFzdXJhLXVzZXItaWQiOiJhYjViYTU4ZS05MzJhLTQwZGMtODdlOC03MzM5OTg3OTRlYzIiLCJ4LWhhc3VyYS11c2VyLWlzQW5vbnltb3VzIjoiZmFsc2UifSwiaWF0IjoxNjUxMTU2MTcwLCJpc3MiOiJoYXN1cmEtYXV0aCIsInN1YiI6ImFiNWJhNThlLTkzMmEtNDBkYy04N2U4LTczMzk5ODc5NGVjMiJ9.AzZ-6v61LfldDPCxpK7WJ2Dx_I8FlcMef_i_nOSangs
> Content-Length: 47186117
> Content-Type: multipart/form-data; boundary=------------------------2924533bb8f14b91
> Expect: 100-continue
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
* Mark bundle as not supporting multiuse
< HTTP/1.1 201 Created
< Content-Type: application/json; charset=utf-8
< Date: Thu, 28 Apr 2022 14:29:58 GMT
< Content-Length: 334
<
* Connection #0 to host localhost left intact
{"processedFiles":[{"id":"d390763f-5f8d-4276-9f9b-c6623b852b86","name":"asd","size":47185920,"bucketId":"default","etag":"\"9737457d272bf7e8f1d10ebde39c95c9\"","createdAt":"2022-04-28T14:29:58.070801+00:00","updatedAt":"2022-04-28T14:29:58.326133+00:00","isUploaded":true,"mimeType":"application/octet-stream","uploadedByUserId":""}]}%

Answer 6 · 2022-04-28T15:04:00.000Z

I am sending in the Authorization Header the Access Token that is generated by hasura_auth at user login (even with the refresh token implemented).

Even sending only one of the headers (tried with x-hasura-admin-secret), I still have the same log, in the app I get the following response (with or without permissions).

{
  "error": {
    "message": "you are not authorized"
  }
}

Answer 7 · 2022-04-28T15:15:40.000Z

Ok, I am afraid I couldn't reproduce so it is hard to know what's going on. Could you start by trying the following command:

curl http://localhost:8000/v1/storage/files/ \
  -v \
  -H "Content-Type: multipart/form-data" \
  -H "x-hasura-admin-secret: NIWB24maLuDT5Iqa" \
  -F "file[]=@/path/to/file/to/upload"

Make sure that /path/to/file/to/upload points to a valid file. Then paste here the output of the command the logs generated by hasura-storage, hasura and minio (if any), please.

Answer 8 · 2022-04-28T15:24:53.000Z

This is the request:

curl http://localhost:8000/v1/storage/files/ \                                       
  -v \
  -H "Content-Type: multipart/form-data" \
  -H "x-hasura-admin-secret: NIWB24maLuDT5Iqa" \
  -F "file[]=@i3.png"
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8000 (#0)
> POST /v1/storage/files/ HTTP/1.1
> Host: localhost:8000
> User-Agent: curl/7.64.1
> Accept: */*
> x-hasura-admin-secret: NIWB24maLuDT5Iqa
> Content-Length: 13709629
> Content-Type: multipart/form-data; boundary=------------------------f770d15b0880ebfc
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 403 Forbidden
< Content-Type: application/json; charset=utf-8
< Date: Thu, 28 Apr 2022 15:24:29 GMT
< Content-Length: 46
<
* Connection #0 to host localhost left intact
{"error":{"message":"you are not authorized"}}* Closing connection 0

These are the hasura storage logs:

[GIN-debug] Listening and serving HTTP on :8000

time="2022-04-28T15:24:29Z" level=error msg="call completed with some errors" client_ip=172.23.0.1 errors="[problem processing request: problem executing query: Message: field \"bucket\" not found in type: 'query_root', Locations: []]" latency_time=440.803833ms method=POST status_code=403 url=/v1/storage/files/

These are the hasura logs:

{"status":200,"http_version":"HTTP/1.1","url":"/v2/query","ip":"172.23.0.1","method":"POST","content_encoding":"gzip"}}}

{"type":"http-log","timestamp":"2022-04-28T15:07:47.832+0000","level":"info","detail":{"operation":{"query_execution_time":0.171230792,"user_vars":{"x-hasura-role":"admin"},"request_id":"7ad44547-d5e4-4cf1-bdce-e6485f6d80ba","response_size":4208,"request_mode":"non-graphql","request_read_time":3.209e-6},"request_id":"7ad44547-d5e4-4cf1-bdce-e6485f6d80ba","http_info":{"status":200,"http_version":"HTTP/1.1","url":"/v2/query","ip":"172.23.0.1","method":"POST","content_encoding":"gzip"}}}

{"type":"http-log","timestamp":"2022-04-28T15:19:25.869+0000","level":"error","detail":{"operation":{"user_vars":{"x-hasura-role":"admin"},"error":{"path":"$.selectionSet.bucket","error":"field \"bucket\" not found in type: 'query_root'","code":"validation-failed"},"request_id":"c5378ae3-9924-4355-8a41-d7dffb95d1c7","response_size":118,"request_mode":"error"},"request_id":"c5378ae3-9924-4355-8a41-d7dffb95d1c7","http_info":{"status":200,"http_version":"HTTP/1.1","url":"/v1/graphql","ip":"172.23.0.4","method":"POST","content_encoding":null}}}

The minio doesn't have any logs.

Thank you very much in advance for helping me 🙏

Answer 9 · 2022-04-28T16:34:59.000Z

Ok, so the authentication is actually working, in the hasura logs you can see {"x-hasura-role":"admin"} while it actually complains that "error":{"path":"$.selectionSet.bucket","error":"field \"bucket\" not found in type: 'query_root'","code":"validation-failed"} so I think your metadata may be corrupted. Would it be possible to start from scratch? Try deleting all the volumes and starting over and see if it works with a clean slate.

Answer 10 · 2022-04-28T16:48:24.000Z

Yes, if I clean everything and start from scratch I can upload files 🙌 However I lose all my metadata 😢

Answer 11 · 2022-04-28T17:23:39.000Z

I understand, just wanted to make sure you weren't doing something wrong or that there was some bug somewhere. I suspect that somehow your metadata got corrupted and needs to be fixed. It is hard to know what the problem is without looking into it but I'd try comparing the databases and relation inside default->storage (files and buckets tables) are the same in the working and non-working setups. If you send screenshots of the "modify" screens for both files and buckets I can try to give you a hand

Answer 12 · 2022-04-29T08:08:00.000Z

@dbarrosop I just now compared the fields that exist in both tables in the "Modify" tab and the only thing that changes is the Custom GraphQL Root Fields. In the setup that is working (if you make a new project) they are defined, while in the setup that I have currently there is none. It's really the only thing that's different.

Answer 13 · 2022-04-29T08:22:37.000Z

@dbarrosop I already solved the problem. In each column of the table I had to add a GraphQL Field Name in camelcase. And to define the Custom GraphQL Root Fields. This in both tables (storage.buckets and storage.files).

Thank you very much for helping me 🙏