bug: gateway-conformance support for query parameter
SgtPooki opened this issue · 2 comments
SgtPooki commented
i'm doing some tests on ipfs/helia-http-gateway#67 and noticed one oddity. we're not converting dag-json properly? the content-type is being set, incorrectly, to text/plain; charset=utf-8
We should be falling into the block similar to
helia-verified-fetch/packages/verified-fetch/src/verified-fetch.ts
Lines 243 to 249 in 39bb14e
raw to dag-json ?
requesting http://helia-http-gateway.localhost/ipfs/bafkreialihlqnf5uwo4byh4n3cmwlntwqzxxs2fg5vanqdi3d7tb2l5xkm?format=dag-json results in the following output from verified-fetch:
[helia] 2024-03-30T00:01:23.669Z helia:verified-fetch fetch http://helia-http-gateway.localhost/ipfs/bafkreialihlqnf5uwo4byh4n3cmwlntwqzxxs2fg5vanqdi3d7tb2l5xkm?format=dag-json
[helia] 2024-03-30T00:01:23.676Z helia:verified-fetch incoming accept header "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8"
[helia] 2024-03-30T00:01:23.676Z helia:verified-fetch incoming query format "dag-json", mapped to application/vnd.ipld.dag-json
[helia] 2024-03-30T00:01:23.676Z helia:verified-fetch output type application/octet-stream
[helia] 2024-03-30T00:01:23.676Z helia:verified-fetch:trace finding handler for cid code "85" and output type "application/octet-stream"
[helia] 2024-03-30T00:01:23.676Z helia:verified-fetch:trace calling handler "handleRaw"
[helia] 2024-03-30T00:01:23.676Z helia:verified-fetch:byte-range-context:trace no range request detected
[helia] 2024-03-30T00:01:23.677Z helia:trustless-gateway-block-broker getting block for bafkreialihlqnf5uwo4byh4n3cmwlntwqzxxs2fg5vanqdi3d7tb2l5xkm from http://127.0.0.1:8081/
[helia] 2024-03-30T00:01:23.720Z helia:trustless-gateway-block-broker:trace got block for bafkreialihlqnf5uwo4byh4n3cmwlntwqzxxs2fg5vanqdi3d7tb2l5xkm from http://127.0.0.1:8081/ipfs/bafkreialihlqnf5uwo4byh4n3cmwlntwqzxxs2fg5vanqdi3d7tb2l5xkm?format=raw
[helia] 2024-03-30T00:01:23.721Z helia:verified-fetch:byte-range-context:trace set _fileSize to 34
[helia] 2024-03-30T00:01:23.721Z helia:verified-fetch:byte-range-context:trace requestRangeStart and requestRangeEnd are null
[helia] 2024-03-30T00:01:23.721Z helia:verified-fetch:byte-range-context:trace set request body with fileSize 34
[helia] 2024-03-30T00:01:23.721Z helia:verified-fetch:byte-range-context:trace returning body unmodified for non-range, or invalid range, request
[helia] 2024-03-30T00:01:23.724Z helia:verified-fetch:trace setting content type to "text/html; charset=utf-8"
gateway conformance output is:
[gateway-conformance] �--- FAIL: TestGatewayJsonCbor/GET_UnixFS_file_with_JSON_bytes_is_returned_with_application%2Fjson_Content-Type_-_with_headers (0.02s)
[gateway-conformance] �--- FAIL: TestGatewayJsonCbor (0.04s)
[gateway-conformance] �--- FAIL: TestDagPbConversion/GET_UnixFS_file_as_DAG-JSON_with_format=dag-json_converts_to_the_expected_Content-Type/Header_Content-Type (0.00s)
[gateway-conformance] run.go:53:
[gateway-conformance] Name: GET UnixFS file as DAG-JSON with format=dag-json converts to the expected Content-Type
[gateway-conformance] Hint:
[gateway-conformance]
[gateway-conformance] Error: Header 'Content-Type' expected 'application/vnd.ipld.dag-json', got 'text/plain; charset=utf-8'
[gateway-conformance]
[gateway-conformance] Expected Request:
[gateway-conformance] {
[gateway-conformance] "method": "GET",
[gateway-conformance] "path": "/ipfs/bafkreialihlqnf5uwo4byh4n3cmwlntwqzxxs2fg5vanqdi3d7tb2l5xkm",
[gateway-conformance] "query": {
[gateway-conformance] "format": [
[gateway-conformance] "dag-json"
[gateway-conformance] ]
[gateway-conformance] }
[gateway-conformance] }
[gateway-conformance]
[gateway-conformance] Actual Request:
[gateway-conformance] GET /ipfs/bafkreialihlqnf5uwo4byh4n3cmwlntwqzxxs2fg5vanqdi3d7tb2l5xkm?format=dag-json HTTP/1.1
[gateway-conformance] Host: host.docker.internal:8080
[gateway-conformance] User-Agent: Go-http-client/1.1
[gateway-conformance] Accept-Encoding: gzip
[gateway-conformance]
[gateway-conformance]
[gateway-conformance]
[gateway-conformance] Expected Response:
[gateway-conformance] {
[gateway-conformance] "statusCode": 200,
[gateway-conformance] "headers": [
[gateway-conformance] {
[gateway-conformance] "key": "Content-Type",
[gateway-conformance] "check": {}
[gateway-conformance] },
[gateway-conformance] {
[gateway-conformance] "key": "Content-Disposition",
[gateway-conformance] "check": {}
[gateway-conformance] },
[gateway-conformance] {
[gateway-conformance] "key": "Content-Type",
[gateway-conformance] "check": {},
[gateway-conformance] "not": true
[gateway-conformance] }
[gateway-conformance] ],
[gateway-conformance] "body": "eyIvIjp7ImJ5dGVzIjoiU1NCaGJTQmhJSFI0ZENCbWFXeGxJRzl1SUhCaGRHZ2dkMmwwYUNCMWRHWTRDZyJ9fQ=="
[gateway-conformance] }
[gateway-conformance]
[gateway-conformance] Actual Response:
[gateway-conformance] HTTP/1.1 200 OK
[gateway-conformance] Content-Length: 31
[gateway-conformance] Access-Control-Allow-Headers: Content-Type, Range, User-Agent, X-Requested-With
[gateway-conformance] Access-Control-Allow-Methods: GET, HEAD, OPTIONS
[gateway-conformance] Access-Control-Allow-Origin: *
[gateway-conformance] Access-Control-Expose-Headers: Content-Range, Content-Length, X-Ipfs-Path, X-Ipfs-Roots, X-Chunked-Output, X-Stream-Output
[gateway-conformance] Connection: keep-alive
[gateway-conformance] Content-Type: text/plain; charset=utf-8
[gateway-conformance] Date: Fri, 29 Mar 2024 23:56:49 GMT
[gateway-conformance] Keep-Alive: timeout=72
[gateway-conformance]
[gateway-conformance]
[gateway-conformance]
SgtPooki commented
it looks like the incoming accept header is overriding things:
const incomingAcceptHeader = requestHeaders.get('accept');
if (incomingAcceptHeader != null) {
this.log('incoming accept header "%s"', incomingAcceptHeader);
}
const queryFormatMapping = queryFormatToAcceptHeader(query.format);
if (query.format != null) {
this.log('incoming query format "%s", mapped to %s', query.format, queryFormatMapping);
}
const acceptHeader = incomingAcceptHeader ?? queryFormatMapping;
From a browser request, this is automatically being set for me:
helia:verified-fetch incoming accept header "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8"
for a fetch request from the browser, this is also being set for me:
var resp = await fetch("http://helia-http-gateway.localhost/ipfs/bafkreialihlqnf5uwo4byh4n3cmwlntwqzxxs2fg5vanqdi3d7tb2l5xkm?format=dag-json")
var resp = await fetch("http://helia-http-gateway.localhost/ipfs/bafkreialihlqnf5uwo4byh4n3cmwlntwqzxxs2fg5vanqdi3d7tb2l5xkm?format=dag-json", {headers: {}})
// both result in: helia:verified-fetch incoming accept header "*/*"curl also results in accept: */*
SgtPooki commented
if I try var resp = await fetch("http://helia-http-gateway.localhost/ipfs/bafkreialihlqnf5uwo4byh4n3cmwlntwqzxxs2fg5vanqdi3d7tb2l5xkm?format=dag-json", {headers: { accept: undefined }})
I get
[helia] 2024-03-30T00:28:30.967Z helia-http-gateway:server fetching url "http://helia-http-gateway.localhost/ipfs/bafkreialihlqnf5uwo4byh4n3cmwlntwqzxxs2fg5vanqdi3d7tb2l5xkm?format=dag-json" with @helia/verified-fetch
[helia] 2024-03-30T00:28:30.967Z helia:verified-fetch fetch http://helia-http-gateway.localhost/ipfs/bafkreialihlqnf5uwo4byh4n3cmwlntwqzxxs2fg5vanqdi3d7tb2l5xkm?format=dag-json
[helia] 2024-03-30T00:28:30.967Z helia:verified-fetch incoming accept header "undefined"
[helia] 2024-03-30T00:28:30.967Z helia:verified-fetch incoming query format "dag-json", mapped to application/vnd.ipld.dag-json
[helia] 2024-03-30T00:28:30.967Z helia:verified-fetch output type undefined
[helia] 2024-03-30T00:28:30.968Z helia-http-gateway:server verified-fetch response not ok: 406
[helia] 2024-03-30T00:28:30.969Z helia-http-gateway:server:trace request destroyed for url "http://helia-http-gateway.localhost/ipfs/bafkreialihlqnf5uwo4byh4n3cmwlntwqzxxs2fg5vanqdi3d7tb2l5xkm?format=dag-json"