Binary File upload

[rodrgomes]

Following the help, I manage to download a binary file using

select textsend(content) from http_get('http://127.0.0.1/some_binary_file.bin')

But i'm trying without success do some binary upload. Tried

select * from http(('PUT','https://transfer.sh/upload.bin',null,'application/octet-stream',( bytea '\x3132330405007C'))::http_request)

select * from http(('PUT','https://transfer.sh/upload.bin',null,'application/octet-stream',urlencode( bytea '\x3132330405007C'))::http_request)

but nothings seems work... Is it possible?

[pramsey]

It seems like the definition of http_request might mitigate against it... the trick would be getting the binary data into the varchar content field, so the reverse problem of the GET. It looks like textrecv does what you want?

[pramsey]

Maybe try the two new functions in https://github.com/pramsey/pgsql-http/tree/1.6 and see how it goes. text_to_bytea() and bytea_to_text(). Use the latter for your upload to convert the upwards heading bytea into text for the put payload.

[rodrgomes]

About textrecv, can't use it because the parameter is "internal" type... can't call that directly.

But did another test... called

select * from http(('PUT','https://transfer.sh/upload.bin',null,'application/octet-stream',
(select content from http_get('http://127.0.0.1/some_binary_file.bin')
)::http_request)

where http://127.0.0.1/some_binary_file.bin return a file with chr(0) inside and its works... can't use it to send bytea, but seems the problem is really convert bytea to text with special chars.

I'll try that bytea_to_text now

[rodrgomes]

Maybe try the two new functions in https://github.com/pramsey/pgsql-http/tree/1.6 and see how it goes. text_to_bytea() and bytea_to_text(). Use the latter for your upload to convert the upwards heading bytea into text for the put payload.

That worked! I tried

select * from http_put('https://transfer.sh/test_1.6.bin',bytea_to_text(bytea '\x32401000104000'),'application/octet-stream')

And after that downloaded the file from transfer.sh and got all 7 bytes... ty.

Btw, a ';' is missing in first line of http--1.5--1.6.sql, in

ALTER DOMAIN http_method drop CONSTRAINT http_method_check

[pramsey]

Wonderful, thanks for the quick feedback.

[Iced-Sun]

Maybe try the two new functions in https://github.com/pramsey/pgsql-http/tree/1.6 and see how it goes. text_to_bytea() and bytea_to_text(). Use the latter for your upload to convert the upwards heading bytea into text for the put payload.

That worked! I tried

select * from http_put('https://transfer.sh/test_1.6.bin',bytea_to_text(bytea '\x32401000104000'),'application/octet-stream')

And after that downloaded the file from transfer.sh and got all 7 bytes... ty.

Btw, a ';' is missing in first line of http--1.5--1.6.sql, in

ALTER DOMAIN http_method drop CONSTRAINT http_method_check

The binary upload doesn't work for POST.

http_put works as expected:

select *
  from http_put('https://httpbin.org/put',bytea_to_text(bytea '\x32401000104000'),'application/octet-stream')

status	content_type	headers	content
200	application/json	{"(date,\"Mon, 22 Apr 2024 08:14:04 GMT\")","(content-type,application/json)","(content-length,585)","(server,gunicorn/19.9.0)","(access-control-allow-origin,*)","(access-control-allow-credentials,true)"}	{
  "args": {},
  "data": "2@\u0010\u0000\u0010@\u0000",
  "files": {},
  "form": {},
  "headers": {
    "Accept": "*/*",
    "Accept-Encoding": "deflate, gzip, br",
    "Charsets": "utf-8",
    "Content-Length": "7",
    "Content-Type": "application/octet-stream",
    "Host": "httpbin.org",
    "User-Agent": "PostgreSQL 16.2 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 11.4.1 20230605 (Red Hat 11.4.1-2), 64-bit",
    "X-Amzn-Trace-Id": "Root=1-66261c4b-5ca1ce3e2613899f002fad7e"
  },
  "json": null,
  "url": "https://httpbin.org/put"
}

http_post breaks with \0:

select *
  from http_post('https://httpbin.org/post',bytea_to_text(bytea '\x32401000104000'),'application/octet-stream')

status	content_type	headers	content
200	application/json	{"(date,\"Mon, 22 Apr 2024 08:14:17 GMT\")","(content-type,application/json)","(content-length,567)","(server,gunicorn/19.9.0)","(access-control-allow-origin,*)","(access-control-allow-credentials,true)"}	{
  "args": {},
  "data": "2@\u0010",
  "files": {},
  "form": {},
  "headers": {
    "Accept": "*/*",
    "Accept-Encoding": "deflate, gzip, br",
    "Charsets": "utf-8",
    "Content-Length": "3",
    "Content-Type": "application/octet-stream",
    "Host": "httpbin.org",
    "User-Agent": "PostgreSQL 16.2 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 11.4.1 20230605 (Red Hat 11.4.1-2), 64-bit",
    "X-Amzn-Trace-Id": "Root=1-66261c59-00232c7a3f27ae037d045316"
  },
  "json": null,
  "url": "https://httpbin.org/post"
}