SSL file upload

Question

SSL file upload

vojta7 opened this issue 6 years ago · 11 comments

Hi,
I had build meracat on current arch linux and it seems to be working well. But when I try to upload file with ssl enabled I receive only random mess of data with no boundary. To me it looks like it did not get deciphered or it is compressed. If i disable ssl everything works fine.

I tried to dig into source code, but I am unable to find where it gets messed up. Could you please point me to right direction where to look or help me to solve this problem.

Thanks for your time.
Vojta

Answer 1 · 2018-08-29T09:17:15.000Z

Hi!

The HTTPS support is still farily new (and yet unreleased). I've been meaning to get back to Merecat to finalize the support so I can start testing letsencrypt. Would be great to get some help looking into this! :-)

I guess you mean CGI upload, right? That code path is in src/libhttpd.c, likely the cgi_child() function handles this. Not really sure I got around to inspecting that, could be that the forked child doesn't inherit the SSL socket, or that I've just forgot to add the correct wrapper methods. You can add some more syslog(LOG_DEBUG, ...) calls to help track it down further, use -l debug, to enable debug messages in the log.

Answer 2 · 2018-08-30T08:03:00.000Z

Thank, you for your response. Yes I mean CGI upload. I looked into cgi_child(), but I am unable to find it.
At least I have some observations that could be helpful:

regular post of form data arrives to CGI without any problem.
file upload gets mangled but first three bytes are still the same no matter the file content:
\0x17\0x03\0x03 rest look to me like it is still encrypted.

Answer 3 · 2018-08-30T15:27:52.000Z

OK, it's on line 3665 in src/libhttpd.c

Do you have an example CGI and config file, if any, I can test with?

Answer 4 · 2018-08-30T17:50:30.000Z

Hi,
I meant that I can't find the bug, not the function :). Here is my basic shell CGI example. There are two forms one with file input and second with basic text input. When I set ssl=false both works like expected, when I turn ssl on only text input works like expected.

merecat.conf

ssl = true
port = 8443
certfile = certs/cert.pem
keyfile  = private/key.pem

directory = /tmp/www

send.sh

#!/bin/bash

echo 'echo "Content-type: text/html"
echo

<!DOCTYPE html>
<html>

<head>
	<meta http-equiv="Content-Type" content="text/html;charset=UTF-8">
</head>

<body>
    <form method="post" action="/cgi-bin/recieve.sh" enctype="multipart/form-data">
        <input type="file" name="file_data" />
        <input type="submit" />
    </form>
    <form method="post" action="/cgi-bin/recieve.sh">
        <input type="text" name="test" />
        <input type="submit" />
    </form>
</body>

</html>'

recieve.sh

#!/bin/bash

if [ "$REQUEST_METHOD" = "POST" ]; then
    DATA=$( head -c $CONTENT_LENGTH )
fi

echo 'echo "Content-type: text/html"
echo

<!DOCTYPE html>
<html>

<head>
	<meta http-equiv="Content-Type" content="text/html;charset=UTF-8">
</head>

<body>
    <p>
        Data recieved
        <br />
        '"$DATA"'
        <br />
        <a href=send.sh>back</a>
    </p>
</body>

</html>'

echo "$DATA" > /tmp/data

Answer 5 · 2018-08-30T19:00:18.000Z

It was a bit ambiguous what you meant :-)

Thanks for the CGI and .conf, I'll have a look at it tomorrow or during the weekend at the latest!

Answer 6 · 2018-09-02T09:13:47.000Z

OK, finally managed to reproduce your problem. Had some trouble setting up a working self-signed cert with the latest firefox/chrome.

I think I've found the problem. When in SSL mode Merecat uses the internal functions httpd_ssl_write() and httpd_ssl_read() which are wrappers around the file descriptor to do the SSL magic. However, the CGI is a forked off process that writes raw data to the file descriptor.

This is bad news since it means there's potentially a lot of work to get HTTPS working with CGI. I'll see what I can do, currently thinking about setting up a pipe() between the CGI child and the parent process, but it's likely gonna take me a while to get around to since i can only work on Merecat on my spare time.

Answer 7 · 2018-09-02T11:53:02.000Z

Heh, funnily enough I may have fixed it! :-)

Turns out 1) there already was a pipe() and pre/post-processing of CGI, 2) when in SSL mode we read a LOT (all?) of data pre-processing, so all I had to do was to change a comparison > to >= ... works for my test case.

Love to head back from you if this fixes your problem as well.

Answer 8 · 2018-09-03T09:47:09.000Z

Yes it seems to fix my problem too :). Thank you.

Answer 9 · 2018-09-03T15:03:00.000Z

Awesome, thanks for getting back! (closing issue)

What remains (for me) before the next release is adding support for HTTPS and HTTP at the same time, with automatic redirect if so configured. Should be fairly straight forward, but unfortunately not prioritized atm. Hope you can make do with the current GIT master.

Answer 10 · 2018-09-05T06:56:37.000Z

Thanks for fix. HTTPS and HTTP at the same time would be really nice, but I am ok with current master :).

I have one more question, is it intended to print stderr to web?

Answer 11 · 2018-09-05T11:33:50.000Z

OK, great!

You mean from a cgi? Yes, I believe so. You'll have to redirect stderr yourself in the CGI. Either with syslog(), in C/Python, or maybe with logger, in shell scripts.