awslabs/aws-sigv4-proxy

On Upload Object Fail

edgwork opened this issue · 6 comments

edgwork commented

Hi all,

When I try to upload an object I get the following error:
    <?xml version=\"1.0\" encoding=\"UTF-8\"?>
    <Error>
        <Code>AccessDenied</Code>
        <Message>There were headers present in the request which were not signed</Message>
        <HeadersNotSigned>content-md5</HeadersNotSigned>
        <RequestId>E6ZWRP6AH7X7KW1K</RequestId>
        <HostId>zKDm2Xhp9D3kyUlDYq/YjLFNFYIElsDwOrCuKVs23jH8t/bZlR+vlpDwhvL3J2HMdNei3WoCKSY=</HostId>
    </Error>

I already tried to add the headers before and after the signature, without success

You can support me because this happens I will appreciate it very much, this is the fragment of the log when the call is made:

time="2021-08-16T19:26:17Z" level=info msg="Stripping headers []" StripHeaders="[]"
time="2021-08-16T19:26:17Z" level=info msg="Listening on :8080" port=":8080"
time="2021-08-16T19:26:30Z" level=debug msg="Initial request dump:" request="PUT /edgwork-my-bucket/edgwork-ac-upload.txt HTTP/1.1\r\nHost: localhost:8080\r\nAccept-Encoding: identity\r\nAuthorization: AWS4-HMAC-SHA256 Credential=ASIAVLCN4DG7MYCHQTG5/20210816/us-east-1/s3/aws4_request, SignedHeaders=content-md5;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=5349eaa0fb81aefae7e1c774da8762dab01deb4839c1b5c46d468b875ac3b248\r\nContent-Length: 161\r\nContent-Md5: BEfrRIi1xGJS1dGfkeBzkA==\r\nContent-Type: text/plain\r\nExpect: 100-continue\r\nUser-Agent: aws-cli/1.18.147 Python/2.7.18 Linux/4.14.238-182.422.amzn2.x86_64 botocore/1.18.6\r\nX-Amz-Content-Sha256: 369d88d309b4d1184784ac0f60490faf168e75a2747095e0bfc74f307f43b8c6\r\nX-Amz-Date: 20210816T192630Z\r\nX-Amz-Security-Token: IQoJb3JpZ2luX2VjEDwaCXVzLWVhc3QtMSJHMEUCIQC9EyElXnmyk/UxC7R++d/v6XfG3fsk6U+nkIe+LzM+lAIgHuBYJF/MJPQQ9HEuWI9pCPA6TyAA2VOom7Mqa5N3B/Aq+gMIZBAAGgwzNjczODMwMjYxMTAiDHC4yMKBE1SN56ff2yrXAyluPo4aSYuza77stxvUOJssXDHZJq96KnFJXJbPzmCIkwxgmV/HFdwrkAE5/hmFe7J7HGEdrctokXsUfbRr+/ifos41gY1dLr1ZSg0fZ5kKWK9CgQNYGKiyXwwlTsWQATIag2T5VvV1r7i88Ek64sx+cdBB/E7Vfrmos6yZvneeUiWVu6R81OB0UlozdsdG9bxenN0lxyrH3MAVQW5UsKGCDQ3i89JjMl3KDOgJTNOI6kslaBxgm+zpb8pCRcUdsCjL7ZxCLLGTww0HXAMDCVqszHHwaTWA5RXHsY4/DHSngAEiqwYl9dpQb1LMrK9kcjLLKLNvVz3FX2Wxc5x/96VJdF2ORjQ7emT3J8cZqPzAbcv22TP4rBJnmxy18//brz4jkRikNuB/PzgSRLBQ3XqXBzJ1uLnXAySX9gy6xN84OZjt4OAADqb+CUSx2EKQl4XblQvQSfILGPC1j4tO2e1g3Bb5tflpbAonjuy7YAUEz98HCRo2N3YrkL+4XvqtIOk0PNQo3ikLLjyCyiC2NrqpP7BiF0loqXpI1983f+1BM2sRZ3dCqt2koP+51TQIEqLJ34ghNSGOqWv7MjExBUqCvDbZghmFTBMx/M9PfDbskqBgsxgMPTDI8eqIBjqlARcK2wPRKIqIW10qJ8daq6ZxE94xVDYtLV304qyPE9CVaqFNdWO7hfxPL+JmwktIVYpE8efJy8tIog7/W7F5wxH1KKddV//VoTy8h/AHnI8nTxDq/PXcU2tKBhX8sTmZtWvVs1TRhVaNj0XpANptYQ6zEN+2CnAsqhmiEg6V3xAGFugbcZEpGJ0vEf9JH9OjhMjwT8yNTqPq14HChTe83euhqPp9aA==\r\n\r\n0 Hola Edgwork!\n1 Hola Edgwork!\n2 Hola Edgwork!\n3 Hola Edgwork!\n4 Hola Edgwork!\n5 Hola Edgwork!\n6 Hola Edgwork!\n7 Hola Edgwork!\n8 Hola Edgwork!\n9 Hola Edgwork!\n\n"
time="2021-08-16T19:26:30Z" level=debug msg="signed request" region=us-east-1 service=s3
time="2021-08-16T19:26:30Z" level=debug msg="proxying request" request="PUT /edgwork-my-bucket/edgwork-ac-upload.txt HTTP/1.1\r\nHost: s3.amazonaws.com\r\nAccept-Encoding: identity\r\nAuthorization: AWS4-HMAC-SHA256 Credential=ASIAVLCN4DG7MYCHQTG5/20210816/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=6c0423f517c7667ed7a2badbd6af05ffe7e887f870d130e8211ea3bbbe13f912\r\nContent-Length: 161\r\nContent-Md5: BEfrRIi1xGJS1dGfkeBzkA==\r\nContent-Type: text/plain\r\nExpect: 100-continue\r\nUser-Agent: aws-cli/1.18.147 Python/2.7.18 Linux/4.14.238-182.422.amzn2.x86_64 botocore/1.18.6\r\nX-Amz-Content-Sha256: 369d88d309b4d1184784ac0f60490faf168e75a2747095e0bfc74f307f43b8c6\r\nX-Amz-Date: 20210816T192630Z\r\nX-Amz-Security-Token: IQoJb3JpZ2luX2VjEDwaCXVzLWVhc3QtMSJHMEUCIQC9EyElXnmyk/UxC7R++d/v6XfG3fsk6U+nkIe+LzM+lAIgHuBYJF/MJPQQ9HEuWI9pCPA6TyAA2VOom7Mqa5N3B/Aq+gMIZBAAGgwzNjczODMwMjYxMTAiDHC4yMKBE1SN56ff2yrXAyluPo4aSYuza77stxvUOJssXDHZJq96KnFJXJbPzmCIkwxgmV/HFdwrkAE5/hmFe7J7HGEdrctokXsUfbRr+/ifos41gY1dLr1ZSg0fZ5kKWK9CgQNYGKiyXwwlTsWQATIag2T5VvV1r7i88Ek64sx+cdBB/E7Vfrmos6yZvneeUiWVu6R81OB0UlozdsdG9bxenN0lxyrH3MAVQW5UsKGCDQ3i89JjMl3KDOgJTNOI6kslaBxgm+zpb8pCRcUdsCjL7ZxCLLGTww0HXAMDCVqszHHwaTWA5RXHsY4/DHSngAEiqwYl9dpQb1LMrK9kcjLLKLNvVz3FX2Wxc5x/96VJdF2ORjQ7emT3J8cZqPzAbcv22TP4rBJnmxy18//brz4jkRikNuB/PzgSRLBQ3XqXBzJ1uLnXAySX9gy6xN84OZjt4OAADqb+CUSx2EKQl4XblQvQSfILGPC1j4tO2e1g3Bb5tflpbAonjuy7YAUEz98HCRo2N3YrkL+4XvqtIOk0PNQo3ikLLjyCyiC2NrqpP7BiF0loqXpI1983f+1BM2sRZ3dCqt2koP+51TQIEqLJ34ghNSGOqWv7MjExBUqCvDbZghmFTBMx/M9PfDbskqBgsxgMPTDI8eqIBjqlARcK2wPRKIqIW10qJ8daq6ZxE94xVDYtLV304qyPE9CVaqFNdWO7hfxPL+JmwktIVYpE8efJy8tIog7/W7F5wxH1KKddV//VoTy8h/AHnI8nTxDq/PXcU2tKBhX8sTmZtWvVs1TRhVaNj0XpANptYQ6zEN+2CnAsqhmiEg6V3xAGFugbcZEpGJ0vEf9JH9OjhMjwT8yNTqPq14HChTe83euhqPp9aA==\r\n\r\n0 Hola Edgwork!\n1 Hola Edgwork!\n2 Hola Edgwork!\n3 Hola Edgwork!\n4 Hola Edgwork!\n5 Hola Edgwork!\n6 Hola Edgwork!\n7 Hola Edgwork!\n8 Hola Edgwork!\n9 Hola Edgwork!\n\n"
time="2021-08-16T19:26:30Z" level=error msg="error proxying request" message="<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>There were headers present in the request which were not signed</Message><HeadersNotSigned>content-md5</HeadersNotSigned><RequestId>E6ZWRP6AH7X7KW1K</RequestId><HostId>zKDm2Xhp9D3kyUlDYq/YjLFNFYIElsDwOrCuKVs23jH8t/bZlR+vlpDwhvL3J2HMdNei3WoCKSY=</HostId></Error>"

Greetings

tmehlinger commented

My team is having similar issues. Listing buckets through the proxy works fine but making PUT requests produces this error.

Is this a configuration issue or a bug in the proxy?

alvinlin123 commented

Looking at the example request in the issue it looks like the request to the proxy is already signed (judging by the user-agent header, it seems like a request generated by AWS SDK, which is already signed), in which case the proxy is not needed. I am curious why the proxy is used in this case?

The reason for the error is because the current implementation of the proxy does not sign the incoming request headers:

aws-sigv4-proxy/handler/proxy_client.go

Line 132 in 5e967d4

// Add origin headers after request is signed (no overwrite)
but S3 requires the content-md5 header to be signed; hence, the error.

I am not entirely sure why the proxy does not sign the; my guess is that there are header whose value would change hop by hop (e.g. x-forwarded-for), and should not be signed. So the easiest way to avoid the problem is to not sign the incoming headers.

edgwork commented
alvinlin123 commented

@edgwork can you confirm #70 fixes this issue like afterhill mentioned?

keyolk commented

I tested with v1.7 docker image
But still has the same problem
With some other clients, it also puts errors below

<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>There were headers present in the request which were not signed</Message><HeadersNotSigned>x-amz-decoded-content-length</HeadersNotSigned><RequestId>9B1D4MK7TQM16BK2</RequestId><HostId>CA04fsB/FaKaAXn1a0RzwL4oZfePI2ESaCa2Ki4YqCh8zQdRLuxSe00AWeTy68bYaZMmIM2xLBM=</HostId></Error>

the error is gone after passing the flags, but not sure if it is just OK

--strip x-amz-decoded-content-length
--strip content-md5
edgwork commented

@edgwork can you confirm #70 fixes this issue like afterhill mentioned?

We are sorry but we used another tool due to the pressure we had at that time, so we no longer continue testing

  • 😕1