okigan/awscurl

Strings must be encoded before hashing

namedgraph opened this issue · 8 comments

namedgraph commented

Just installed awscurl on EC2 (Ubuntu) and trying to access the Neptune DB endpoint, but

awscurl 'https://xxxxxxxxxxxcluster.cluster-ro-cnol6sn9sq5j.us-east-1.neptune.amazonaws.com:8182/sparql' \
        -X POST \
        --data-binary \
        -d 'query=ASK {}' \
        --region us-east-1 \
        --service neptune-db

this gives me an error:

Traceback (most recent call last):
  File "/home/ubuntu/.local/bin/awscurl", line 8, in <module>
    sys.exit(main())
  File "/home/ubuntu/.local/lib/python3.10/site-packages/awscurl/awscurl.py", line 543, in main
    inner_main(sys.argv[1:])
  File "/home/ubuntu/.local/lib/python3.10/site-packages/awscurl/awscurl.py", line 516, in inner_main
    response = make_request(args.request,
  File "/home/ubuntu/.local/lib/python3.10/site-packages/awscurl/awscurl.py", line 107, in make_request
    canonical_request, payload_hash, signed_headers = task_1_create_a_canonical_request(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/awscurl/awscurl.py", line 219, in task_1_create_a_canonical_request
    payload_hash = sha256_hash_for_binary_data(data) if data_binary else sha256_hash(data)
  File "/home/ubuntu/.local/lib/python3.10/site-packages/awscurl/utils.py", line 20, in sha256_hash_for_binary_data
    return hashlib.sha256(val).hexdigest()
TypeError: Strings must be encoded before hashing

Am I missing something? :)

Also, is intentional that the --data-binary option does not take an argument? Because it does in plain curl.

okigan commented
namedgraph commented 
$ python3 --version
Python 3.10.12
okigan commented
namedgraph commented

I was just following the examples from Using RDF and SPARQL to access the graph in Amazon Neptune and replacing curl with awscurl :)

If I remove the --data-binary arg

awscurl 'https://octopustriplestorecluster.cluster-ro-cnol6sn9sq5j.us-east-1.neptune.amazonaws.com:8182/sparql' \
        -X POST \
        -d 'query=ASK {}' \
        --region us-east-1 \
        --service neptune-db

I get a Neptune error:

{"requestId":"2ec4e671-f734-5519-03fd-6971f5229051","detailedMessage":"Missing 'query' or 'update' parameter for POST request","code":"MissingParameterException"}
okigan commented
namedgraph commented

This works!

awscurl 'https://xxxxxxxxxxxxcluster.cluster-ro-cnol6sn9sq5j.us-east-1.neptune.amazonaws.com:8182/sparql' \
        -X POST \
        -d 'query=ASK {}' \
        --region us-east-1 \
        --service neptune-db \
        --header 'Content-Type: application/x-www-form-urlencoded'
namedgraph commented

@okigan what about --data-binary? Why does it differ from plain curl?

jaw111 commented

The following works for me for to send a CONSTRUCT query and requesting response as N-Triples (using URL from your example for consistency):

awscurl "https://xxxxxxxxxxxxcluster.cluster-ro-cnol6sn9sq5j.us-east-1.neptune.amazonaws.com:8182/sparql" \
  --service neptune-db \
  --data @query.rq \
  -H "Content-Type: application/sparql-query" \
  -H "Accept: application/n-triples"

Note that with query via POST with URL-encoded parameters the content type header of the HTTP request must be set to application/x-www-form-urlencoded. Perhaps awscurl does not automatically add that header using -d and --data, you could run in verbose mode to check that.

  • 👍1