EU-EDPS/website-evidence-collector

Issue with sending cookies information through a file cookies.txt

Closed this issue · 6 comments

iAsimov commented

Hello,

I have setup a Netscape formatted file in order to ingest cookies before starting the WEC. The information within the file isn't taken into consideration. There is no error message within the terminal other than

{"level":"info","message":"info","timestamp":"2021-03-07T14:12:21.857Z"}

Here is the information within the cookies.txt file

.example.com TRUE / TRUE 1646661702 Name Value

I would like to import all the cookie information exported by the Get cookies.txt chrome extension.

Thanks.

Deleted user commented

@bitnesswise, can you help here?

bitnesswise commented

Hi iAsimov

I just double-checked with a fresh install of the latest version and it should still work. I created a cookie file with curl and then passed the same cookie file to the website-evidence-collector and they were picked up correctly. I also don't see any issues with the format of your file at first glance.

So I suspect it might have to do with how you pass them to the tool.
There's a few ways you can debug this:

  1. If cookies are succesfully passed as an argument of the website-evidence-collector command, then the debug information will display this. You should be able to find it in the first few lines of the output; it looks similar to this:

{"level":"info","message":"read cookie parameter from the existing file: /cookies/cookie.txt","timestamp":"2021-03-08T21:59:19.291Z"}

  1. If you don't see this output, then the cookie file was probably not passed correctly as an argument. In that case, please double check your command or otherwise let me know the exact command that you're using and I'll have a look at it.
  2. Alternatively, you can first try to pass cookies as an argument without using a file. This should give you some confidence that they're picked up and that you can locate the debug-output which shows you that they're parsed and processed. You can do this with the same -c parameter (for example: -c "cookie_one=value1;cookie_two=value2")
iAsimov commented

Thanks for your feedback....

Here's what I have....

{"level":"info","message":"read cookie parameter from the existing file: /Users/xxx/Downloads/cookies.txt","timestamp":"2021-03-13T19:40:59.986Z"}
{"level":"error","message":"invalid formatted line - skipping it : ","timestamp":"2021-03-13T19:40:59.987Z"}
{"level":"info","message":"info","timestamp":"2021-03-13T19:40:59.987Z"}
{"level":"info","message":"info","timestamp":"2021-03-13T19:40:59.987Z"}

here is the content of my cookies.txt file

# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This is a generated file!  Do not edit.

.example.com	TRUE	/	FALSE	1644351800	didomi_token	eyJ1c2VyX2lkIjoiMTc2ZDIzMmMtY2NlNi02NmJiLTg2YTYtMDQzMTlhMmViMjgxIiwiY3JlYXRlZCI6IjIwMjEtMDItMDhUMjA6MjM6MjAuNDM0WiIsInVwZGF0ZWQiOiIyMDIxLTAyLTA4VDIwOjIzOjIwLjQzNFoiLCJ2ZW5kb3JzIjp7ImVuYWJsZWQiOlsiZmFjZWJvb2siLCJ0d2l0dGVyIiwiZ29vZ2xlIiwiYzpob3RqYXIiLCJjOmdyYXBlc2hvdCIsImM6Y2ltLUttaDlVZXJkIiwiYzpzZWxsaWdlbnQtVWNQa2tSeDQiLCJjOnRyYWRldHJhYy1ycUZrNmpIZyIsImM6cmViZWxhaS1IWWlFcEFKViIsImM6eW91dHViZSJdfSwicHVycG9zZXMiOnsiZW5hYmxlZCI6WyJnZW9fbWFya2V0aW5nX3N0dWRpZXMiLCJnZW9fYWRzIiwiZGV2aWNlX2NoYXJhY3RlcmlzdGljcyIsImdlb2xvY2F0aW9uX2RhdGEiXX0sInZlbmRvcnNfbGkiOnsiZW5hYmxlZCI6WyJnb29nbGUiXX0sInZlcnNpb24iOjIsImFjIjoiQ25XQUdCME1FQ2dLZFFBQS5DbldBR0IwTUVDZ0tkUUFBIn0=
.example.com	TRUE	/	FALSE	1644351800	euconsent-v2	CPBUCk0PBUCk0AHABBENBMCsAP_AAH_AAAAAHWtf_X_fb3_j-_59_9t0eY1f9_7_v-0zjhfds-8Nyf_X_L8X42M7vF36pq4KuR4Eu3LBIQdlHOHcTUmw6okVrTPsbk2Mr7NKJ7PEinMbe2dYGH9_n9XTuZKY79_s___z__-__v__7_f_r-3_3_vp9X---_e_V3dgdYASYal8BFmJY4Ek0aVQogQhXEh0AoAKKEYWiawgJXBTsrgI9QQMAEBqAjAiBBiCjFgEAAAAASURASAHggEQBEAgABACpAQgAI0AQWAFgYBAAKAaFgBFAEIEhBkcFRymBARItFBPJWAJRd7GGEIZRYAUCj-iowEAAAAA.f_gAD_gAAAAA

do you think there is an issue with language of my computer ? French.

Thanks for your support.

bitnesswise commented

I've detected two problems with your example:

  1. The blank line caused an error to be reported (invalid formatted line). While that is mostly a cosmetic problem it can be confusing, so I fixed that
  2. a mismatch occured for the domain name because of the leading dot and the code not handling subdomains correctly

I created a pull request with a fix for that.

Thank you for reporting this problem @iAsimov !

Deleted user commented

Fixed in b51340b. Thank you @bitnesswise