Custom user agent

[AyrA]

You should use a custom user agent string that somewhat identifies the toolkit, for example twa/${TWA_VERSION} +https://github.com/trailofbits/twa

This can potentially get around the problem of some sites blocking generic curl user agents. It's also nice in general to tell what the source of automated requests is

[woodruffw]

Thanks for the report!

I agree that twa shouldn't use the default curl user agent. However, I think we should actually default to a Firefox or Chrome user agent rather than a unique identifier: many sites will conditionally serve (or block) requests to agents that don't look like web browsers, leading to inaccurate scorings for things like cookies.

My plan is to add a TWA_USER_AGENT variable that users can override. If you'd like your twa calls to use a user-agent like the one you've suggested, you'll be able to set it to do so.

[woodruffw]

(Until I add this, you can use the TWA_CURLOPTS environment variable to feed -A <agent-name> to twa's curl calls).