net/url: RequestURI encoded path should not encode '!'

Question

net/url: RequestURI encoded path should not encode '!'

Opened this issue 12 years ago · 17 comments

see https://groups.google.com/forum/#!topic/golang-nuts/5er6Ud_V0-U

Answer 1 · 2013-11-18T08:50:15.000Z

Comment 1:

see http://play.golang.org/p/X6LGcNbHzA, it's more obvious.
it's affected by all struts2 framework, it's serious.

Answer 2 · 2013-11-18T09:38:39.000Z

Comment 2:

Here is a smaller reproduction,
http://play.golang.org/p/xPQ61lbUqE
The bone of contention is the encoding of !. I am not sure if this is a problem or not.

Answer 3 · 2013-11-18T10:31:33.000Z

Comment 3:

The https://www.shipin7.com/user/userAction%21goRegister.action page is incorrect.
https://www.shipin7.com/user/userAction!goRegister.action page is ok.
you can compare above in Browser.
http.Client.Do(), http.Get() internal encode '!' and send whole Request to server.
I think '!' should not encode because of RFC3986

Answer 4 · 2013-11-23T12:17:56.000Z

Comment 4 by hongruiqi:

In RFC 2396:
      reserved    = ";" | "/" | "?" | ":" | "@" | "&" | "=" | "+" |
                    "$" | ","
      unreserved  = alphanum | mark
      mark        = "-" | "_" | "." | "!" | "~" | "*" | "'" | "(" | ")"
In RFC 3986:
      reserved    = gen-delims / sub-delims
      gen-delims  = ":" / "/" / "?" / "#" / "[" / "]" / "@"
      sub-delims  = "!" / "$" / "&" / "'" / "(" / ")"
                  / "*" / "+" / "," / ";" / "="
      unreserved  = ALPHA / DIGIT / "-" / "." / "_" / "~"
https://code.google.com/p/go/source/detail?r=6b46fb967ca4a48caf486f4452c4358251f91aad
The CL above only removes !*\() from unreserved part(the \ may be wrong, it should be
"'"), 
but doesn't add []!'()* to the reserved part. So I think it's a bug.

Answer 5 · 2013-11-23T12:37:20.000Z

Comment 5 by hongruiqi:

sorry, I mistake '\'' as '\', nothing wrong here.

Answer 6 · 2013-11-27T18:44:28.000Z

Comment 6:

Labels changed: added go1.3maybe.

Answer 7 · 2013-12-02T04:33:43.000Z

Comment 7:

Labels changed: added priority-soon, packagebug, removed priority-triage.

Answer 8 · 2013-12-04T01:31:05.000Z

Comment 8:

Labels changed: added release-none, removed go1.3maybe.

Answer 9 · 2013-12-04T01:43:14.000Z

Comment 9:

Labels changed: added repo-main.

Answer 10 · 2013-12-04T16:25:48.000Z

Comment 10:

This also triggers with the hashbang style single-page web app url fragments:
http://play.golang.org/p/-kx5yULrzl
    u, err := url.Parse("http://foo.bar/#!quux")
    // http://foo.bar/#%21quux
See https://developers.google.com/webmasters/ajax-crawling/ for more.

Answer 11 · 2013-12-05T06:40:20.000Z

Comment 11:

https://golang.org/cl/31400043/
Does anyone know why ! is left out in the first place? was that intentional?

Status changed to Started.

Answer 12 · 2013-12-10T19:48:01.000Z

Comment 12:

Some examples of URLs where parentheses don't work if they're escaped:
http://web.signaltiretrader.com/(S(5iexcz551ptpgo45g03mgz45))/Themes/css/ploneColumns.css
and the LinkedIn API URLs discussed at
https://groups.google.com/forum/#!searchin/golang-nuts/url$20escaping/golang-nuts/Mro8TGrb3y8/eW8QCx_iFYMJ

Answer 13 · 2013-12-12T16:02:18.000Z

Comment 13:

jkbbwr on IRC pointed out that slashes in queries get quoted too:
http://play.golang.org/p/EiRhkOT8im
Relevant RFC: http://tools.ietf.org/html/rfc3986#section-3.4

Answer 14 · 2013-12-13T02:01:37.000Z

Comment 14 by hongruiqi:

Slashes not quoted in queries may causes some server failed to handle?

Answer 15 · 2013-12-30T22:49:17.000Z

Comment 15:

This appears to be a duplicate of 5684.

Answer 16 · 2013-12-30T22:52:41.000Z

Comment 16:

Referenced on StackOverflow:
http://stackoverflow.com/questions/20847357/golang-http-client-always-escaped-the-url/

Answer 17 · 2014-01-15T20:55:57.000Z

Comment 17:

Status changed to Duplicate.

Merged into issue #5684.