HTTP headers may not be saved in the correct case
Closed this issue · 2 comments
Today, @jakubd told me that he thinks some HTTP headers are coming back all lowercase when they had varying case at the client. This is a serious problem and we need capitalization for our HTTP headers. We moved away from requests because it didn't allow us to save capitalization, but we are currently using httplib, which requests is built on top of.
Someone needs to try to reproduce the issue by fetching pages with case sensitive headers (you can see the headers with curl -D <filename for headers> <website>. If the issue is reproducible and a problem with httplib, we will need to move to a different HTTP primitive.
@Tikitaco, I don't actually know who you are, but I saw that you were responsible for the HTTP primitive pull request yesterday. Would you be willing to try and reproduce?
Another reason this may be a problem: It would be very, very good to preserve things like 0x20 encodings, should we decide to do tests with that for DNS.
I was able to reproduce the issue with httplib and there doesn't seem to be any obvious fix for it besides switching libraries. Pycurl may be a nice alternative since it preserves the HTTP header capitalization if we decide to switch