1863BFAF
apparently we're the RFC police.
1863BFAF is a project to validate adherence to HTTP/1.1 as defined by RFC2616
storytime
we found this problem while writing some exploratory Ruby code using Sinatra and Sequel, which boiled down to:
require 'sequel'
require 'sinatra'
# model
class Foo < Sequel::Model
...
end
# database
db = Sequel.connect('sqlite://test.db')
db.create_table? :bar do
primary_key :id
String :name
end
# routes
post '/api/baz' do
db[:foo].insert(params['fizz'])
end
and when trying to test it with wget --post-data 'fizz=buzz' http://localhost:4567/api/baz
, were seeing unexpected results to seemingly innocuous HTTP requests..
so we tried curl -d fizz=buzz -X POST http://localhost:4567/api/baz
, didn't have the same problem, and kept moving forward with the POC.
but something was wrong - and for ~once, it wasn't our code.
what is the problem?
(presumed/usually) regular expressions/parsing mechanisms that do not strictly follow RFC2616.
and less apocalyptically: Sinatra was following Ruby standards and returning the result of the last line of the route method - which in our case was the ID of the entry in the DB
tests
language | library | language version | library version | success? | context |
---|---|---|---|---|---|
binary | curl |
7.37.1 |
N/A | yes | |
binary | wget |
1.15 |
N/A | no | (number) ERROR -1: Malformed status line. , additionally: reported with exit code 4, 'Network failure.' |
java | java.net.UrlHTTPConnection |
1.7 |
1.7 |
yes/no | (number) body is conflated as HTTP status code, that even when invalid, no error is raised - so we raise our own |
perl | HTTP::Tiny |
5.10.0 , 5.18.2 |
0.025 |
no | (number) ERROR: 599 |
perl | LWP::UserAgent |
5.10.0 , 5.18.2 |
6.05 |
yes | |
php | http_get |
5.5.14 |
468225 |
no | (number) <unknown, bug in reporting> |
python | httplib2 |
2.7.5 |
2.7.5 |
no | (number) BadStatusLine: HTTP/1.1 36 |
python | urllib2 |
2.7.5 |
2.7 |
no | (number) BadStatusLine: HTTP/1.1 16 |
ruby | net-http |
2.2.2 |
2.2.2 |
no | (number) wrong status line: "HTTP/1.1 40 " |
599
, 36
, 16
and 40
are not error codes returned by the library/binary, but the random number returned by our number
route
running tests yourself
the entry point to all* tests is test/<language>/<library>/test.sh
which does some dependency checking and exits with 0 to avoid false positives.
most developer machines will already have all pre-requisites
* for tests that use binary files directly, the expected path changes to test/binary/<binary>/test.sh
pre-built gem installation (stable)
$ gem install 1863BFAF
...
$ echo $?
0
from-source installation (latest)
$ git clone https://github.com/chorankates/1863BFAF.git
$ cd 1836BFAF
$ bundle install
...
$ rake server
== Sinatra (v1.4.7) has taken the stage on 4567 for development with backup from WEBrick
$ rake test
...
failed[7]
tests[
binary/wget
java/java.net
perl/httptiny
php/http_get
python/httplib2
python/urllib2
ruby/net-https]
$ echo $?
0
$ rake test:perl test:java
...
$ echo $?
0
...
$ rake server:stop
== Sinatra has ended his set (crowd applauds)