0) Index ******** 0........Index 1........About 2........License 3........Compiling The Source 3.1....Feature Configuration 3.2....Make Targets 4........HTTP Server 4.1....Command line 4.2....Socket configuration 4.3....Virtual Host Configuration 4.4....Request path resolution 4.5....HTTPS support 4.6....Privilege Dropping 5........Database Server 6........JSON Parser & Serializer 7........Session managament 1) About ******** This directory contains a REST framework/HTTP server written in C89, but utilizing Linux specific system calls and GNU library extensions to the utmost extent possible to keep the server small, simple and efficient. I developed this framework for university courses and private projects that required me to build a REST service or other web applications, after being annoyed with Tomcat+Jax-RS or similarly heavy weight C++ frameworks. Currently supported features: - Supports multiple virtual host configurations - Supports binding to different addresses/ports - Supports IPv4, IPv6 and UNIX domain sockets - Supports HTTP GET, HEAD, POST, PUT and DELETE methods - Supports HTTP keep-alive - Supports cookies - Guesses content-type from static files using file extension - Serve gzip compressed, static files if the client supports it - Dynamically generated data can be compressed on the fly - Use zero-copy splice syscall to serve static files - Tell clients to do caching on static files - Propperly handle If-Modified requests for static files - Map a specific path to REST service callback - Includes a simple JSON parser that generates C-structs from JSON - Includes a simple JSON serializer that genereats JSON from C-structs - Simple document/page templateing system - Minimal SQLite based data base server - Built-in session management - Hard timeouts and limits on requests to guarntee that request handling terminates - Gracefully handles segfaults when serving requests. Log the request and send an error page. - Configurable featureset, so the server can be only used to only server static files etc... However, there are currently a few limitations: - Only absolute paths supported - No directory views (requests to directories are rejected) 2) License ********** For proprietary licensing options, please contact the author. The copy of the source code of the HTTP and Database servers that you received (all source files included in this directory and sub directories) are subject to the terms and conditions of the GNU AFFERO GENERAL PUBLIC LICENSE version 3. See LICENSE file for details. 3) Compiling The Source *********************** The GNU build system, aka autotools are required for compiling the server binaries (that plus make and a C compiler). The data base server requires the SQLite library. The HTTP server requires the zlib library for compression of dynamic pages. The programs make heavy use of GNU library extensions and Linux specific system calls, so they should be available in the systems C standard library. 3.1) Feature Configuration The following command line switches can be supplied when running the configure script to tweak the HTTP & data base server binaries: --disable-json-parser Compile the server without JSON-to-C-struct deserializer. --disable-json-serializer Compile the server without C-struct-to-JSON serializer. --disable-session Compile the database server without session managament. --disable-rest Compile the server without dynamic page / rest API backend. --disable-static Compile the server without static file backend. --with-client-timeout=<number> Specify the maximum number of milli seconds to wait for a client to send a request. Default is 2000 (= 2 seconds). --with-request-timeout=<number> Specify the maximum number of seconds for handling a request. A SIGALRM is used to interrupt request handling if it takes longer than this and an error page is sent to the client. Default is 5 seconds. --with-download-timeout=<number> Specify the maximum number of seconds for processing a request for a static file. A SIGALRM is used to interrupt request handling if it takes longer than this and an error page is sent to the client. Default is 7200 seconds (= 2h, roughly enough to download 48 MiB on a 56k connection). --with-requests=<number> Specify the maximum number of requests to handle per TCP connection for HTTP pipelining/keep-alive. Default is 1000. 3.2) Make Targets Apart from the default automake make targets, a custom 'strip' target has been added for stripping the output binaries. The 'cert' target can be used to generate a simple SSL certificate for use with stunnel. 4) HTTP Server ************** 4.1) Command line To run the HTTP server, start it with the following command line arguments: -c, --cfg <configfile> Specify a configuration file with virtual host configuration. This option is required. -f, --log <file> Append logging output to a specific file -l, --loglevel <num> Higher level means more detailed/verbose output. Possible values are: 0 - only print critical errors 1 - log critical errros and warnings (DEFAULT) 2 - also log generic information 3 - debug output -r, --chroot <path> Filesystem root directory for the server Note: The chroot is switched into before reading the configuration file, but the log file is created before switching into the chroot. The path for the configuration file has to be relative to the chroot directory (if specified). The path is relative to the actual root. The config file for the HTTP server is an .ini file with multiple "host" sections for each virtual hosts and multiple "ipv4", "ipv6" and "unix" sections for sockets to create. A sample configuration is included with the source code. The HTTP server can be forced to reload its configuration by sending it a SIGHUP signal. 4.2) Socket configuration Each "ipv4" and "ipv6" section in the config file must have a "bind" and "port" field specifying what address ("ANY" for all incoming addresses) and port to bind to. The "unix" section only requires a "bind" argument for the unix socket path to bind to. Note: IPv6 sockets only accept IPv6 traffic. If a dual stack setup is required, both IPv4 and IPv6 bindings have to be specified explicitly. 4.3) Virtual Host Configuration Each "host" section in the config file can have the folowing keys: hostname The value of the HTTP "Host:" request field to map. restdir An optional, virtual directory to map to the internal callback system. datadir An optional directory from which to server static files. templatedir An optional directory from which to load document template files. rootfile If the document root ("/") is requested, rewrite the request path to this path (may be underneath the restdir or an oridnary file) 4.4) Request path resolution When a client requests a file, the server first tries to resolve it via the dynamic backend. If that backend also returns a 404 response, the server tries to serve a static file instead. This means, that the dynamic backend can override static files, even on a forwarded directory, if it wants to. When resolving a static file and the client supports gzip compression, the server tries to append ".gz" to the path (unless the path already ends with *.gz). If that file exists, the server returns it _instead_ of the actually requested file and tells the client that it is gzip compressed. If that fails, the server tries to open the actual path and transfers that file. This allows serving precompressed files on the server if the client supports it, with a fallback to uncompressed files. If the requested path referes to a directory (or _anything_ other than a regular file), the server returns a "403 Forbidden" response. The server only tries to serve the index file, if the root document is requested (e.g. "GET /"). Serving the index file for arbitrary sub-directories, or serving directory views is not supported. 4.5) HTTPS support The HTTP server itself does not support TLS connections. To support HTTPS connections, a TLS reverse proxy is required that redirects the unencrypted traffic to the HTTP server internally. A simple demo configuration is provided on how to use stunnel as an TLS wrapper on top of the HTTP server. Run "make cert" to generate private key, certificate and DH parameters for the server. Then run "stunnel ./tls.conf" to start up an TLS server that accepts connections on port 4040, servers the generated certificate and redirects the raw data to a Unix socket that the HTTP server provides. 4.6) Privilege Dropping In the server configuration file, an optional section "user" can be specified with the numeric values "uid" and "gid" to specify values for the real, effective and saved user/group IDs that the server should assume. Here is an example of what that section might look like: [user] uid = 1000 # Set real/effective/saved user ID to 1000 gid = 100 # Set real/effective/saved group ID to 100 In addition, the command line argument "--chroot <path>" can be used to specify a directory that the server should chroot into before changing its user and group IDs. No files are accessed before the chroot, so all file paths (config file, unix socket paths, etc...) must be specified relative to this directory. Only the logfile is created before entering the chroot and can thus be specified outside the chroot directory. 5) Database Server ****************** The database server implements a simple broker model. It receives a very simple protocol over a UNIX domain socket and retrieves data from an SQLite database. Buffer bounds and sanity of transfered values must be thoroughly checked when accessing the payload from either side! For a _clean_ and _secure_ seperation, the HTTP server itself does not transfer acutal SQL queries to the DB server; it sends application specific request tokens that the DB server translates to SQL queries. Idealy, the HTTP server and DB server are run as different users that cannot access each others files, thus allowing the HTTP server to only access data that it is supposed to. To run the databse server, start it with the following command line arguments: -d, --db <dbfile> Specify the data base file to access -s, --sock <unixsocket> Specify the UNIX socket to listen on -f, --log <file> Append logging output to a specific file -l, --loglevel <num> Higher level means more detailed/verbose output. Possible values are: 0 - only print critical errors 1 - log critical errros and warnings (DEFAULT) 2 - also log generic information 3 - debug output 6) JSON Parser & Serializer *************************** A parser for a limited subset of JSON is provided. It deserializes JSON directly to C structures in memory using a describtion table that can be generated semiautomatically via the preprocessor. The parser currently has the following limitations: - The parser only supports arrays of objects (not strings or integers) - Booleans are internally treated as integers - Floating point values are not supported - A JSON document must contain either an array or an object at the root - Objects and arrays must not be empty (i.e. contain at least one element) - Unknown JSON fields are skipped. Skiped fields may contain JSON values without any restrictions to the JSON grammar. In addition to the parser, a serializer is provided that can convert C structs directly to JSON strings. 7) Session managament ********************* If configured to do so, the database server can manage sessions in an in-memory list. The HTTP server has wrapper functions (see user.h) for managing sessions that talk to the database server. A session has a unique, non-zero, randomly generated 32 bit session ID that is (preferably) stored in a cookie called "session" in base 32 representation. For each session, the database server stores an asociated 32 bit user ID and timestamp when the session has been last accessed. It can be easily modified to store additional data. If a session has not been accessed for a certain amount of time (default 10 minutes, see session.h), it is removed. Since the sessions are not stored on a persistent medium, all sessions are lost if the database server is stopped. The same user can have multiple sessions at a time. Idealy, one would store a persisten user list in the database and verify login credentials before handing out sessions. For simplicity, the session demo lets the user pick a user ID and creates a session.