URL Shortening Service Example

Specification

A simple url shortening website, where users can submit a long url, and are given a shorter url in return.

Installation / Dependencies

Tested on Python 3.7. No external dependencies, but it does use python's sqlite3 module. There are two web servers included, a simple wsgi server, used for tests, and a proof-of-concept http server using asyncio.

Running Tests

Run ./run_tests.sh in the top level directory, or run python3 -m shortener.database, for example, to run tests in one file.

Running Service

Run python3 run_service.py, then open http://127.0.0.1:1729/ in your browser.

The full commandline is python3 run_service.py hostname:port database_file.db

Overview of files inside `shortener` module

service.py wsgi app for url shortening.

The ShortenerService calls into the Store to create and retrieve shortened urls.
database.py sqlite3 storage backend

The Store class is a small wrapper around sqlite3 to manage creating databases, and creating/deleting shortened urls
wsgiutil.py wsgi service boilerplate

There is a stub of an http server in here, used for tests, and a rudimentary wsgi application that delegates everything to a on_request method on the class.
asyncserver.py asyncio http service

This contains a bare-bones http parser, so things like "expect-continue" or HEAD requests with content-length headers will not be parsed correctly.

There is a simple http/1.0 server in here, which does not handle pipelining/keeep-alive, and closes the connection after every response.
testrunner.py simple test runner

This has a TestRunner class that takes a bunch of functions, runs them one after the other, and prints a report.
metrics.py simple logging

Two stub functions for reporting when a shortened url is created, and visited.

Addendum: Notes on production and scale.

This is not intended for production use. Frameworks, ORMs, have been avoided to ensure that the demonstration code is easy to run out of the box.

One thing of note: The shortened url is generated by using shake-256, a SHA-3 variant designed for giving variable-length hashes, while preseving pleasing properties like resistance to extension attacks.

Like with any web application, scaling involves caching results from the database, and partitioning the database too. Given the short-urls are relatively uniformly distributed, something like consistent hashing would work for partitioning the store.

tef/shortener