system

What is it?

system owes to component both in spirit and in name. The component library helps implement the reloaded pattern as championed by Stuart Sierra. system is built on top of it, offering a set of readymade components. The idea is to expand the set over time with contributions from the community. It currently includes:

Jetty (HTTP server)
HTTP kit (Async HTTP server)
aleph (Async HTTP server)
Immutant Web (Web component in application server suite)
Datomic (Immutable database)
H2 (H2 relational database)
PostgreSQL (SQL Database)
Monger (MongoDB client)
Neo4j (Graph database)
Sente (Websockets/Ajax communications library)
nREPL (Clojure network REPL )
Langohr (RabbitMQ client)
Etsy (Etsy client)
hara.io.watch (File watcher)
Elasticsearch (Elasticsearch full-text search engine)
Generic app component for typical Ring applications

Motivation

A good REPL experience is a prerogative of Lisp languages. Reloaded components enable this goodness in Clojureland. Since they require an amount of setup, the first steps when starting a new project are generally devoted to infrastructure. My first attempt to tackle the boilerplate was a Leiningen template. The problem is that Leiningen templates are hard to maintain and difficult to retrofit on existing projects. I was finding myself repeatedly updating the template for future use. Then it dawned on me that a library would better fit the bill. And so system came to light. It’s now the first dependency I add to any project, allowing me to focus from the get-go on the substance of my application.

Is it good?

Yes.

Installation

Add the following to your project’s dependencies.

ProTip: Snapshot versions may be available.

Example project

An example project is available under the examples folder. It is a self-contained project that will help you get started. Both the Leinigen and Boot toolchain are covered.

Usage

First, assemble your system(s) in a namespace of your choosing. Here we define two systems, development and production.

(ns my-app.systems
  (:require 
   [com.stuartsierra.component :as component]
   (system.components 
    [jetty :refer [new-web-server]]
    [repl-server :refer [new-repl-server]]
    [datomic :refer [new-datomic-db]]
    [mongo :refer [new-mongo-db]])
   [my-app.webapp :refer [handler]]
   [environ.core :refer [env]]))

(defn dev-system []
  (component/system-map
   :datomic-db (new-datomic-db (env :db-url))
   :mongo-db (new-mongo-db)
   :web (new-web-server (env :http-port) handler)))

(defn prod-system []
  "Assembles and returns components for an application in production"
  []
    (component/system-map
     :datomic-db (new-datomic-db (env :db-url))
     :mongo-db (new-mongo-db (env :mongo-url))
     :web (new-web-server (env :http-port) handler)
     :repl-server (new-repl-server (env :repl-port))))

Then, in user.clj:

(ns user
  (:require [reloaded.repl :refer [system init start stop go reset]]
            [my-app.systems :refer [dev-system]]))

(reloaded.repl/set-init! dev-system)

You can then manipulate the system in the REPL: (go), (reset) or (stop). The system map is accessible at any time, it resides in a var named, as you can guess, #'system.

In production, in core.clj:

(ns my-app.core
  (:gen-class)
  (:require [my-app.systems :refer [prod-system]]))

(defn -main 
  "Start the application"
  []
  (alter-var-root #'prod-system component/start)

Or, if you want to keep a handler on your system in production:

(ns my-app.core
  (:gen-class)
  (:require [reloaded.repl :refer [system init start stop go reset]]
            [my-app.systems :refer [prod-system]]))

(defn -main 
  "Start the application"
  []
  (reloaded.repl/set-init! prod-system)
  (go))

defsystem

A convenience macro, defsystem, allows you to declare systems succinctly:

(ns my-app.systems
  (:require 
   [system.core :refer [defsystem]]
   (system.components 
    [jetty :refer [new-web-server]]
    [repl-server :refer [new-repl-server]]
    [datomic :refer [new-datomic-db]]
    [mongo :refer [new-mongo-db]])
   [my-app.webapp :refer [handler]]
   [environ.core :refer [env]]))

(defsystem dev-system 
  [:datomic-db (new-datomic-db (env :db-url))
   :mongo-db (new-mongo-db)
   :web (new-web-server (env :http-port) handler)])

(defsystem prod-system 
  [:datomic-db (new-datomic-db (env :db-url))
   :mongo-db (new-mongo-db (env :mongo-url))
   :web (new-web-server (env :http-port) handler)
   :repl-server (new-repl-server (env :repl-port))])

Note: Component allows you to define dependency relationships within systems. Please don’t use said macro for those cases. Be sure to consult component’s API to see the range of options available to you.

At runtime: global system map vs dependency injection

At runtime, the system var can be used anywhere after requiring it from the reloaded.repl namespace:

(ns front-end.webapp.handler
 (:require [reloaded.repl :refer [system]]))
 
(code-using system ...)

Note this pattern of directly accessing the global system var is in contrast with the pattern of dependency injection integral to Stuart Sierra’s vision of Component. In this perspective, components are defined in terms of the components on which they depend. System, as a repository of readymade, reusable components, cannot and does not anticipate all the possible ways in which users will want to assemble components together. What it can and does, however, is anticipate common scenarii. Like your typical Ring application, for example, where the web server depends on routes and middleware, which in turn depend on a database.

As with many patterns, DI can be abused. It is easy to get carried away with dependency injection and build a towering dependency graph that is unnecessary and even counter-productive. — Ben Morris in How not to use dependency injection: service locators and injection mania.

Whatever you do, use your best judgment.

Boot-system

System and Boot are a match made in heaven. Some of the properties that boot-system brings to your workflow are:

Manual and automatic mode, ie. either you manipulate the system in the REPL, or you configure it to react to editing changes.
Restartable system. What warrants a system restart is user-configurable. File-based granularity.
Changes that do not require a restart are available in the running system instantly (via namespace reloading).
Full Lisp-style interactive programming via the REPL and hot-reloading in the browser.

The system task is invoked like any boot task.

$ boot system -h

Which outputs, for example:

-h, --help         Print this help info.
-s, --sys SYS      Set the system var to SYS.
-a, --auto-start   Auto-starts the system.
-r, --hot-reload   Enables hot-reloading.
-f, --files FILES  Conj FILES onto a vector of filenames. Restricts hot-reloading to that set.

A tutorial is available in a separate repository.

The Reloaded pattern

Here are a couple of links that are sure to shed more light on the motivations of the reloaded workflow.

The canonical reference: My Clojure Workflow, Reloaded

And more references touching on the topic.

Compatibility

There is a host of components libraries in the Clojure ecosystem, each with its own take, its own philosophy. For example:

Navigating this space can be difficult or overwhelming. Due to the nature of Open Source Software, it is unlikely to see any kind of standardization taking place. Let’s embrace the diversity instead, and emphasize the compatibility of components. As long as a component adheres to Stuart Sierra’s Lifecycle protocol, you can import it in your systems namespace and refer to it as any other native system component.

Choosing

To help choose if system is right for you, here are a couple of tips. Take a component for an often used dependency (a web server, for example, or a database), and compare their source code. The system library puts an emphasis on two properties:

minimalism: system provides a way to instantiate components that fulfill the Licecycle protocol (start and stop). Nothing more, nothing less.
Interactive programming: system is best used in a Lispy, interactive workflow, hence its deep integration with boot.

Contributing

Please fork and issue a pull request to add more components. Please don’t forget to include tests. You can refer to the existing ones to get started.

Credits

I wish to thank Stuart Sierra for the pioneering and guidance. Special thanks to James Reeves for the reloaded.repl library and general inspiration. Thanks to Peter Taoussanis, the friendly OSS contributor, who helped to ‘componentize’ sente, an amazing library on its own right.

License

Distributed under the Eclipse Public License, the same as Clojure.

kajism/system