/lisp-repl-core-dumper

Generate REPL-optimized Lisp cores on demand

Primary LanguageCommon LispGNU General Public License v3.0GPL-3.0

Do you fancy an instantly starting Common Lisp REPL? Even one that automatically includes your favourite libraries?

This tool is for you!

  • It’s portable and should work with any compiler. If your compiler is not supported, please report. (Or better, send a patch!)
  • It works for any REPL.
  • It allows you to include arbitrary libraries.

Simply load the script with a supported Lisp and it will dump an image that preloads ASDF and the user-provided extra libraries. You can then configure your REPL to use this image for an instant startup!

Installation

It’s a stand-alone, portable script: just download and run it!

You may want to install it in your PATH for convenience.

It’s also available as a Guix package.

Usage

See lisp-repl-core-dumper -h.

Example use:

lisp-repl-core-dumper -s 'alexandria serapeum' sbcl

generates an image (if necessary), including Alexandria and Serapeum, then drops you in the resulting REPL.

lisp-repl-core-dumper -s alexandria -d serapeum sbcl

As above, this generates an image that includes Alexadrian and all the dependencies of Serapeum, but not Serapeum itself.

From the REPL, call

(asdf:already-loaded-systems)

to list all pre-loaded systems.

Tested compilers

  • Steel Bank Common Lisp (SBCL)
  • Clozure Common Lisp (CCL)
  • CLISP

Some implementations such as Embedded Common Lisp (ECL) does not seem to support image dumping.

Emacs setup

While this can be used with any editor, here is a configuration snippet for Emacs user to save them some typing.

(let ((dumper "/path/to/lisp-repl-core-dumper"))
  (setq sly-lisp-implementations
        `((ccl (,dumper "ccl"))
          (ccl-with-alex (,dumper "ccl" "alexandria"))
          (clisp (,dumper "clisp"))
          (sbcl (,dumper "sbcl"))
          (sbcl-for-nyxt (,dumper "-d" "nyxt/gtk" "sbcl")))))

or, if lisp-repl-core-dumper is in your path:

(setq sly-lisp-implementations
      `((ccl ("lisp-repl-core-dumper" "ccl"))
        (ccl-with-alex ("lisp-repl-core-dumper" "ccl" "alexandria"))
        (clisp ("lisp-repl-core-dumper" "clisp"))
        (sbcl ("lisp-repl-core-dumper" "sbcl"))))
        (sbcl-for-nyxt ("lisp-repl-core-dumper" "-d" "nyxt/gtk" "sbcl"))))

Implementation notes

Why isn’t this an ASDF system?

(require "asdf") takes too much time.

The time spent checking for an up-to-date image adds further delay before the REPL is usable, so we want to make it the fastest possible.

We could have programmed it fully in Lisp, but then we would have needed a portable way to interact with files, run a program, etc. Which would amount to loading UIOP or reimplementing it, which would be too slow.

The POSIX shell code depends on standard Unix tools and does the image checking job in just a few lines.

How do we deal with core compatibility?

Simple: Make the core executable!

Previously we used to dump non-executable images that would be started by the original Lisp compiler (e.g. with sbcl --core /path/to/image) but this would break as soon the the sbcl binary would get updated (e.g. rebuild with different compilation flags).

Checking core compatibility turns out to be non-trivial. Comparing the executable hash is slow and unreliable (it may not hold the core compatibility information) while checking for runtime errors may not work. (How do we distinguish between image load errors and other errors?)

It’s just simpler to generate a stand-alone image!

Change log

0.7.0

  • Add -d option to preload system dependencies.
  • Rename -p option to -s.

0.6.0

  • Add -o option to specify the output path.

0.5.0

  • Fix issue with missing lisp-repl-core-dumper package. No such package is created anymore.
  • Fix issue with sb-rorate-byte missing when reloading systems that depend on it (like cl-utilities).

0.4.0

  • Pass packages via the -p option and start-flags via positional arguments.

References

hu.dwim has a a similar, more sophisticated, but less general script: https://hub.darcs.net/hu.dwim/hu.dwim.environment/browse/bin/build-development-image.sh