/emacs-package-dev-handbook

An Emacs package development handbook. Built with Emacs, by Emacs package developers, for Emacs package developers.

Primary LanguageJavaScriptGNU General Public License v3.0GPL-3.0

The Emacs Package Developer’s Handbook

After developing some little Emacs packages for a year or so, I began to notice how I’d forget little things that I learned, and then I’d have to go hunting for that information again. I also noticed how there are some issues for which there doesn’t seem to be a “best practice” or “Standard Operating Procedure” to refer to.

So this is intended to be a place to collect and organize information related to Emacs package development. Built with Emacs, by Emacs package developers, for Emacs package developers.

You can read this Org file directly on the repository rendered by GitHub (which lacks support for some features of the document, such as links between sections), or you can read the HTML version.

Note: The primary sections are listed at the top of the page in the horizontal bar.

Contents

Emacs Lisp

Note: Usable Emacs Lisp code snippets (not examples) are tangled to the file epdh.el, which may be found in the repository. You could even install the file as a package with quelpa-use-package, like this:

(use-package epdh
  :quelpa (epdh :fetcher github :repo "alphapapa/emacs-package-dev-handbook"))

Animations / Screencasts

Tools

Most Emacs screencasts are done by first recording a video, then converting it to a GIF. It seems like a waste of resources considering the little animation Emacs features.

Most of the time, one user action equals one visual change. By exploiting this fact, this package tailors GIF-screencasting for Emacs and captures one frame per user action. It’s much more efficient than capturing frames even at a frequency as low as 10 per second. You could roughly expect 3 to 10 times smaller files compared to videos of the same quality. (Benchmarks are welcome.)

Another neat perk of action-based captures is that it produces editable GIF files: programs such as Gifsicle or The GIMP can be used to edit the delays or rearrange frames. This is particularly useful to fix mistakes in “post-processing”, without having to record the demo all over again.

The author of this document can vouch for the fact that this package is the easiest, most powerful way to make screencast animations in Emacs!

While emacs-gif-screencast should usually be your first choice, the way it works, recording one frame per Emacs command, isn’t suitable for every case. For general use, your editor can recommend Bashcaster, which is an easy-to-use script that can record the whole screen or individual windows, to videos or GIFs.

Asynchronicity

See Multiprocessing (generators, threads).

Binding

Information related to variable scope and binding in elisp code (e.g. lexical vs. dynamic scope).

Articles

:archive.today: http://archive.today/T8dHM

Chris Wellons explains how the old cl macro flet changes in its new cl-lib version, cl-flet, and how to use cl-letf to achieve the old functionality. It’s a way to override functions in both lexical and dynamic scope, which is especially useful for unit testing.

Libraries

dash.el

Thunk provides functions and macros to delay the evaluation of forms.

Use thunk-delay to delay the evaluation of a form (requires lexical-binding), and thunk-force to evaluate it. The result of the evaluation is cached, and only happens once.

Here is an example of a form which evaluation is delayed:

(setq delayed (thunk-delay (message "this message is delayed")))

delayed is not evaluated until thunk-force is called, like the following:

(thunk-force delayed)

This file also defines macros thunk-let and thunk-let* that are analogous to let and let* but provide lazy evaluation of bindings by using thunks implicitly (i.e. in the expansion).

Tools

Lexical binding

Buffers

Contents

Best practices

Accessing buffer-local variables

It’s much faster to use buffer-local-value than with-current-buffer to access the value of a variable in a buffer.

(bench-multi :times 1000 :ensure-equal t
  :forms (("buffer-local-value" (--filter (equal 'magit-status-mode (buffer-local-value 'major-mode it))
                                          (buffer-list)))
          ("with-current-buffer" (--filter (equal 'magit-status-mode (with-current-buffer it
                                                                       major-mode))
                                           (buffer-list)))))
Formx faster than nextTotal runtime# of GCsTotal GC runtime
buffer-local-value50.340.04765773400.0
with-current-bufferslowest2.39932345200.0

Inserting strings

Inserting strings into buffers with insert is generally fast, but it can slow down in buffers with lots of markers or overlays. In general, it can be faster to insert one large string (which may include newlines). For example:

(let ((strings (cl-loop for i from 1 to 1000
                        collect (number-to-string i))))
  (garbage-collect)
  (--sort (< (caddr it) (caddr other))
          (cl-loop for times in '(1 10 100)
                   append (a-list "(loop do (insert ..."
                                  (cons times
                                        (benchmark-run-compiled times
                                          (with-temp-buffer
                                            (cl-loop for string in strings
                                                     do (insert string)))))
                                  "(apply #'insert ..."
                                  (cons times
                                        (benchmark-run-compiled times
                                          (with-temp-buffer
                                            (apply #'insert strings))))
                                  "(insert (apply #'concat ..."
                                  (cons times
                                        (benchmark-run-compiled times
                                          (with-temp-buffer
                                            (insert (apply #'concat strings)))))))))

#+RESULTS[aa866ca87dbf71476c735ed51fca7373934bbf4f]:

(insert (apply #’concat …1000.00014208500.0
(insert (apply #’concat …100.00016117200.0
(insert (apply #’concat …10.0001876400.0
(apply #’insert …100.00066547200.0
(apply #’insert …1000.00067847100.0
(apply #’insert …10.00075532900.0
(loop do (insert …100.00081703100.0
(loop do (insert …1000.00086977900.0
(loop do (insert …10.00149039700.0

The fastest method here is to call insert once with the result of calling concat once, using apply to pass all of the strings. With 100 iterations, it’s about 6x faster than the next-fastest method, and even with 1 iteration, it’s over 2x faster.

Libraries

BUI (Buffer User Interface) is an Emacs library that can be used to make user interfaces to display some kind of entries (like packages, buffers, functions, etc.). The intention of BUI is to be a high-level library which is convenient to be used both by:

package makers, as there is no need to bother about implementing routine details and usual features (like buffer history, filtering displayed entries, etc.);

users, as it provides familiar and intuitive interfaces with usual keys (for moving by lines, marking, sorting, switching between buttons); and what is also important, the defined interfaces are highly configurable through various generated variables.

Checkers / linters

Flycheck-package

Collections (lists, vectors, hash-tables, etc.)

Contents

Best practices

Collecting items into a list

Here are some examples of fast ways to collect items into a list.

(bench-multi-lexical :times 500000 :ensure-equal t
  :forms (("cl-loop" (let ((l '(1 2 3 4)))
                       (cl-loop for val in l
                                collect val)))
          ("push-nreverse with setf/pop" (let ((l '(1 2 3 4))
                                              val r)
                                          (while (setf val (pop l))
                                            (push val r))
                                          (nreverse r)))
          ("push-nreverse with when-let*/pop" (let ((l '(1 2 3 4))
                                                    r)
                                                (while (when-let* ((val (pop l)))
                                                         (push val r)))
                                                (nreverse r)))
          ("nconc with when-let*/pop" (let ((l '(1 2 3 4))
                                            r)
                                        (while (when-let* ((val (pop l)))
                                                 (setf r (nconc r (list val)))))
                                        r))
          ("nconc with setf/pop" (let ((l '(1 2 3 4))
                                            val r)
                                        (while (setf val (pop l))
                                          (setf r (nconc r (list val))))
                                        r))))
Formx faster than nextTotal runtime# of GCsTotal GC runtime
cl-loop1.010.15457800
push-nreverse with setf/pop1.020.15593000
push-nreverse with when-let*/pop1.230.15921100
nconc with setf/pop1.060.19568500
nconc with when-let*/popslowest0.20710300

As is usually the case, the cl-loop macro expands to the most efficient code, which uses (setq val (car ..., push, and nreverse:

(cl-block nil
  (let* ((--cl-var-- l)
         (val nil)
         (--cl-var-- nil))
    (while (consp --cl-var--)
      (setq val (car --cl-var--))
      (push val --cl-var--)
      (setq --cl-var-- (cdr --cl-var--)))
    (nreverse --cl-var--)))

However, in some cases cl-loop may expand to code which uses nconc, which, as the benchmark shows, is much slower. In that case, you may write the loop without cl-loop to avoid using nconc.

Diffing two lists

As expected, seq-difference is the slowest, because it’s a generic function that dispatches based on the types of its arguments, which is relatively slow in Emacs. And it’s not surprising that cl-nset-difference is generally slightly faster than cl-set-difference, since it’s destructive.

However, it is surprising how much faster -difference is than cl-nset-difference.

It’s also nonintuitive that -difference suffers a large performance penalty by binding -compare-fn (the equivalent of the :test argument to cl-set-difference): while one might expect that setting it to string= would give a slight performance increase, it’s actually faster to let -difference use its default, equal.

Note that since this benchmark compares lists of strings, cl-nset-difference requires setting the :test argument, since it uses eql by default, which does not work for comparing strings.

(defmacro test/set-lists ()
  `(setf list1 (cl-loop for i from 0 below 1000
                        collect (number-to-string i))
         list2 (cl-loop for i from 500 below 1500
                        collect (number-to-string i))))

(let (list1 list2)
  (bench-multi-lexical :times 10 :ensure-equal t
    :forms (("-difference"
             (progn
               (test/set-lists)
               (-difference list1 list2)))
            ("-difference string="
             (progn
               ;; This is much slower because of the way `-contains?'
               ;; works when `-compare-fn' is non-nil.
               (test/set-lists)
               (let ((-compare-fn #'string=))
                 (-difference list1 list2))))
            ("cl-set-difference equal"
             (progn
               (test/set-lists)
               (cl-set-difference list1 list2 :test #'equal)))
            ("cl-set-difference string="
             (progn
               (test/set-lists)
               (cl-set-difference list1 list2 :test #'string=)))
            ("cl-nset-difference equal"
             (progn
               (test/set-lists)
               (cl-nset-difference list1 list2 :test #'equal)))
            ("cl-nset-difference string="
             (progn
               (test/set-lists)
               (cl-nset-difference list1 list2 :test #'string=)))
            ("seq-difference"
             (progn
               (test/set-lists)
               (seq-difference list1 list2)))
            ("seq-difference string="
             (progn
               (test/set-lists)
               (seq-difference list1 list2 #'string=))))))
Formx faster than nextTotal runtime# of GCsTotal GC runtime
-difference7.160.08448400
cl-nset-difference equal1.050.60519300
cl-set-difference string=1.010.63697300
cl-set-difference equal1.010.64491900
cl-nset-difference string=1.190.65070800
-difference string=1.590.77391900
seq-difference1.051.23261600
seq-difference string=slowest1.29303000

Filtering a list

Using -select from dash.el seems to be the fastest way:

(let ((list (cl-loop for i from 1 to 1000
                     collect i)))
  (bench-multi :times 100
    :ensure-equal t
    :forms (("(-non-nil (--map (when ..." (-non-nil
                                           (--map (when (cl-evenp it) it) list)))
            ("(delq nil (--map (when ..." (delq nil
                                                (--map (when (cl-evenp it) it) list)))
            ("cl-loop" (cl-loop for i in list
                                when (cl-evenp i)
                                collect i))
            ("-select" (-select #'cl-evenp list))
            ("cl-remove-if-not" (cl-remove-if-not #'cl-evenp list))
            ("seq-filter" (seq-filter #'cl-evenp  list)))))

#+RESULTS[6b2e97c1ebead84a53fd771684cc3e155e7f6b1e]:

Formx faster than nextTotal runtime# of GCsTotal GC runtime
-select1.170.0154039100.0
cl-loop1.050.0180822600.0
seq-filter1.130.0189170800.0
(delq nil (–map (when …1.150.0213472700.0
cl-remove-if-not1.180.0245947800.0
(-non-nil (–map (when …slowest0.0290399900.0

Examples

Alists

Creation
;;;; Built-in methods

(list (cons 'one 1) (cons 'two 2))  ;; => ((one . 1) (two . 2))

'((one . 1) (two . 2))  ;; => ((one . 1) (two . 2))

(let ((numbers (list)))
  (map-put numbers 'one 1)
  (map-put numbers 'two 2))  ;; => ((two . 2) (one . 1))


;;;; Packages

;; `a-list' from a.el is the best way to make a new alist.

(a-list 'one 1
        'two 2)  ;; => ((one . 1) (two . 2))
Adding to
Single elements
;;;; Built-in methods

;; `map-put' is the best built-in way.  Requires Emacs 25.1+.

(let ((numbers (list (cons 'one 1))))
  (map-put numbers 'two 2)
  numbers)  ; => ((two . 2) (one . 1))

;; More primitive methods

;; Not recommended, but not too complicated:
(let ((numbers (list (cons 'one 1)))
      (more-numbers (a-list 'two 2
                            'three 3)))
  (append numbers more-numbers)) ;; => ((one . 1) (two . 2) (three . 3))

;; Don't do it this way, but it does demonstrate list/cons-cell
;; structure:
(let ((numbers (list (cons 'one 1))))
  (cons (cons 'three 3)
        (cons (cons 'two 2)
              numbers)))  ;; => ((three . 3) (two . 2) (one . 1))
Multiple elements
;;;; Built-in methods

;; `map-merge': if you're restricted to built-in packages, this works
;; well (requires Emacs 25.1+):
(let ((numbers (list (cons 'one 1)))
      (more-numbers (a-list 'two 2
                            'three 3)))
  (map-merge 'list numbers more-numbers))  ;; => ((three . 3) (two . 2) (one . 1))


;; Without map.el, you could use `append':
(let ((numbers (list (cons 'one 1)))
      (more-numbers (a-list 'two 2
                            'three 3)))
  (append numbers more-numbers)) ;; => ((one . 1) (two . 2) (three . 3))

;;;; Packages

;; `a-merge' from a.el is probably the best way:
(let ((numbers (list (cons 'one 1)))
      (more-numbers (a-list 'two 2
                            'three 3)))
  (a-merge numbers more-numbers))  ;; => ((three . 3) (two . 2) (one . 1))

Libraries

This library provides a consistent and comprehensive set of functions for working with hash tables: they’re named consistently, take a natural and consistent argument order, and cover operations that the standard Emacs functions don’t.

Similar to dash.el, but with slightly different behavior that may be useful, and some unique features. These functions are provided:

make-tconclist-utils-depth
tconc-plist-utils-flat-length
tconc-listlist-utils-flatten
tconclist-utils-alist-or-flat-length
list-utils-cons-cell-plist-utils-alist-flatten
list-utils-cyclic-lengthlist-utils-insert-before
list-utils-improper-plist-utils-insert-after
list-utils-make-proper-copylist-utils-insert-before-pos
list-utils-make-proper-inplacelist-utils-insert-after-pos
list-utils-make-improper-copylist-utils-and
list-utils-make-improper-inplacelist-utils-not
list-utils-linear-plist-utils-xor
list-utils-linear-subseqlist-utils-uniq
list-utils-cyclic-plist-utils-dupes
list-utils-cyclic-subseqlist-utils-singlets
list-utils-make-linear-copylist-utils-partition-dupes
list-utils-make-linear-inplacelist-utils-plist-reverse
list-utils-safe-lengthlist-utils-plist-del
list-utils-safe-equal

map is included with Emacs, but the latest version, which may include improvements since the last Emacs release, is now available separately on GNU ELPA.

stream.el provides an implementation of streams, implemented as delayed evaluation of cons cells.

Functions defined in seq.el can also take a stream as input.

Streams could be created from any sequential input data:

  • sequences, making operation on them lazy
  • a set of 2 forms (first and rest), making it easy to represent infinite sequences
  • buffers (by character)
  • buffers (by line)
  • buffers (by page)
  • IO streams
  • orgmode table cells

Tools

let-alist

=with-dict=, =with-plist-vals=

Data structure

Articles

:archive.today: http://archive.today/YxwP5

Date / Time

Libraries

ts aids in parsing, formatting, and manipulating timestamps.

ts is a date and time library for Emacs. It aims to be more convenient than patterns like (string-to-number (format-time-string "%Y")) by providing easy accessors, like (ts-year (ts-now)).

To improve performance (significantly), formatted date parts are computed lazily rather than when a timestamp object is instantiated, and the computed parts are then cached for later access without recomputing. Behind the scenes, this avoids unnecessary (string-to-number (format-time-string... calls, which are surprisingly expensive.

Examples

Get parts of the current date:

;; When the current date is 2018-12-08 23:09:14 -0600:
(ts-year (ts-now))       ;=> 2018
(ts-month (ts-now))      ;=> 12
(ts-day (ts-now))        ;=> 8
(ts-hour (ts-now))       ;=> 23
(ts-minute (ts-now))     ;=> 9
(ts-second (ts-now))     ;=> 14
(ts-tz-offset (ts-now))  ;=> "-0600"

(ts-dow (ts-now))        ;=> 6
(ts-day-abbr (ts-now))   ;=> "Sat"
(ts-day-name (ts-now))   ;=> "Saturday"

(ts-month-abbr (ts-now)) ;=> "Dec"
(ts-month-name (ts-now)) ;=> "December"

(ts-tz-abbr (ts-now))    ;=> "CST"

Increment the current date:

;; By 10 years:
(list :now (ts-format)
      :future (ts-format (ts-adjust 'year 10 (ts-now))))
;;=> (   :now "2018-12-15 22:00:34 -0600"
;;    :future "2028-12-15 22:00:34 -0600")

;; By 10 years, 2 months, 3 days, 5 hours, and 4 seconds:
(list :now (ts-format)
      :future (ts-format
               (ts-adjust 'year 10 'month 2 'day 3
                          'hour 5 'second 4
                          (ts-now))))
;;=> (   :now "2018-12-15 22:02:31 -0600"
;;    :future "2029-02-19 03:02:35 -0600")

What day of the week was 2 days ago?

(ts-day-name (ts-dec 'day 2 (ts-now)))             ;=> "Thursday"

;; Or, with threading macros:
(thread-last (ts-now) (ts-dec 'day 2) ts-day-name) ;=> "Thursday"
(->> (ts-now) (ts-dec 'day 2) ts-day-name)         ;=> "Thursday"

Get timestamp for this time last week:

(ts-unix (ts-adjust 'day -7 (ts-now)))
;;=> 1543728398.0

;; To confirm that the difference really is 7 days:
(/ (- (ts-unix (ts-now))
      (ts-unix (ts-adjust 'day -7 (ts-now))))
   86400)
;;=> 7.000000567521762

;; Or human-friendly as a list:
(ts-human-duration
 (ts-difference (ts-now)
                (ts-dec 'day 7 (ts-now))))
;;=> (:years 0 :days 7 :hours 0 :minutes 0 :seconds 0)

;; Or as a string:
(ts-human-format-duration
 (ts-difference (ts-now)
                (ts-dec 'day 7 (ts-now))))
;;=> "7 days"

;; Or confirm by formatting:
(list :now (ts-format)
      :last-week (ts-format (ts-dec 'day 7 (ts-now))))
;;=> (      :now "2018-12-08 23:31:37 -0600"
;;    :last-week "2018-12-01 23:31:37 -0600")

The primary function provided is: (datetime-format SYM-OR-FMT &optional TIME &rest OPTION)

(datetime-format "%Y-%m-%d")  ;=> "2018-08-22"
(datetime-format 'atom)  ;=> "2018-08-22T18:23:47-05:00"
(datetime-format 'atom "2112-09-03 00:00:00" :timezone "UTC")  ;=> "2112-09-03T00:00:00+00:00"

There are several other symbols provided besides atom, such as rfc-3339, which formats dates according to that RFC.

Debugging

Tools

Edebug

Edebug is a built-in stepping debugger in Emacs. It’s thoroughly documented in the elisp manual.

Declaring debug forms for keyword arguments

Declaring debug forms for functions and macros that take keyword arguments can be confusing. Here’s a contrived example:

(cl-defmacro make-fn (name docstring &key args bindings body)
  (declare (indent defun)
           (debug (&define symbolp stringp
                           &rest [&or [":body" def-form] [keywordp listp]])))
  `(defun ,name ,args
     ,docstring
     (let* ,bindings
       ,body)))

(make-fn my-fn
  "This is my function."
  :bindings ((one 1)
             (two 2))
  :body (list one two))
Submit this as an improvement to the Elisp manual

Probably should first replace the :bindings part with this, which correctly matches let bindings: (&rest &or symbolp (gate symbolp &optional def-form)).

Destructuring

See Pattern matching.

Documentation

Tools

This is helpful when exporting Org files to Info manuals.

This package provides some extensions for Org’s texinfo exporter defined in ox-texinfo.

  1. Create @deffn and similar definition items by writing list items in Org that look similar to what they will look like in Info.
  2. Optionally share a section’s node with some or all of its child sections.
  3. Optionally modify the Org file before exporting it.
  4. Fully respect the local value of indent-tabs-mode from the Org file when editing source blocks and exporting. This affects all source blocks and all exporters.

Editing

Tools

This package, by Damien Cassou and Matus Goljer, helps navigation by redefining the M-< and M-> keys do, depending on the major-mode.

iedit makes it easy to rename symbols within a function or in a whole buffer. Simply activate iedit-mode with point on a symbol, and it will be highlighted in the chosen scope, and any changes you make to the symbol are made in each highlighted occurrence. It’s like a smart, purposeful version of multiple-cursors.

The editor of this handbook uses iedit with these customizations:

ap/iedit-or-flyspell
Globally bound to C-;. In a prog-mode-derived buffer, either corrects the last misspelled word with flyspell when point is in a comment or string, or activates iedit-mode. In non- prog-mode-derived buffers, corrects with flyspell.
(defun ap/iedit-or-flyspell ()
  "Call `iedit-mode' or correct misspelling with flyspell, depending..."
  (interactive)
  (if (or iedit-mode
          (and (derived-mode-p 'prog-mode)
               (not (or (nth 4 (syntax-ppss))
                        (nth 3 (syntax-ppss))))))
      ;; prog-mode is active and point is in a comment, string, or
      ;; already in iedit-mode
      (call-interactively #'ap/iedit-mode)
    ;; Not prog-mode or not in comment or string
    (if (not (equal flyspell-previous-command this-command))
        ;; FIXME: This mostly works, but if there are two words on the
        ;; same line that are misspelled, it doesn't work quite right
        ;; when correcting the earlier word after correcting the later
        ;; one

        ;; First correction; autocorrect
        (call-interactively 'flyspell-auto-correct-previous-word)
      ;; First correction was not wanted; use popup to choose
      (progn
        (save-excursion
          (undo)) ; This doesn't move point, which I think may be the problem.
        (flyspell-region (line-beginning-position) (line-end-position))
        (call-interactively 'flyspell-correct-previous-word-generic)))))
ap/iedit-mode
Calls iedit-mode with function-local scope by default, or global scope when called with a universal prefix.
(defun ap/iedit-mode (orig-fn)
  "Call `iedit-mode' with function-local scope by default, or global scope if called with a universal prefix."
  (interactive)
  (pcase current-prefix-arg
    ('nil (funcall orig-fn '(0)))
    ('(4) (funcall orig-fn))
    (_ (user-error "`ap/iedit-mode' called with prefix: %s" prefix))))

;; Override default `iedit-mode' function with advice.
(advice-add #'iedit-mode :around #'ap/iedit-mode)
Helpful minibuffer message
Confirms when an iedit session has started.
(advice-add 'iedit-mode :after (lambda (&optional ignore)
                                 (when iedit-mode
                                   (minibuffer-message "iedit session started. Press C-; to end."))))
Refer to version published in unpackaged.el

Functions

Including anonymous functions (lambdas).

Articles

:archive.today: http://archive.today/ppIuJ

General

Libraries

This is the built-in cl-lib package which implements Common Lisp functions and control structures for Emacs Lisp.

Dash is a powerful general-purpose library that provides many useful functions and macros.

Less commonly used functions that complement basic APIs, often implemented in C code (like hash-tables and strings), and are not eligible for inclusion in subr.el.

This is a built-in package that provides several useful functions and macros, such as thread-first / last, if-let / when-let, hash-table functions, and string functions. It’s easy to forget about this, since:

Do not document these functions in the lispref. http://lists.gnu.org/archive/html/emacs-devel/2014-01/msg01006.html

Tools

This package may be especially helpful for developing in one’s own environment and testing in another, like default Emacs, Spacemacs, etc.

Chemacs is an Emacs profile switcher, it makes it easy to run multiple Emacs configurations side by side. Think of it as a bootloader for Emacs.

Emacs configuration is either kept in a ~/.emacs file or, more commonly, in a ~/.emacs.d directory. These paths are hard-coded. If you want to try out someone else’s configuration, or run different distributions like Prelude or Spacemacs, then you either need to swap out ~/.emacs.d, or run Emacs with a different $HOME directory set. This last approach is quite common, but has some real drawbacks, since now packages will no longer know where your actual home directory is.

All of these makes trying out different Emacs configurations and distributions needlessly cumbersome. Various approaches to solving this have been floated over the years. There’s an Emacs patch around that adds an extra command line option, and various examples of how to add a command line option in userspace from Emacs Lisp.

Chemacs tries to implement this idea in a user-friendly way, taking care of the various edge cases and use cases that come up.

Byte-compile and load directory

Byte-compile and load all elisp files in DIRECTORY. Interactively, directory defaults to default-directory and asks for confirmation.

;;;###autoload
(defun epdh/byte-compile-and-load-directory (directory)
  "Byte-compile and load all elisp files in DIRECTORY.
Interactively, directory defaults to `default-directory' and asks
for confirmation."
  (interactive (list default-directory))
  (if (or (not (called-interactively-p))
          (yes-or-no-p (format "Compile and load all files in %s?" directory)))
      ;; Not sure if binding `load-path' is necessary.
      (let* ((load-path (cons directory load-path))
             (files (directory-files directory 't (rx ".el" eos))))
        (dolist (file files)
          (byte-compile-file file 'load)))))

emacs-lisp-macroreplace

Replace macro form before or after point with its expansion.

;;;###autoload
(defun epdh/emacs-lisp-macroreplace ()
  "Replace macro form before or after point with its expansion."
  (interactive)
  (if-let* ((beg (point))
            (end t)
            (form (or (ignore-errors
                        (save-excursion
                          (prog1 (read (current-buffer))
                            (setq end (point)))))
                      (ignore-errors
                        (forward-sexp -1)
                        (setq beg (point))
                        (prog1 (read (current-buffer))
                          (setq end (point))))))
            (expansion (macroexpand-all form)))
      (setf (buffer-substring beg end) (pp-to-string expansion))
    (user-error "Unable to expand")))

Highlighting / font-locking

Packages

Packages that do highlighting/font-locking.

Tools

Tools for developing highlighting/font-locking packages.

Multiprocessing (generators, threads)

Articles

:archive.today: http://archive.today/Irane

Chris Wellons explains the new generators and threads that Emacs 26 provides. He also shows an example of writing a cl-case form that uses the new switch jump table opcode in Emacs 26.

:archive.today: http://archive.today/AfL0y :archive.today: http://archive.today/ZS6pU

Libraries

aio is to Emacs Lisp as asyncio is to Python. This package builds upon Emacs 25 generators to provide functions that pause while they wait on asynchronous events. They do not block any thread while paused.

Manual

Networking

HTTP

Libraries

For simple use cases, and some more complex ones, the built-in url library should be sufficient. Libraries that use curl, such as request, can provide better performance and more flexibility. However, in this author’s experience, both of those tools, while mostly reliable, tend to have some obscure bugs that can occasionally be problematic.

Not a standalone package, but part of Elfeed. A solid, well-designed library, but purpose-built for Elfeed. Could easily be adapted to other packages or factored out as a separate package.

A very simple curl wrapper, last updated in 2012. Not published on MELPA.

A flexible, featureful curl wrapper, last updated in 2015.

request is the most commonly used third-party HTTP library. It has both curl and url.el backends.

url

url is included with Emacs and used by a variety of packages.

Packaging

Contents

Best practices

Autoloads

Autoloading macro-generated functions

This may actually be a bug, or at least an unanswered question.

How to use autoload cookies for custom defun-like macros? : emacs:

Say I have a macro deffoo that expands to some custom kind of defun, and I want to use an autoload cookie to autoload the result. According to the manual,

;;;###autoload (deffoo bar   ...)
  

copies the entire form to autoloads.el, and something like

;;;###autoload (autoload 'bar "this-file") (deffoo bar   ...)
  

should be used instead. What confuses me is this StackOverflow comment by who appears to be Stefan Monnier, saying that Emacs should expand the macro before generating the autoload, and that it’s probably a bug when this does not happen.

Can anyone clear up what the intended behaviour is?

[2018-01-15 Mon 03:37] The correct way to do this is documented in this bug report.

Articles
:archive.today: http://archive.today/UZHhS

Integration with other packages

Optional support

Sometimes you want your package to integrate with other packages, but you don’t want to require users to install those other packages. For example, you might want your package to work with Helm, Ivy, or the built-in Emacs completing-read, but you don’t want to declare a dependency on and require Helm or Ivy, which would force users to install them to use your package.

The best way to handle this is with the =with-eval-after-load= macro. The Emacs manual has a page on it, and this StackOverflow question has some more info. You can also see an example, which also uses =declare-function= to prevent byte-compiler errors.

You should always use lexical binding by setting the header in the first line of the file:

;;; filename.el --- File description  -*- lexical-binding: t; -*-
Articles
:archive.today: http://archive.today/0nfB4 :archive.today: http://archive.today/LUtfZ

Sebastian Wiesner provides a detailed explanation.

:archive.today: http://archive.today/2VtOU

A lot of good examples and discussion.

:archive.today: http://archive.today/xm5zq

I recently had a discussion with Xah Lee about lexical scope in Emacs Lisp. The topic was why lexical-binding exists at a file-level when there was already lexical-let (from cl-lib), prompted by my previous article on JIT byte-code compilation. The specific context is Emacs Lisp, but these concepts apply to language design in general.

Template

When you make a new package, the auto-insert command will insert a set of standard package headers for you. However, here is a more comprehensive template you can use:

;;; package-name.el --- Package description (don't include the word "Emacs")  -*- lexical-binding: t; -*-

;; Copyright (C) 2017 First Last

;; Author: First Last <name@example.com>
;; URL: http://example.com/package-name.el
;; Version: 0.1-pre
;; Package-Requires: ((emacs "25.2"))
;; Keywords: something

;; This file is not part of GNU Emacs.

;;; Commentary:

;; This is my package.  It is nice.  You should try it.

;;;; Installation

;;;;; MELPA

;; If you installed from MELPA, you're done.

;;;;; Manual

;; Install these required packages:

;; + foo
;; + bar

;; Then put this file in your load-path, and put this in your init
;; file:

;; (require 'package-name)

;;;; Usage

;; Run one of these commands:

;; `package-name-command': Frobnicate the flange.

;;;; Tips

;; + You can customize settings in the `package-name' group.

;;;; Credits

;; This package would not have been possible without the following
;; packages: foo[1], which showed me how to bifurcate, and bar[2],
;; which takes care of flanges.
;;
;;  [1] https://example.com/foo.el
;;  [2] https://example.com/bar.el

;;; License:

;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with this program.  If not, see <http://www.gnu.org/licenses/>.

;;; Code:

;;;; Requirements

(require 'foo)
(require 'bar)

;;;; Customization

(defgroup package-name nil
  "Settings for `package-name'."
  :link '(url-link "http://example.com/package-name.el"))

(defcustom package-name-something nil
  "This setting does something."
  :type 'something)

;;;; Variables

(defvar package-name-var nil
  "A variable.")

;;;;; Keymaps

;; This technique makes it easier and less verbose to define keymaps.

(defvar package-name-map
  ;; This makes it easy and much less verbose to define keys
  (let ((map (make-sparse-keymap "package-name map"))
        (maps (list
               ;; Mappings go here, e.g.:
               "RET" #'package-name-RET-command
               [remap search-forward] #'package-name-search-forward
               )))
    (cl-loop for (key fn) on maps by #'cddr
             do (progn
                  (when (stringp key)
                    (setq key (kbd key)))
                  (define-key map key fn)))
    map))

;;;; Functions

;;;;; Commands

;;;###autoload
(defun package-name-command (args)
  "Frobnicate the flange."
  (interactive)
  (package-name--something)
  (bar))

;;;;; Support

(defun package-name--something (args)
  "This function helps frobnicate the flange."
  (foo))

;;;; Footer

(provide 'package-name)

;;; package-name.el ends here

Readme

You should always include a readme with your project. Typically it will be include most of the commentary section. Here’s a template that goes with the package template above:

#+TITLE: package-name

#+PROPERTY: LOGGING nil

# Note: This readme works with the org-make-toc <https://github.com/alphapapa/org-make-toc> package, which automatically updates the table of contents.

[[https://melpa.org/#/package-name][file:https://melpa.org/packages/package-name-badge.svg]] [[https://stable.melpa.org/#/package-name][file:https://stable.melpa.org/packages/package-name-badge.svg]]

This is my package.  It is nice.  You should try it.

* Screenshots

This screenshot shows how to frobnicate the fripulator:

[[screenshot1.png]]

* Contents                                                         :noexport:
:PROPERTIES:
:TOC:      this
:END:
  -  [[#installation][Installation]]
  -  [[#usage][Usage]]
  -  [[#changelog][Changelog]]
  -  [[#credits][Credits]]
  -  [[#development][Development]]
  -  [[#license][License]]

* Installation
:PROPERTIES:
:TOC:      0
:END:

** MELPA

If you installed from MELPA, you're done.  Just run one of the commands below.

** Manual

  Install these required packages:

  + =foo=
  + =bar=

  Then put this file in your load-path, and put this in your init file:

  #+BEGIN_SRC elisp
(require 'package-name)
  #+END_SRC

* Usage
:PROPERTIES:
:TOC:      0
:END:

  Run one of these commands:

  + =package-name-command=: Frobnicate the flange.

** Tips

+ You can customize settings in the =package-name= group.

* Changelog
:PROPERTIES:
:TOC:      0
:END:

** 1.1.0

*Additions*
+  Add command =package-name-debarnacle= to de-barnacle the hull.

*Changes*
+ Command =package-name-anchor= now takes an argument, =weigh= or =let-go=.

*Internal*
+  Rewrote input parsing.
+  Factored out anchor-weighing.

** 1.0.1

*Fixes*
+  Ensure anchor is secure before returning from =package-name-anchor=.

** 1.0.0

Initial release.

* Credits

  This package would not have been possible without the following packages: [[https://example.com/foo.el][foo]] which showed me how to bifurcate, and [[https://example.com/bar.el][bar]], which takes care of flanges.

* Development

Bug reports, feature requests, suggestions — /oh my/!

* License

GPLv3

# Local Variables:
# eval: (require 'org-make-toc)
# before-save-hook: org-make-toc
# org-export-with-properties: ()
# org-export-with-title: t
# End:

Version numbers

Version numbers which are valid in Emacs are those accepted by the function version-to-list, which uses the variables version-separator and version-regexp-alist. See their documentation for specific, up-to-date information. version-to-list’s documentation (as of Emacs 26.1) is reproduced here for convenience:

The version syntax is given by the following EBNF:

VERSION ::= NUMBER ( SEPARATOR NUMBER )*.

NUMBER ::= (0|1|2|3|4|5|6|7|8|9)+.

SEPARATOR ::= ‘version-separator’ (which see)
| ‘version-regexp-alist’ (which see).

The NUMBER part is optional if SEPARATOR is a match for an element
in ‘version-regexp-alist’.

Examples of valid version syntax:

1.0pre2 1.0.7.5 22.8beta3 0.9alpha1 6.9.30Beta 2.4.snapshot .5

Examples of invalid version syntax:

1.0prepre2 1.0..7.5 22.8X3 alpha3.2

Examples of version conversion:

Version String Version as a List of Integers
".5" (0 5)
"0.9 alpha" (0 9 -3)
"0.9AlphA1" (0 9 -3 1)
"0.9snapshot" (0 9 -4)
"1.0-git" (1 0 -4)
"1.0.7.5" (1 0 7 5)
"1.0.cvs" (1 0 -4)
"1.0PRE2" (1 0 -1 2)
"1.0pre2" (1 0 -1 2)
"22.8 Beta3" (22 8 -2 3)
"22.8beta3" (22 8 -2 3)

Libraries

lisp-mnt.el (lm)

This library includes functions helpful for working with and verifying the format of Emacs Lisp package files, including headers, commentary, etc. It’s easy to overlook and hard to re-discover this package because of its lm symbol prefix. It’s listed here because your editor keeps forgetting what it’s called.

Reference

Package headers and structure

The Emacs manual gives this example (I’ve added the lexical-binding part). Also see template.

;;; superfrobnicator.el --- Frobnicate and bifurcate flanges  -*- lexical-binding: t; -*-

;; Copyright (C) 2011 Free Software Foundation, Inc.

;; Author: J. R. Hacker <jrh@example.com>
;; Version: 1.3
;; Package-Requires: ((flange "1.0"))
;; Keywords: multimedia, frobnicate
;; URL: http://example.com/jrhacker/superfrobnicate

...

;;; Commentary:

;; This package provides a minor mode to frobnicate and/or
;; bifurcate any flanges you desire. To activate it, just type
...

;;;###autoload
(define-minor-mode superfrobnicator-mode
...

Tools

Package installation/management

  • State “TODO” from [2018-07-29 Sun 13:11]
  • State “TODO” from [2018-07-29 Sun 13:11]

Developed by the current maintainer of Emacs, himself, John Wiegley.

Pattern matching

Contents

Articles

:archive.today: http://archive.today/J4DqY

Pattern matching is invaluable in elisp. Lists are ubiquitous, and a small amount of pattern matching can often replace a ton of verbose list fiddling.

Since this is Lisp, we have lots of choices! In this post, we’ll compare cl.el, pcase.el, dash.el, and shadchen, so you can choose the best fit for your project. We’ll look at the most common use cases, and end with some recommendations.

For the sake of this post, we’ll consider both pattern matching and destructuring, as they’re closely related concepts.

:archive.today: http://archive.today/vmITX

John Kitchin demonstrates some macros that make it easy to access plist values.

Libraries

Dash is a powerful library, and one of its features is powerful destructuring with its -let macro, and several others that work the same way.

pcase is built-in to Emacs. Its syntax can be confusing, but it is very powerful.

Articles

There are lots of examples here.

:archive.today: http://archive.today/FAzd8 :archive.today: http://archive.today/0Y3Md
Examples
dash

[2018-07-27 Fri 23:29] Dash has new abilities, including -setq, and destructuring plists with implied variable names (i.e. just the keys can be specified, reducing repetition).

pcase-let

This example shows the use of pcase-let* to destructure and bind a nested alist:

(let ((alphabets (a-list 'English (a-list 'first "a"
                                          'second "b")
                         'Greek (a-list 'first "α"
                                        'second "β"))))
  (pcase-let* (((map English) alphabets)
               ((map ('first letter) second) English))
    (list letter second)))  ;; => ("a" "b")

A powerful, Racket-style pattern-matching library.

Tools

let-alist

let-alist is the best thing to happen to associative lists since the invention of the cons cell. This little macro lets you easily access the contents of an alist, concisely and efficiently, without having to specify them preemptively. It comes built-in with 25.1, and is also available on GNU Elpa for older Emacsen.

Example:

(defun sx-question-list--print-info (question-data)
  "DOC"
  (let-alist question-data
    (list
     question-data
     (vector
      (int-to-string .score)
      (int-to-string .answer_count)
      .title " "
      .owner.display_name
      .last_activity_date sx-question-list-ago-string
      " " .tags))))
Articles
:archive.today: http://archive.today/2wNFm

Here Artur introduces the macro and gives examples.

with-dict, with-plist-vals

Courtesy of John Kitchin:[fn:1:Copyright by John Kitchin, licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.]

(defmacro with-dict (key-vals &rest body)
  "A context-manager for a plist where each key is a callable
function that returns the value."
  (declare (indent 1))
  (let* ((g (if (symbolp key-vals)
                (symbol-value key-vals)
              key-vals))
         (keys (-slice g 0 nil 2)))
    `(labels ,(loop for key in keys
                    collect
                    (list key '() `(plist-get ',g ,key)))
       ,@body)))

;; Used as:

(with-dict (:a 1 :b 'some-symbol :c 3)
           (:b))

(let ((d '(:key1 1 :key2 some-other-symbol :key3 3)))
  (with-dict d
             (format "We got %s" (:key2))))

And:

(defmacro with-plist-vals (plist &rest body)
  "Bind the values of a plist to variables with the name of the keys."
  (declare (indent 1))
  `(let ,(loop for key in (-slice plist 0 nil 2)
               for val in (-slice plist 1 nil 2)
               collect (list (intern
                              (substring (symbol-name key) 1))
                             val))
     ,@body))

;; Used like:

(with-plist-vals (:a 4 :b 6)
                 (* 2 a))

Profiling / Optimization

Contents

Articles

:archive.today: http://archive.today/xe0Js

Chris Wellons explains five ways to write faster Emacs Lisp code.

Some Performance Advantages of Lexical Scope « null program

What’s in an Emacs Lambda « null program

Macros

bench

From Phil Lord’s m-buffer-el:

;;;###autoload
(cl-defmacro bench (&optional (times 100000) &rest body)
  "Call `benchmark-run-compiled' on BODY with TIMES iterations, returning list suitable for Org source block evaluation.
Garbage is collected before calling `benchmark-run-compiled' to
avoid counting existing garbage which needs collection."
  (declare (indent defun))
  `(progn
     (garbage-collect)
     (list '("Total runtime" "# of GCs" "Total GC runtime")
           'hline
           (benchmark-run-compiled ,times
             (progn
               ,@body)))))

Used like this:

(bench 1000000
  (cons 'time (current-time)))

When called from an Org source block, it gives output like this:

Total runtime# of GCsTotal GC runtime
1.65783826631.4723854609999876

bench-multi macros

These macros make comparing multiple forms easy:

;;;###autoload
(cl-defmacro bench-multi (&key (times 1) forms ensure-equal raw)
  "Return Org table as a list with benchmark results for FORMS.
Runs FORMS with `benchmark-run-compiled' for TIMES iterations.

When ENSURE-EQUAL is non-nil, the results of FORMS are compared,
and an error is raised if they aren't `equal'. If the results are
sequences, the difference between them is shown with
`seq-difference'.

When RAW is non-nil, the raw results from
`benchmark-run-compiled' are returned instead of an Org table
list.

If the first element of a form is a string, it's used as the
form's description in the bench-multi-results; otherwise, forms
are numbered from 0.

Before each form is run, `garbage-collect' is called."
  ;; MAYBE: Since `bench-multi-lexical' byte-compiles the file, I'm not sure if
  ;; `benchmark-run-compiled' is necessary over `benchmark-run', or if it matters.
  (declare (indent defun))
  (let*((keys (gensym "keys"))
        (result-times (gensym "result-times"))
        (header '(("Form" "x faster than next" "Total runtime" "# of GCs" "Total GC runtime")
                  hline))
        ;; Copy forms so that a subsequent call of the macro will get the original forms.
        (forms (copy-list forms))
        (descriptions (cl-loop for form in forms
                               for i from 0
                               collect (if (stringp (car form))
                                           (prog1 (car form)
                                             (setf (nth i forms) (cadr (nth i forms))))
                                         i))))
    `(unwind-protect
         (progn
           (defvar bench-multi-results nil)
           (let* ((bench-multi-results (make-hash-table))
                  (,result-times (sort (list ,@(cl-loop for form in forms
                                                        for i from 0
                                                        for description = (nth i descriptions)
                                                        collect `(progn
                                                                   (garbage-collect)
                                                                   (cons ,description
                                                                         (benchmark-run-compiled ,times
                                                                           ,(if ensure-equal
                                                                                `(puthash ,description ,form bench-multi-results)
                                                                              form))))))
                                       (lambda (a b)
                                         (< (second a) (second b))))))
             ,(when ensure-equal
                `(cl-loop with ,keys = (hash-table-keys bench-multi-results)
                          for i from 0 to (- (length ,keys) 2)
                          unless (equal (gethash (nth i ,keys) bench-multi-results)
                                        (gethash (nth (1+ i) ,keys) bench-multi-results))
                          do (if (sequencep (gethash (car (hash-table-keys bench-multi-results)) bench-multi-results))
                                 (let* ((k1) (k2)
                                        ;; If the difference in one order is nil, try in other order.
                                        (difference (or (setq k1 (nth i ,keys)
                                                              k2 (nth (1+ i) ,keys)
                                                              difference (seq-difference (gethash k1 bench-multi-results)
                                                                                         (gethash k2 bench-multi-results)))
                                                        (setq k1 (nth (1+ i) ,keys)
                                                              k2 (nth i ,keys)
                                                              difference (seq-difference (gethash k1 bench-multi-results)
                                                                                         (gethash k2 bench-multi-results))))))
                                   (user-error "Forms' bench-multi-results not equal: difference (%s - %s): %S"
                                               k1 k2 difference))
                               ;; Not a sequence
                               (user-error "Forms' bench-multi-results not equal: %s:%S %s:%S"
                                           (nth i ,keys) (nth (1+ i) ,keys)
                                           (gethash (nth i ,keys) bench-multi-results)
                                           (gethash (nth (1+ i) ,keys) bench-multi-results)))))
             ;; Add factors to times and return table
             (if ,raw
                 ,result-times
               (append ',header
                       (bench-multi-process-results ,result-times)))))
       (unintern 'bench-multi-results nil))))

(defun bench-multi-process-results (results)
  "Return sorted RESULTS with factors added."
  (setq results (sort results (-on #'< #'second)))
  (cl-loop with length = (length results)
           for i from 0 to (1- length)
           for description = (car (nth i results))
           for factor = (if (< i (1- length))
                            (format "%.2f" (/ (second (nth (1+ i) results))
                                              (second (nth i results))))
                          "slowest")
           collect (append (list description factor)
                           (list (format "%.6f" (second (nth i results)))
                                 (third (nth i results))
                                 (if (> (fourth (nth i results)) 0)
                                     (format "%.6f" (fourth (nth i results)))
                                   0)))))

Used like:

(bench-multi
  :forms (("org-map-entries" (sort (org-map-entries (lambda ()
                                                      (nth 4 (org-heading-components)))
                                                    "/+MAYBE" 'agenda)
                                   #'string<))
          ("regexp" (sort (-flatten
                           (-non-nil
                            (mapcar (lambda (file)
                                      (let ((case-fold-search t))
                                        (with-current-buffer (find-buffer-visiting file)
                                          (org-with-wide-buffer
                                           (goto-char (point-min))
                                           (cl-loop with regexp = (format org-heading-keyword-regexp-format "MAYBE")
                                                    while (re-search-forward regexp nil t)
                                                    collect (nth 4 (org-heading-components)))))))
                                    (org-agenda-files))))
                          #'string<))))

#+RESULTS[3316dc4375a3b162e32790bb7e72d715d7f756fb]:

Formx faster than nextTotal runtime# of GCsTotal GC runtime
regexp61.970.04244000
org-map-entriesslowest2.63001900

It can also help catch bugs by ensuring that each form returns the same results. For example, the benchmark above contains a subtle bug: because case-fold-search in the regexp form is non-nil, the regexp is compared case-insensitively, so it matches Org headings which start with Maybe rather than only ones which start with MAYBE. Using the :ensure-equal t argument to bench-multi compares the results and raises an error showing the difference between the two sequences the forms evaluate to:

(bench-multi :ensure-equal t
  :forms (("org-map-entries" (sort (org-map-entries (lambda ()
                                                      (nth 4 (org-heading-components)))
                                                    "/+MAYBE" 'agenda)
                                   #'string<))
          ("regexp" (sort (-flatten
                           (-non-nil
                            (mapcar (lambda (file)
                                      (let ((case-fold-search t))
                                        (with-current-buffer (find-buffer-visiting file)
                                          (org-with-wide-buffer
                                           (goto-char (point-min))
                                           (cl-loop with regexp = (format org-heading-keyword-regexp-format "MAYBE")
                                                    while (re-search-forward regexp nil t)
                                                    collect (nth 4 (org-heading-components)))))))
                                    (org-agenda-files))))
                          #'string<))))
user-error: Forms’ results not equal: difference (regexp - org-map-entries): ("Maybe this is not the case?")

Fixing the error, by setting case-fold-search to nil, not only makes the forms give the same result but, in this case, doubles the performance of the faster form:

(bench-multi :ensure-equal t
  :forms (("org-map-entries" (sort (org-map-entries (lambda ()
                                                      (nth 4 (org-heading-components)))
                                                    "/+MAYBE" 'agenda)
                                   #'string<))
          ("regexp" (sort (-flatten
                           (-non-nil
                            (mapcar (lambda (file)
                                      (let ((case-fold-search nil))
                                        (with-current-buffer (find-buffer-visiting file)
                                          (org-with-wide-buffer
                                           (goto-char (point-min))
                                           (cl-loop with regexp = (format org-heading-keyword-regexp-format "MAYBE")
                                                    while (re-search-forward regexp nil t)
                                                    collect (nth 4 (org-heading-components)))))))
                                    (org-agenda-files))))
                          #'string<))))

#+RESULTS[773b94ff27f73dcfcb694429054710a581b7bec5]:

Formx faster than nextTotal runtime# of GCsTotal GC runtime
regexp134.090.01955000
org-map-entriesslowest2.62137700

So this macro showed which code is faster and helped catch a subtle bug.

bench-multi-lexical

To evaluate forms with lexical binding enabled, use this macro:

;;;###autoload
(cl-defmacro bench-multi-lexical (&key (times 1) forms ensure-equal raw)
  "Return Org table as a list with benchmark results for FORMS.
Runs FORMS from a byte-compiled temp file with `lexical-binding'
enabled, using `bench-multi', which see.

Afterward, the temp file is deleted and the function used to run
the benchmark is uninterned."
  (declare (indent defun))
  `(let* ((temp-file (concat (make-temp-file "bench-multi-lexical-") ".el"))
          (fn (gensym "bench-multi-lexical-run-")))
     (with-temp-file temp-file
       (insert ";; -*- lexical-binding: t; -*-" "\n\n"
               "(defvar bench-multi-results)" "\n\n"
               (format "(defun %s () (bench-multi :times %d :ensure-equal %s :raw %s :forms %S))"
                       fn ,times ,ensure-equal ,raw ',forms)))
     (unwind-protect
         (if (byte-compile-file temp-file 'load)
             (funcall (intern (symbol-name fn)))
           (user-error "Error byte-compiling and loading temp file"))
       (delete-file temp-file)
       (unintern (symbol-name fn) nil))))

Used just like bench-multi:

(bench-multi-lexical :ensure-equal t
  :forms (("org-map-entries" (sort (org-map-entries (lambda ()
                                                      (nth 4 (org-heading-components)))
                                                    "/+MAYBE" 'agenda)
                                   #'string<))
          ("regexp" (sort (-flatten
                           (-non-nil
                            (mapcar (lambda (file)
                                      (let ((case-fold-search nil))
                                        (with-current-buffer (find-buffer-visiting file)
                                          (org-with-wide-buffer
                                           (goto-char (point-min))
                                           (cl-loop with regexp = (format org-heading-keyword-regexp-format "MAYBE")
                                                    while (re-search-forward regexp nil t)
                                                    collect (nth 4 (org-heading-components)))))))
                                    (org-agenda-files))))
                          #'string<))))

#+RESULTS[a8ffc10fa4e21eb632122657312040f139b33204]:

Formx faster than nextTotal runtime# of GCsTotal GC runtime
regexp134.260.01964000
org-map-entriesslowest2.63694300

This shows that lexical-binding doesn’t make much difference in this example. But in another example, it does:

(bench-multi :times 1000 :ensure-equal t
  :forms (("buffer-local-value" (--filter (equal 'magit-status-mode (buffer-local-value 'major-mode it))
                                          (buffer-list)))
          ("with-current-buffer" (--filter (equal 'magit-status-mode (with-current-buffer it
                                                                       major-mode))
                                           (buffer-list)))))

#+RESULTS[5ef620a1a1ab52900312e3adb52dd2e6a39dd9b2]:

Formx faster than nextTotal runtime# of GCsTotal GC runtime
buffer-local-value69.640.01325500
with-current-bufferslowest0.92315900
(bench-multi-lexical :times 1000 :ensure-equal t
  :forms (("buffer-local-value" (--filter (equal 'magit-status-mode (buffer-local-value 'major-mode it))
                                          (buffer-list)))
          ("with-current-buffer" (--filter (equal 'magit-status-mode (with-current-buffer it
                                                                       major-mode))
                                           (buffer-list)))))

#+RESULTS[aa64952f5203400ddb1e700814a460fe81f9d2e6]:

Formx faster than nextTotal runtime# of GCsTotal GC runtime
buffer-local-value86.980.01051200
with-current-bufferslowest0.91427400

The buffer-local-value form improved by about 24% when using lexical binding.

bench-dynamic-vs-lexical-binding

This macro compares dynamic and lexical binding.

;;;###autoload
(cl-defmacro bench-dynamic-vs-lexical-binding (&key (times 1) forms ensure-equal)
  "Benchmark FORMS with both dynamic and lexical binding.
Calls `bench-multi' and `bench-multi-lexical', which see."
  (declare (indent defun))
  `(let ((dynamic (bench-multi :times ,times :ensure-equal ,ensure-equal :raw t
                    :forms ,forms))
         (lexical (bench-multi-lexical :times ,times :ensure-equal ,ensure-equal :raw t
                    :forms ,forms))
         (header '("Form" "x faster than next" "Total runtime" "# of GCs" "Total GC runtime")))
     (cl-loop for result in-ref dynamic
              do (setf (car result) (format "Dynamic: %s" (car result))))
     (cl-loop for result in-ref lexical
              do (setf (car result) (format "Lexical: %s" (car result))))
     (append (list header)
             (list 'hline)
             (bench-multi-process-results (append dynamic lexical)))))

Example:

(bench-dynamic-vs-lexical-binding :times 1000 :ensure-equal t
  :forms (("buffer-local-value" (--filter (equal 'magit-status-mode (buffer-local-value 'major-mode it))
                                          (buffer-list)))
          ("with-current-buffer" (--filter (equal 'magit-status-mode (with-current-buffer it
                                                                       major-mode))
                                           (buffer-list)))))

#+RESULTS[73cc92a5949dd2d48f029cab9557eb6132bdf1cf]:

Formx faster than nextTotal runtime# of GCsTotal GC runtime
Lexical: buffer-local-value1.290.01117000
Dynamic: buffer-local-value67.100.01440700
Lexical: with-current-buffer1.010.96665100
Dynamic: with-current-bufferslowest0.97483000
bench-multi-lets

This macro benchmarks multiple forms in multiple environments, which is helpful for testing code that behaves differently depending on global variables.

;;;###autoload
(cl-defmacro bench-multi-lets (&key (times 1) lets forms ensure-equal)
  "Benchmark FORMS in each of lexical environments defined in LETS.
LETS is a list of (\"NAME\" BINDING-FORM) forms.

FORMS is a list of (\"NAME\" FORM) forms.

Calls `bench-multi-lexical', which see."
  (declare (indent defun))
  (let ((benchmarks (cl-loop for (let-name let) in lets
                             collect (list 'list let-name
                                           `(let ,let
                                              (bench-multi-lexical :times ,times :ensure-equal ,ensure-equal :raw t
                                                :forms ,forms))))))
    `(let* ((results (list ,@benchmarks))
            (header '("Form" "x faster than next" "Total runtime" "# of GCs" "Total GC runtime"))
            (results (cl-loop for (let-name let) in results
                              append (cl-loop for result in-ref let
                                              do (setf (car result) (format "%s: %s" let-name (car result)))
                                              collect result))))
       (append (list header)
               (list 'hline)
               (bench-multi-process-results results)))))

Used like:

(bench-multi-lets :times 100000 :ensure-equal t
  :lets (("1" ((var "1")))
         ("12345" ((var "12345")))
         ("1234567890" ((var "1234567890"))))
  :forms (("concat" (concat "VAR: " var))
          ("format" (format "VAR: %s" var))))

#+RESULTS[1e38aeddf90e38a2d2e1cf00aa77f13e7da51cb3]:

Formx faster than nextTotal runtime# of GCsTotal GC runtime
1: concat1.030.02370800
12345: concat1.070.02453300
1234567890: concat1.280.02621000
1: format1.310.03346300
12345: format1.210.04389700
1234567890: formatslowest0.05326300

elp-profile

Call this macro from an Org source block and you’ll get a results block showing which 20 functions were called the most times, how long they took to run, etc. prefixes should be a list of symbols matching the prefixes of the functions you want to instrument.

;;;###autoload
(defmacro elp-profile (times prefixes &rest body)
  (declare (indent defun))
  `(let (output)
     (dolist (prefix ,prefixes)
       (elp-instrument-package (symbol-name prefix)))
     (dotimes (x ,times)
       ,@body)
     (elp-results)
     (elp-restore-all)
     (point-min)
     (forward-line 20)
     (delete-region (point) (point-max))
     (setq output (buffer-substring-no-properties (point-min) (point-max)))
     (kill-buffer)
     (delete-window)
     (let ((rows (s-lines output)))
       (append (list (list "Function" "Times called" "Total time" "Average time")
                     'hline)
               (cl-loop for row in rows
                        collect (s-split (rx (1+ space)) row 'omit-nulls))))))
;; Use like this:
(elp-profile 10 '(map search goto-char car append)
  (goto-char (point-min))
  (search-forward "something"))

This gives a table like:

FunctionTimes calledTotal timeAverage time
mapcar300.00360041300.0001200137
search-forward102.089…e-052.089…e-06
goto-char106.926e-066.926e-07
car133.956…e-063.043…e-07
append15.96e-075.96e-07
mapatoms100.0

Refactoring

Tools

Emacs Refactor (EMR) is a framework for providing language-specific refactoring in Emacs. It includes refactoring commands for a variety of languages, including elisp itself!

Regular expressions

Contents

Articles

:archive.today: http://archive.today/xPWJP

Libraries

lex is a regular expression matching engine with syntax similar to rx. It appears to be more implemented in elisp than standard Emacs regexp tools, so it may be slower, but its additional capabilities may be useful.

Format of regexps is the same as used for `rx’ and `sregex’. Additions:

  • (ere RE) specify regexps using the ERE syntax.
  • (inter REs...) (aka &) make a regexp that only matches if all its branches match. E.g. (inter (ere ".*a.*") (ere ".*b.*")) match any string that contain both an a and a b, in any order.
  • (case-fold REs...) and (case-sensitive REs...) make a regexp that is case sensitive or not, regardless of case-fold-search.

Tools

ample-regexps complements the built-in rx macro by flexibly defining regular expressions with reusable parts. In the following example, the define-arx macro defines three things:

  • A macro url-rx, which expands to a regular expression string at compile time
  • A function url-rx-to-string, which can be used at runtime
  • A variable url-rx-constituents, containing form definitions to use
(define-arx url-rx
  '((http (seq bos (group "http") "://") )
    (https (seq bos (group "https") "://") )
    (https? (seq bos (group "http" (optional "s")) "://") )
    (protocol (seq bos (group (1+ (not (any ":")))) "://"))
    (host (group (1+ (not (any "/")))))
    (path (group "/" (1+ (not (any "?")))))
    (query (seq "?" (group (1+ (not (any "#"))))))
    (fragment (seq "#" (group (1+ anything))))))

The url-rx macro can then be used to test and select parts of URLs:

;; Accept HTTP or HTTPS
(let ((url "http://server/path?query#fragment"))
  (when (string-match (url-rx https? host path (optional query) (optional fragment)) url)
    (list (match-string 0 url)
          (match-string 1 url)
          (match-string 2 url)
          (match-string 3 url)
          (match-string 4 url)
          (match-string 5 url)))) ;=> ("http://server/path?query#fragment" "http" "server" "/path" "query" "fragment")

;; Only accept HTTPS, not plain HTTP
(let ((url "http://server/path?query#fragment"))
  (when (string-match (url-rx https host path (optional query) (optional fragment)) url)
    (list (match-string 0 url)))) ;=> nil

;; Accept any protocol, not just HTTP
(let ((url "ftp://server/path"))
  (when (string-match (url-rx protocol host path (optional query) (optional fragment)) url)
    (list (match-string 0 url)
          (match-string 1 url)
          (match-string 2 url)
          (match-string 3 url)
          (match-string 4 url)
          (match-string 5 url)))) ;=> ("ftp://server/path" "ftp" "server" "/path" nil nil)

This example shows the use of a function to expand a list of strings into a sequence:

(define-arx cond-assignment-rx
  '((alpha_ (regexp "[[:alpha:]_]"))
    (alnum_ (regexp "[[:alnum:]_]"))
    (ws (* blank))
    (sym (:func (lambda (_form &rest args)
                  `(seq symbol-start (or ,@args) symbol-end))))
    (cond-keyword (sym "if" "elif" "while"))
    (id (sym (+ alpha_) (* alnum_))))) ;; -> cond-assignment-rx

(cond-assignment-rx cond-keyword ws id ":" id ws "=" ws id) ;; -> "\\_<\\(?:elif\\|if\\|while\\)\\_>[[:blank:]]*\\_<\\(?:[[:alpha:]_]+\\|[[:alnum:]_]*\\)\\_>:\\_<\\(?:[[:alpha:]_]+\\|[[:alnum:]_]*\\)\\_>[[:blank:]]*=[[:blank:]]*\\_<\\(?:[[:alpha:]_]+\\|[[:alnum:]_]*\\)\\_>"

Strings

Contents

Libraries

Tools

format$ macro

The format$ macro (currently hosted here) allows for easy string interpolation, including optional % sequences as used by format. For example, this:

(format$ "Amount: ${amount% .02f} $name %s" date)

Expands to:

(format "Amount: % .02f %s %s" amount name date)

Since this happens at macro expansion time rather than at runtime, there is no performance penalty, in contrast to using s-lex-format.

Testing

Libraries

Assess provides additional support for testing Emacs packages.

It provides:

  • a set of predicates for comparing strings, buffers and file contents.
  • explainer functions for all predicates giving useful output
  • macros for creating many temporary buffers at once, and for restoring the buffer list.
  • methods for testing indentation, by comparision or “roundtripping”.
  • methods for testing fontification.

Assess aims to be a stateless as possible, leaving Emacs unchanged whether the tests succeed or fail, with respect to buffers, open files and so on; this helps to keep tests independent from each other.

This package provides an Emacs Lisp macro, with-simulated-input, which evaluates one or more forms while simulating a sequence of input events for those forms to read. The result is the same as if you had evaluated the forms and then manually typed in the same input. This macro is useful for non-interactive testing of normally interactive commands and functions, such as completing-read.

Some interactive functions rely on idle timers to do their work, so you might need a way to simulate idleness. For that, there is the wsi-simulate-idle-time function. You can insert calls to this function in between input strings.

Tools

Buttercup is a behavior-driven development framework for testing Emacs Lisp code. It allows to group related tests so they can share common set-up and tear-down code, and allows the programmer to “spy” on functions to ensure they are called with the right arguments during testing.

The framework is heavily inspired by Jasmine.

Cask is a project management tool for Emacs that helps automate the package development cycle; development, dependencies, testing, building, packaging and more.

There are plenty of unit/regression testing tools for Emacs, and even some for functional testing. What Emacs is missing though is a really good testing framework for integration testing. This is where Ecukes comes in.

Cucumber is a great integration testing tool, used mostly for testing web applications. Ecukes is Cucumber for Emacs. No, it’s not a major mode to edit feature files. It is a package that makes it possible to write Cucumber like tests for your Emacs packages.

This is the standard, built-in Emacs testing library, used by core code and third-party packages alike.

Test Elisp with services like Travis CI without the fuss of Cask – just you, your project, and (Emacs-)Make.

Things EMake does:

  • parses, installs, and runs tests for your package
  • provides all the power of Elisp to extend its capabilities on-demand

Things EMake will never do (or ‘reasons you may still need Cask’):

  • manage your development environment or provide tools to do so
  • provide ’bundler-like’ exec abilities (this includes Cask’s emacs and eval commands)

makel is a project consisting of a Makefile (makel.mk) that Emacs package authors can use to facilitate quality checking (linting and tests). The Makefile can be used both locally on the developer machine and remotely on a continuous integration machine.

makem.sh is a script to help build and test Emacs Lisp packages (not Emacs itself). It aims to make linting and testing as simple as possible. Source files and test files are discovered automatically from the project’s Git repo. Output is simple: by default, there is no output unless errors occur. With increasing verbosity levels (with -v or -vv options), more detail gives positive feedback. Output is colored by default to make reading easy.

User interface

Libraries

BUI (Buffer User Interface) is an Emacs library that can be used to make user interfaces to display some kind of entries (like packages, buffers, functions, etc.).

The intention of BUI is to be a high-level library which is convenient to be used both by:

  • package makers, as there is no need to bother about implementing routine details and usual features (like buffer history, filtering displayed entries, etc.);
  • users, as it provides familiar and intuitive interfaces with usual keys (for moving by lines, marking, sorting, switching between buttons); and what is also important, the defined interfaces are highly configurable through various generated variables. A summary of available key bindings can be displayed by pressing h.

Usage

BUI provides means to display entries in 2 types of buffers:

  • list: it is based on tabulated-list-mode, thus it looks similar to a list of Emacs packages (M-x list-packages);
  • info: it can be used to display more verbose info, like various buttons, text and other stuff related to the displayed entry (or entries).

In short, you define how a list / info interface looks like (using bui-define-interface macro), and then you can make some user commands that will display entries (using bui-get-display-entries and similar functions).

This program displays a calendar view in the Emacs buffer.

It is also usable as a library to display items on a calendar.

ctable.el is a table component for Emacs Lisp. Emacs Lisp programs can display a nice table view from an abstract data model. The many emacs programs have the code for displaying table views, such as dired, list-process, buffer-list and so on. So, ctable.el would provide functions and a table framework for the table views.

Emacs’s Widget for Object Collections (ewoc)

The Ewoc package constructs buffer text that represents a structure of Lisp objects, and updates the text to follow changes in that structure. This is like the “view” component in the “model–view–controller” design paradigm. Ewoc means “Emacs’s Widget for Object Collections”.

An ewoc is a structure that organizes information required to construct buffer text that represents certain Lisp data. The buffer text of the ewoc has three parts, in order: first, fixed header text; next, textual descriptions of a series of data elements (Lisp objects that you specify); and last, fixed footer text.

This is a package for GNU Emacs that can be used to tie related commands into a family of short bindings with a common prefix - a Hydra.

The navigel package is a library that makes it simpler for Emacs Lisp developers to define user-interfaces based on tablists (also known as tabulated-lists). Overriding a few (CL) methods and calling navigel-open is all that’s required to get a nice UI to navigate your domain objects (files, music library, database, etc.).

tabulated-list-mode

Tabulated List mode is a major mode for displaying tabulated data, i.e., data consisting of entries, each entry occupying one row of text with its contents divided into columns. Tabulated List mode provides facilities for pretty-printing rows and columns, and sorting the rows according to the values in each column.

The library that powers Magit’s command/option UI.

Taking inspiration from prefix keys and prefix arguments, Transient implements a similar abstraction involving a prefix command, infix arguments and suffix commands.

This is an experiment in building purely text-based user interfaces (TUI’s). The ultimate goal is to explore new paradigms for user interface design and development using Emacs. To this end, tui.el implements an API based on the popular React JavaScript framework in order to reduce the demands involved with designing and building complex text-based UI’s. This is all currently experimental! Expect things to change as I get feedback about what works, what does not!

widget

Most graphical user interface toolkits provide a number of standard user interface controls (sometimes known as “widgets” or “gadgets”). Emacs doesn’t really support anything like this, except for an incredibly powerful text “widget.” On the other hand, Emacs does provide the necessary primitives to implement many other widgets within a text buffer. The widget package simplifies this task.

This is a GUI framework for Emacs Lisp. It is designed for programmers who are familiar with conventional Web MVC frameworks.

Version control

Tools

One of the “killer apps” for Emacs–and for git!

XML / HTML

Libraries

Probably the most featureful, usable library at the moment.

This library provides to formats for xml code generation. The primary form is esxml. esxml is the form that is returned by such functions as libxml-parse-xml-region and is used internally by emacs in many xml related libraries.

It also provides esxml-query:

;; Traditionally people pick one of the following options when faced
;; with the task of extracting data from XML in Emacs Lisp:
;;
;; - Using regular expressions on the unparsed document
;; - Manual tree traversal with `assoc', `car' and `cdr'
;;
;; Browsers faced a similar problem until jQuery happened, shortly
;; afterwards they started providing the `node.querySelector' and
;; `node.querySelectorAll' API for retrieving one or all nodes
;; matching a given CSS selector. This code implements the same API
;; with the `esxml-query' and `esxml-query-all' functions. The
;; following table summarizes the currently supported modifiers and
;; combinators:
;;
;; | Name                               | Supported? | Syntax      | 
;; |------------------------------------+------------+-------------|
;; | Namespaces                         | No         | foo|bar     | 
;; | Commas                             | Yes        | foo, bar    | 
;; | Descendant combinator              | Yes        | foo bar     | 
;; | Child combinator                   | Yes        | foo>bar     | 
;; | Adjacent sibling combinator        | No         | foo+bar     | 
;; | General sibling combinator         | No         | foo~bar     | 
;; | Universal selector                 | Yes        | *           | 
;; | Type selector                      | Yes        | tag         | 
;; | ID selector                        | Yes        | #foo        | 
;; | Class selector                     | Yes        | .foo        | 
;; | Attribute selector                 | Yes        | [foo]       | 
;; | Exact match attribute selector     | Yes        | [foo=bar]   | 
;; | Prefix match attribute selector    | Yes        | [foo^=bar]  | 
;; | Suffix match attribute selector    | Yes        | [foo$=bar]  | 
;; | Substring match attribute selector | Yes        | [foo*=bar]  | 
;; | Include match attribute selector   | Yes        | [foo~=bar]  | 
;; | Dash match attribute selector      | Yes        | [foo|=bar]  | 
;; | Attribute selector modifiers       | No         | [foo=bar i] | 
;; | Pseudo elements                    | No         | ::foo       | 
;; | Pseudo classes                     | No         | :foo        | 

Example:

(defun org-books--amazon (url)
  "Return plist of data for book at Amazon URL."
  (cl-flet ((field (target-field list)
                   (cl-loop for li in list
                            for (field value) = (ignore-errors
                                                  (-let (((_ _ (_ _ field) value) li))
                                                    (list field value)))
                            when (equal field target-field)
                            return (s-trim value))))
    (let* ((html (org-web-tools--get-url url))
           (tree (with-temp-buffer
                   (insert html)
                   (libxml-parse-html-region (point-min) (point-max))))
           (author (esxml-query "span.author a.contributorNameID *" tree))
           (title (esxml-query "div#booksTitle h1#title > span *" tree))
           (details (esxml-query-all "table#productDetailsTable ul li" tree))
           (date (if-let ((printed (third (esxml-query-all "div#booksTitle h1#title span *" tree))))
                     ;; Printed book
                     (s-replace "" "" printed)
                   ;; Kindle book
                   (field "Publication Date:" details)))
           (asin (field "ASIN:" details))
           (publisher (-some->> (field "Publisher:" details)
                                (replace-regexp-in-string (rx " (" (1+ anything) ")") "")))
           (isbn-10 (field "ISBN-10:" details))
           (isbn-13 (field "ISBN-13:" details)))
      (list :author author :title title :publisher publisher :date date
            :asin asin :isbn-10 isbn-10 :isbn-13 isbn-13))))

It’s like jQuery, but way less useful.

Example:

<html style="height: 100vh">
  <head class="kek"><title class="kek" data-bar="foo">Complex HTML Page</title></head>
  <body class="kek bur" style="height: 100%">
    <h1 id="bar" class="kek wow">Wow this is an example</h1>
    <input id="quux" class="kek foo"/>
    <iframe id="baz" sandbox="allow-same-origin allow-scripts allow-popups allow-forms"
            width="100%" height="100%" src="example.org">
    </iframe>
  </body>
</html>
(let ((html (elq-read-file "~/kek.html")))
  (elq-el (car (elq-$ ".kek#quux" html))) ; => "input"
  (mapcar 'elq-el (elq-$ ".kek" html)) ; => ("input" "h1" "body" "title" "head")
  (mapcar (lambda (el) (elq-el (elq-parent el)))
          (elq-$ ".kek" html)) ; => ("body" "body" "html" "head" "html")
  (mapcar (lambda (el) (mapcar 'elq-el (elq-siblings el)))
          (elq-$ ".kek" html)) ; => (("h1" "input" "iframe") ("h1" "input" "iframe") ("head" "body") ("title") ("head" "body"))
  (elq-$ ".kek" html) ; => Hope you didn't like your messages buffer
  (elq-write html nil)) ; => "<html style=\"height: 100vh\"> ... </html>"

Provides lisp-based (rather than string-based) selectors. This library is primarily aimed at internal elfeed use rather than general use, however it may be useful to others. The author is considering publishing it separately.

;; Grab the top-level paragraph content from XHTML.
(xml-query-all '(html body p *) xhtml)

;; Extract all the links from an Atom feed.
(xml-query-all '(feed entry link [rel "alternate"] :href) xml)

This provides a limited set of lisp-based selectors (rather than string-based selectors).

Example:

(require 'enlive)

(enlive-text
 (enlive-query (enlive-fetch "http://gnu.org/") [title])) ; => "The GNU Operating System and the Free Software Movement"

Mostly undocumented, providing three main functions:

;; Utility functions for xml parse trees.
;; - `xml+-query-all' and `xml+-query-first' are query functions that search
;; descendants in node lists. They don't work with namespace-aware parsing yet
;;
;; - `xml+-node-text' gets node text

Blogs

This is the main community aggregator. You can find just about everyone’s Emacs-related blog posts here.

Sacha Chua’s /Emacs News/

This is Sacha’s weekly Emacs news digest. Don’t miss it!

Artur Malabarba’s /Endless Parentheses/

One of the top Emacs blogs, frequently updated, and often highlights other interesting blog entries in the community.

Oleh Krehel’s /(or emacs/

Sacha Chua’s /Living an Awesome Life/

People

The Emacs community is so full of brilliant, generous people that I can’t keep track of them all! I will surely overlook many, and I will add them in no particular order, but merely as I come across them again and again.

Anders Lindgren

Anders, aka Lindydancer, has written numerous packages to help with developing highlighting and font-lock packages, as well as some other useful tools.

Packages

el2markdown: Convert Emacs Lisp comments to MarkDown

face-explorer: Library and tools for faces and text properties

faceup: Regression test system for font-lock keywords

font-lock-profiler: Coverage and timing tool for font-lock keywords

font-lock-regression-suite: Regression test suite for font-lock keywords of Emacs standard modes

font-lock-studio: Debugger for Font Lock keywords

highlight-refontification: Visualize how font-lock refontifies a buffer

lisp-extra-font-lock: Highlight bound variables and quoted expressions in lisp

multicolumn: Multiple side-by-side windows support

Artur Malabarba

Another prolific Emacs contributor, package developer, and blogger.

Packages

aggressive-indent-mode

paradox

Damien Cassou

Packages

beginend.el

navigel

John Wiegley

John is the current Emacs maintainer.

Packages

use-package

Jonas Bernoulli

Jonas is a prolific Emacs package developer and maintainer. You could spend hours on his GitHub repo.

Packages

Magit

ox-texinfo+

Jorgen Schäfer

Packages

buttercup: Behavior-Driven Emacs Lisp Testing

Magnar Sveen

Packages

dash.el

expand-region.el

multiple-cursors.el

s.el

Matus Goljer

Packages

dash.el

smartparens

Oleh Krehel

Oleh is a prolific package author, having contributed many very high-quality packages. He also writes at his blog.

Packages

lispy: short and sweet LISP editing

Phil Lord

Packages

lentic: Create views of the same content in two buffers

m-buffer-el

Roland Walker

Roland has published a wide variety of useful Emacs packages.

Packages

list-utils: List-manipulation utility functions

Sacha Chua

Sacha could easily be nominated the official Emacs ambassador, were there to be one. Her contributions to the Emacs and Org-mode communities are innumerable. One of her greatest recent contributions is her weekly Emacs news posts that serve as a digest of everything that happened in the Emacs world over the past week.

Wilfred Hughes

Wilfred has published several useful packages, and he’s also leading the Rust Emacs port.

Packages

emacs-refactor

ht.el

suggest.el

Contributions

Yes, please! Send pull requests and file issues on the GitHub repo. This is intended to be a community project.

Guidelines

Catalog and tag appropriately
New entries in the outline should have the appropriate tags and should follow the existing hierarchy. For example, articles should be tagged articles, and generally filed under an Articles heading using tag inheritance to apply the tag.
“Loosely” or “usefully” opinionated
Rather than being a place to dump links for users to sort out, we should do that ourselves. Links should have summaries and examples. Where there are multiple links to similar projects, we should compare them and guide users to what we think is generally the best solution, or the best solution for each use case.
Archive reference material
Much of the shared wisdom in the Emacs community is written in a variety of blogs by users and developers, as well as posts on Reddit, StackOverflow, etc. These tend to hang around for a long time, but being the Internet, this is never guaranteed. When linking to an article or other reference material, we should store a link to an archived version using this code.

Requirements

org-make-toc
This package updates the table of contents. It’s automatically used by this document through file-local variables, which you should be prompted to allow when opening the file.

Tasks

These resources should be added to the appropriate sections above. Since it takes some work to catalog and organize them, they are dumped here for future reference. Pull requests for these are welcome!

easy-mmode.el

Especially easy-mmode-defmap.

Articles to add [0/13]

Why package.el? (1 min read)

Emacs script pitfalls (13 min read)

Add people [0/6]

He has a lot of interesting libraries on his repo, and some of them are extensively documented. An aspiring Emacs Lisp developer could learn a lot from his code.

Add more of Roland Walker’s packages

Clemens Radermacher

Add tips for new developers

e.g.:

  • Commonly used minor modes
    • highlight-funcalls
    • highlight-quoted
    • outline-minor-mode

Add MELPA

Mention @milkypostman, @purcell, @syohex, etc. Mention sandbox.

Add Modern Emacs site

Dynamic modules section

Add resources from its readme

For my own notes here are all the resources on dynamic modules I know of:

Here are the official Emacs header and example: emacs-module.h: http://git.savannah.gnu.org/cgit/emacs.git/tree/src/emacs-module.h?id=e18ee60b02d08b2f075903005798d3d6064dc013 mod_test.c: http://git.savannah.gnu.org/cgit/emacs.git/tree/modules/mod-test/mod-test.c?id=e18ee60b02d08b2f075903005798d3d6064dc013

This simple example in C http://diobla.info/blog-archive/modules-tut.html

joymacs
http://nullprogram.com/blog/2016/11/05/
mruby
https://github.com/syohex/emacs-mruby-test
https://github.com/tromey/emacs-ffi
an actual ffi for emacs
elfuse
https://github.com/vkazanov/elfuse a file system in Emacs
asynchronous events
http://nullprogram.com/blog/2017/02/14/ related to elfuse
emacs-sqlite3
sqlite3 binding of Emacs Lisp
emacs-parson
JSON parser with dynamic module feature with parson
libyaml
libyaml
emacs-perl
Embed Perl into Emacs
https://github.com/syohex/emacs-eject
eject a cd
emacs-capstone
elisp bindings for the capstone disassembler
emacs-csound
EmacsLisp link to Csound’s API via Emacs Modules
emacs-cmigemo
Emacs dynamic module for cmigemo
emacs-cipher
OpenSSL cipher binding of Emacs Lisp
emacs-lua
Lua engine from Emacs Lisp
emacs-ztd
libzstd binding of Emacs Lisp
mem-cached
libmemcached
https://coldnew.github.io/2d16cc25/
in Chinese, but with code

A collection of module resources: https://github.com/emacs-pe/emacs-modules

golang
https://github.com/sigma/go-emacs writing modules in go

This may not be a dynamic module but claims an ffi haskell https://github.com/knupfer/haskell-emacs

Documentation best practices

Describe things like exporting an Org readme to an Info manual, e.g. like Magit, org-super-agenda, etc.

Testing

Add databases section

Tree-traversal

  • State “TODO” from [2017-09-06 Wed 00:21]

Test in MELPA sandbox

  • State “TODO” from [2017-12-16 Sat 20:16]

[2017-07-29 Sat 00:33] Not only should you test installing and using your package in the sandbox, but you should also test then exiting the sandbox Emacs, running it again with the package already installed, and loading it. This is because, when the sandbox installs the package, the byte-compilation seems to load some things that won’t be loaded the same way when only loading the byte-compiled file (especially if you have any eval-when-compile lines, or unusual macros or things that modify the environment when loaded).

Sequence shuffling examples and benchmarks

Benchmarking sequence shuffling

See melpa/melpa#6191 (comment)

(defun key-quiz--shuffle-list (list)
  "Shuffles LIST randomly, modying it in-place."
  (dolist (i (reverse (number-sequence 1 (1- (length list)))))
    (let ((j (random (1+ i)))
	  (tmp (elt list i)))
      (setf (elt list i) (elt list j))
      (setf (elt list j) tmp)))
  list)

(defun key-quiz--shuffle-list-nreverse (list)
  "Shuffles LIST randomly, modying it in-place."
  (dolist (i (nreverse (number-sequence 1 (1- (length list)))))
    (let ((j (random (1+ i)))
	  (tmp (elt list i)))
      (setf (elt list i) (elt list j))
      (setf (elt list j) tmp)))
  list)

(defun elfeed--shuffle (seq)
  "Destructively shuffle SEQ."
  (let ((n (length seq)))
    (prog1 seq                  ; don't use dotimes result (bug#16206)
      (dotimes (i n)
        (cl-rotatef (elt seq i) (elt seq (+ i (random (- n i)))))))))

(defun faster-seq-sort-by (function pred sequence)
  "Sort SEQUENCE using PRED as a comparison function.
Elements of SEQUENCE are transformed by FUNCTION before being
sorted.  FUNCTION must be a function of one argument."
  ;; This version is modified to avoid calling "random" twice every time the predicate is called.
  (seq-map 'cdr
           (sort (seq-map (lambda (x) (cons (funcall function x) x)) sequence)
                 (lambda (a b)
                   (funcall pred (car a) (car b))))))

(defun seq-sort-by--shuffle (seq)
  (seq-sort-by (lambda (_) (random)) #'<= seq))

(defun faster-seq-sort-by--shuffle (seq)
  (faster-seq-sort-by (lambda (_) (random)) #'<= seq))

Lists

(let ((big-list (seq-into (seq-take obarray 5000) 'list)))
  (bench-multi-lexical :times 100
    :forms (("key-quiz--shuffle-list" (key-quiz--shuffle-list big-list))
            ("key-quiz--shuffle-list-nreverse" (key-quiz--shuffle-list-nreverse big-list))
            ("elfeed--shuffle" (elfeed--shuffle big-list))
            ("seq-sort-by--shuffle" (seq-sort-by--shuffle big-list))
            ("faster-seq-sort-by--shuffle" (faster-seq-sort-by--shuffle big-list)))))
Formx faster than nextTotal runtime# of GCsTotal GC runtime
faster-seq-sort-by–shuffle1.381.72503700
seq-sort-by–shuffle15.012.37823400
key-quiz–shuffle-list-nreverse1.0335.7033162717.892723
key-quiz–shuffle-list1.2436.6303202818.768216
elfeed–shuffleslowest45.4394053221.130538

Vectors

(let ((big-list (seq-into (seq-take obarray 5000) 'vector)))
  (bench-multi-lexical :times 100
    :forms (("key-quiz--shuffle-list" (key-quiz--shuffle-list big-list))
            ("key-quiz--shuffle-list-nreverse" (key-quiz--shuffle-list-nreverse big-list))
            ("elfeed--shuffle" (elfeed--shuffle big-list))
            ("seq-sort-by--shuffle" (seq-sort-by--shuffle big-list))
            ("faster-seq-sort-by--shuffle" (faster-seq-sort-by--shuffle big-list)))))
Formx faster than nextTotal runtime# of GCsTotal GC runtime
faster-seq-sort-by–shuffle1.391.71899000
seq-sort-by–shuffle10.422.39086000
key-quiz–shuffle-list-nreverse1.0224.9187742717.971779
key-quiz–shuffle-list1.1025.4526652818.487015
elfeed–shuffleslowest27.9913053221.215224
  • State “TODO” from [2019-10-05 Sat 12:20]

Code

This section contains code used to add to and update this document.

UNDERWAY Automate adding new links and summaries

  • State “UNDERWAY” from “TODO” [2017-08-03 Thu 15:12]

Get summary of page

Get archive.today link for page

  • Note taken on [2018-11-24 Sat 10:44]
    Using org-web-tools instead of the Python script.
  • State “DONE” from “TODO” [2017-08-03 Thu 15:11]
(require 'org-web-tools)

(defun emacs-package-dev-handbook--archive.today-capture (url)
  "Return archive.today archived URL for URL."
  (when-let* ((id (org-web-tools-archive--archive.today-url-id url)))
    (format "http://%s/%s" org-web-tools-archive-hostname id)))

(cl-defun emacs-package-dev-handbook-insert-archive.today-property (url)
  "Set the \"archive.today\" property for entry at point to the archived version of URL.
Interactively, assumes heading on/before point is an Org link to
a web page."
  (interactive (list (save-excursion
                       (unless (org-at-heading-p)
                         (org-back-to-heading))
                       (beginning-of-line)
                       (when (re-search-forward org-bracket-link-regexp (line-end-position) 'noerror)
                         (org-link-unescape (match-string-no-properties 1))))))
  (when-let ((archive-url (emacs-package-dev-handbook--archive.today-capture url)))
    (org-set-property "archive.today" archive-url)))

Insert new entry at point

Maybe use capture templates and refile?

Table of Contents

The ToC is generated using org-make-toc.

Config

I love Emacs and Org mode. This makes it so easy to make the document…alive! And automated! Beautiful.

File-local properties

File-local variables