shx for Emacs

Description
Install
Setup
Key bindings
Markup in the shell
Extra shell commands
Related

Description

shx or “shell-extras” extends comint-mode in Emacs (e.g. M-x shell).

It’s compatible with any underlying REPL (zsh, bash, psql, ipython, etc.).

It parses the output stream in a few useful ways:

Display graphics and plots in the shell with a simple markup language (e.g. <view image.png>)
Add event-driven and timed behaviors to any shell session
Open any filename or URL by arrowing up to it and pressing RET (shx will even try to guess the correct directory)
Yank any line to the prompt by arrowing up to it and pressing C-RET
Check the time a command was run by mousing over its prompt

shx makes it easy to add new shell commands written in elisp. Some are already built in:

:clear clears the buffer (like clear or Command-K on macOS)
:e filename.txt opens a file for editing
:ssh user@host:port starts a remote shell session using tramp
:view image_file.png embeds an image in the shell
:plotline data_file.txt embeds a line plot
etc.

It also extends shell-mode’s syntax highlighting, recenters and highlights content for better viewing when you run commands like comint-previous-prompt and comint-kill-input, and improves compatibility with evil-mode by anticipating when to switch to insert mode.

Use M-x shx RET to start a new shell session with shx-mode enabled.

This version is tested with Emacs 26.1. Check out the release log.

Install

From MELPA

M-x package-install RET shx RET to install shx from MELPA.

From GNU Guix

guix install emacs-shx to install shx from GNU Guix.

Manually

Add the following to your .emacs:

(add-to-list 'load-path "~/path/to/shx/") ; add shx.el's directory to the load-path
(require 'shx)                            ; load shell-extras

Setup

Quick-start

Type M-x shx RET. Try out the following commands:

:e ~/.bashrc to edit your .bashrc (for example)
:man ls to display the man page for ls
:help to a start a completing read for other shx commands

Enable automatically

If you like shx-mode, you can enable it everywhere:

(shx-global-mode 1)  ; toggle shx-mode on globally

Now shx will run automatically in any comint-mode buffer. If you don’t want shx to run in every comint-mode buffer, you can use M-x shx-mode on a case-by-case basis, or just add hooks to the mode in question, for example:

(add-hook 'inferior-python-mode-hook #'shx-mode)

Customize

Use M-x customize-group RET shx RET to see shx’s many customization options. Here’s an example customization using setq:

(setq
  ;; resync the shell's default-directory with Emacs on "z" commands:
  shx-directory-tracker-regexp "^z "
  ;; vastly improve display performance by breaking up long output lines
  shx-max-output 1024
  ;; prevent input longer than macOS's typeahead buffer from going through
  shx-max-input 1024
  ;; prefer inlined images and plots to have a height of 250 pixels
  shx-img-height 250
  ;; don't show any incidental hint messages about how to use shx
  shx-show-hints nil
  ;; flash the previous comint prompt for a full second when using C-c C-p
  shx-flash-prompt-time 1.0
  ;; use `#' to prefix shx commands instead of the default `:'
  shx-leader "#")

Key bindings

Key binding	Description
`C-RET`	If the cursor is not on the prompt, paste the current line to the input
`RET`	If the cursor is on a filename or a URL, try to open it
`SPC`	If the prompt is `:`, send `SPC` straight through to the process
`q`	If the prompt is `:`, send `q` straight through to the process

Note the prompt will be : when reading through the output of less or a man page if you run the following:

(setenv "LESS" "--dumb --prompt=s")

Markup in the shell

shx’s markup can enhance basic command-line applications and drive other events.

If the output ever contains <view mountains.png> on a line by itself, then a scaled rendering of mountains.png will be inlined within the text in the shell. This works because view is a shx command. shx will execute any (safe) shx command that appears with the following syntax:

<command arg1 arg2 ...>

where command is a shx command and arg1 ... argn is a space-separated list of arguments. Arguments don’t need to be surrounded by quotes – the command will figure out how to parse them.

You can use this markup to create a barplot (:plotbar) after collecting some stats, or generate an :alert when a task is finished, and so forth.

Extra shell commands

shx’s ‘extra’ commands are invoked by typing a : followed by the command’s name. (You can change the : prefix by customizing the shx-leader variable.) These commands are written in elisp and so can access all of Emacs’ facilities. Type :help to see a complete listing of shx commands.

One command I use frequently is the :edit (shorthand :e) command:

# edit the .emacs file:
:edit ~/.emacs

# use tramp to edit .emacs on a remote host through ssh:
:e /ssh:remote-host.com:~/.emacs

# use tramp to edit .bashrc on a running docker container:
:e /docker:02fbc948e009:~/.bashrc

# edit a local file as root
:sedit /etc/passwd

Thanks to CeleritasCelery it’s also possible to use environment variables in the argument list:

:e $HOME/.emacs.d

(To see an environment variable’s value, use (getenv "<var>").)

The :ssh and :docker commands are popular for opening “remote” shells:

# open a shell on a remote host:
:ssh user@remote-host.com

# connect to a running docker container
:docker 8a8335d63ff3

# reopen the shell on the localhost:
:ssh

Jordan Besly points out that you can customize the default interpreter for each “remote” using connection-profile-set-local-variables.

I also use the :kept and :keep commands frequently:

# write a complicated command:
wget https://bootstrap.pypa.io/get-pip.py && python get-pip.py

# save the last command:
:keep

# search for commands having to do with pip:
:kept pip

Because these commands are written in elisp, shx gives M-x shell a lot of the same advantages as eshell. You can even evaluate elisp code directly in the buffer (see :help eval).

General commands

Command	Description
`:alert`	Reveal the buffer with an alert. Useful for markup
`:clear`	Clear the buffer
`:date`	Show the date (even when the process is blocked)
`:diff file1 file2`	Launch an Emacs diff between two files
`:edit file`	Edit a file. Shortcut: `:e <file>`
`:eval (elisp-sexp)`	Evaluate some elisp code. Example: `:eval (pwd)`
`:find <filename>`	Run a fuzzy-find for <filename>
`:goto-url <url>`	Completing-read for a URL
`:header New header`	Change the current `header-line-format`
`:kept regexp`	Show a list of your ‘kept’ commands matching regexp
`:keep`	Add the previous command to the list of kept commands
`:man topic`	Invoke the Emacs man page browser on a topic
`:ssh <host>`	Restart the shell on the specified host

There are more than this – type :help for a listing of all user commands.

Graphical commands

Command	Description
`:view image_file.jpg`	Display an image
`:plotbar data_file.txt`	Display a bar plot
`:plotline data_file.txt`	Display a line plot
`:plotmatrix data_file.txt`	Display a heatmap
`:plotscatter data_file.txt`	Display a scatter plot
`:plot3d data_file.txt`	Display a 3D plot

These are for displaying inline graphics and plots in the shell buffer. You can control how much vertical space an inline image occupies by customizing the shx-img-height variable.

Note convert (i.e. ImageMagick) and gnuplot need to be installed. If the binaries are installed but these commands aren’t working, customize the shx-path-to-convert and shx-path-to-gnuplot variables to point to the binaries. Also note these graphical commands aren’t yet compatible with shells launched on remote hosts (e.g. over ssh or in a Docker container).

Asynchronous commands

Command	Description
`:delay <sec> <command>`	Run a shell command after a specific delay
`:pulse <sec> <command>`	Repeat a shell command forever with a given delay
`:repeat <count> <sec> <command>`	Repeat a shell command `<count>` times
`:stop <num>`	Cancel a repeating or delayed command

Use these to delay, pulse, or repeat a command a specific number of times. Unfortunately these only support your typical shell commands, and not shx’s extra (colon-prefixed) commands. So this possible:

# Run the 'pwd' command 10 seconds from now:
:delay 10 pwd

But this is not possible:

# Run the 'pwd' shx command 10 seconds from now (DOES NOT WORK)
:delay 10 :pwd

Adding new commands

New shx commands are written by defining single-argument elisp functions named shx-cmd-COMMAND-NAME, where COMMAND-NAME is what the user would type to invoke it.

Example: a command to rename the buffer

If you evaluate the following (or add it to your .emacs),

(defun shx-cmd-rename (name)
  "(SAFE) Rename the current buffer to NAME."
  (if (not (ignore-errors (rename-buffer name)))
      (shx-insert 'error "Can't rename buffer.")
    (shx-insert "Renaming buffer to " name "\n")
    (shx--hint "Emacs won't save buffers starting with *")))

then each shx buffer will immediately have access to the :rename command. When it’s invoked, shx will also display a hint about buffer names.

Note the importance of defining a docstring. This documents the command so that typing :help rename will give the user information on what the command does. Further, since the docstring begins with (SAFE), it becomes part of shx’s markup language. So in this case if:

<rename A new name for the buffer>

appears on a line by itself in the output, the buffer will try to automatically rename itself.

Example: invoking ediff from the shell

A command similar to this one is built into shx:

(defun shx-cmd-diff (files)
  "(SAFE) Launch an Emacs `ediff' between FILES."
  (setq files (shx-tokenize files))
  (if (not (eq (length files) 2))
      (shx-insert 'error "diff <file1> <file2>\n")
    (shx-insert "invoking ediff...\n")
    (shx--asynch-funcall #'ediff (mapcar #'expand-file-name files))))

Note that files is supplied as a string, but it’s immediately parsed into a list of strings using shx-tokenize. Helpfully, this function is able to parse various styles of quoting and escaping, for example (shx-tokenize "'file one' file\\ two") evaluates to ("file one" "file two").

Example: a command to browse URLs

If you execute the following,

(defun shx-cmd-browse (url)
  "Browse the supplied URL."
  (shx-insert "Browsing " 'font-lock-keyword-face url)
  (browse-url url))

then each shx buffer will have access to the :browse command.

Note the docstring does not specify that this command is SAFE. This means <browse url> will not become part of shx’s markup. That makes sense in this case, since you wouldn’t want to give a process the power to open arbitrary URLs without prompting.

If you’re here, these might be interesting:

Shell & Comint Secrets: History commands
PComplete: Context-Sensitive Completion in Emacs
10 tools to power up your command line
Creating dynamic bash prompts
The Keep Utility inspired the kept and keep commands
“Terminals Are Sexy” (portal)
Command-Line Lint, another project I maintain
oh my zsh, a community-driven zsh configuration
bash-it, a community driven bash configuration

And if running a dumb terminal in Emacs isn’t for you, here are some alternatives:

The Tao of tmux, re: working in the terminal with tmux
zsh-syntax-highlighting
Shell configuration tips from Vitaly Belman
Shell integration for iTerm2 on macOS
BitBar adds program output to menus on macOS

riscy/shx-for-emacs