Porthole

RPC servers for Emacs.

🚢

Porthole lets you start RPC servers in Emacs. These servers allow Elisp to be invoked remotely via HTTP requests.

You can expose data that exists within Emacs, or control Emacs from an external program. Perhaps you want to leverage Emacs' excellent editing facilities? It's up to you.

Table of Contents

Typical Workflow
- Multiple Servers
- I Want More Control
Why?
Usage Examples
The JSON-RPC 2.0 Protocol
Symbols & Keyword Arguments
Writing a Client
What About Security?
- Outside Localhost
Installation
FAQ

Typical Workflow

Porthole servers are designed to "just work." All your server needs is a name and clients will be able to find it automatically. Here's a typical workflow:

In Emacs

Pick a name and start your server.
Tell it which functions you want to be available to RPC calls.

Now, continue using Emacs.

In the Client

Load the connection information from your server's session file (this file has a known path).
POST a JSON-RPC request to the server.

*Emacs executes your RPC call and returns the result.*
Parse the JSON-RPC 2.0 object you received.

That's it! There's even a Python Client to handle the client-side automatically. See Example 1 for a demonstration.

Multiple Servers

Many Porthole servers can run at once - they won't interfere with one another, except that Emacs can only process one request at a time. Your package can start a Porthole server without worrying about other packages.

I Want More Control

You can have it! Take a look at the manual setup example.

Why?

I want to open Emacs up. An Emacs session has a lot happening, and it's all text. What if you could gain access to that information?

What if you could:

Edit the contents of any text box with Emacs?
Interact with Emacs from a Python REPL?
Control Emacs with something other than a keyboard, like your voice?
Use Emacs' outstanding editing tools outside Emacs?

All of these need some way of communicating with Emacs. Porthole acts as a foundation that allows Emacs to expose its functionality to other languages, in a way that's simple.

Usage Examples

Let's run through some usage examples.

Example 1: Quickstart with the Python Client

By default, each server just requires a name. Clients can connect automatically, as long as they know the name.

Let's say you want to write a Python package, pyrate-ship, that allows you to use Emacs as a calculator.

In Emacs:

Starting a server is easy. All you need is a name:

;; Start a Porthole RPC server.
(porthole-start-server "pyrate-server")
;; Functions have to be exposed before they can be invoked remotely.
;; Let's expose the `+' function.
(porthole-expose-function "pyrate-server" '+)

In the Client:

The easiest way to send RPC calls to Porthole is with the Porthole Python Client. For now, let's use that. In the next example, we'll send a request manually. This is a minimal example:

import emacs_porthole

# Calling from Python is easy - you only need one line.
result = emacs_porthole.call("pyrate-server", method="+", params=[1, 2, 3])
# If successful, Porthole will return the result.
assert result == 6

If the call fails, the Porthole Python Client will raise a relevant error. Here's what it looks like with error handling:

from emacs_porthole import (
    call,
    PortholeConnectionError,
    MethodNotExposedError,
    InternalMethodError,
    TimeoutError
)

def sum_in_emacs():
    try:
        return call("pyrate-server", method="+", params=[1, 2, 3])
    except TimeoutError:
        print("The request timed out. Is Emacs busy?")
    except PortholeConnectionError:
        print("There was a problem connecting to the server. Is it running?")
    except MethodNotExposedError:
        print("The method wasn't exposed! Remember to expose it in the porthole server.")
    except InternalMethodError as e:
        # This will be raised when the function was executed, and raised an error during execution.
        print("There was an error executing the method. Details:\n"
              "error_type: {}\n".format(e.elisp_error_symbol)
              "error_data: {}".format(e.elisp_error_data))

Example 2: POSTing RPC Calls Directly

It's easy to send RPC calls to Porthole servers yourself. Just POST some JSON. Here's the basic process:

Read the server's port from the session file (this will have a known path).
Send a POST request to localhost:<port> with the JSON-RPC 2.0 request encoded in the body.

We'll write a client that works on Windows, in Python, using requests for our HTTP calls.

In Emacs

Start your server, like in the last example:

;; Start an automatic server.
(porthole-start-server "pryvateer-server")
;; This time, we'll expose the `insert' function.
(porthole-expose-function "pryvateer-server" 'insert)

import requests
import json

def pryvateer_insert():
    server_name = "pryvateer-server"

    # Preparing a call is easy.
    rpc_call = {
        "jsonrpc": "2.0",
        "method": "insert",
        "params": ["This is some text we want to insert"],
        "id": 23084
    }

    # The session info is always stored in the same basic path. The `temp_dir`
    # varies by platform - we'll explain how to manage that later.
    temp_dir = os.environ.get("HOME")
    session_info_path = "{temp_dir}/emacs-porthole/{server_name}/session.json".format(
        temp_dir=temp_dir,
        server_name=server_name
    )

    # Now, we load the server information from the file.
    if os.path.isfile(session_info_path):
        session_info = json.read(open(session_info_path))
    else:
        raise RuntimeError("Server does not appear to be running.")

    # Finally, we can post our request to the server.
    address = "http://localhost:{}".format(session_info["port"])
    auth = requests.HTTPBasicAuth(session_info["username"],session_info["password"])
    response = requests.post(address, json=rpc_call, auth=auth)
    if response.status_code == 200:
        # If the call is successful, the body will be a JSON-RPC 2.0 response.
        # Decode it into Python, and return it.
        return response.json()
    else:
        raise RuntimeError("Response code {} received.".format(response.status_code))

This client isn't perfect - it doesn't cover every edge case. See the source code for the Porthole Python client if you'd like to write a thorough client in another language.

Example 3: Manual Setup

Automatic servers are the intended use case of Porthole. However, you may wish to configure the server manually. Here's an example:

In Emacs:

Start a server on a specific port, with a specific username and password:

;; Start a server on port 8000 with basic authentication.
;; Don't publish any information about the server.
(porthole-start-server
 "spanish-navy-server"
 :PORT 8000
 :USERNAME "my_username"
 :PASSWORD "my_password"
 :PUBLISH-PORT nil
 :PUBLISH-USERNAME nil
 :PUBLISH-PASSWORD nil
 )
;; Remember, functions have to be exposed before they can be invoked remotely.
(porthole-expose-function "spanish-navy-server" 'revert-buffer)

In the Client:

Send a POST request to port 8000 with the JSON-RPC 2.0 request encoded in the body. Put the Basic Authentication credentials in the header.

In Python:

import requests

rpc_call = {
    "jsonrpc": "2.0"
    "method": "revert-buffer",
    # Note how keyword arguments are supplied. See the `json-rpc-server.el`
    # package for more information.
    "params": [":ignore-auto" ":noconfirm"],
    "id": 23084
}
# This is just an example. In production, you would need much more error
# checking.
response = requests.post("http://localhost:8000",
                         json=rpc_call,
                         auth=("my_username", "my_password"))
print(response.json())

See the json-rpc-server package for more information.

The JSON-RPC 2.0 Protocol

This server uses JSON-RPC 2.0 as the underlying protocol for executing functions. This is handled by the json-rpc-server package. For instructions on how to structure JSON-RPC requests for Emacs, see that package's README directly.

Symbols & Keyword Arguments

The JSON-RPC 2.0 protocol only allows you to send certain datatypes. Porthole modifies this syntax to allow you to send Elisp symbols. You should send them as strings, but tag them like you would in Elisp code, e.g. "'symbol" or "'another-symbol". Keyword arguments can be sent similarly: ":keyword", ":another-keyword"

In addition, because Elisp arguments are always positional (even keyword arguments are actually sent as a list), the standard JSON-RPC 2.0 method for sending keyword arguments is not supported. Arguments must always be sent as lists, just like in Elisp.

Here's an example:

from emacs_porhole import call

call("steamboat-server",
     method="a-cl-function",
     params=["positional-arg", "'positional-symbol",
             ":keyword-1", "value-1",
             ":another-keyword", "'symbol-value"])

In raw JSON:

{
  "jsonrpc": "2.0",
  "method": "a-cl-function",
  "params": [
    "positional-arg", "'positional-symbol", ":keyword-1", "value-1",
    ":another-keyword", "'symbol-value"
  ]

}

This is just an overview. See the json-rpc-server for a detailed explanation of how the JSON-RPC part works.

Writing a Client

Servers store their session information on a predictable file path, available only to the current user - that's how clients are able to connect automatically. The session file will be a session.json file with the following format:

{
  "port": 38790,
  "username": "9ce7abca51f93adca8e7239c91db41f5be0f925efddffddd51bdbae66bc03fb6",
  "password": "506c06f36d48f950b975783dba4c1bd9563d1c5aac30c7c86214d8804e4d3c9a"
}

The client should read from this file when it wants to connect. The file path will be predictable:

"{temp_dir}/emacs-porthole/{server_name}/session.json"

{server-name} is the name of the server. {temp_dir} represents a user-only temporary directory and it will be different depending on the platform. Here is how it's derived:

Windows

temp_dir is set to %temp%. The full path is:

;; This is the path on Windows
"%temp%/emacs-porthole/{server_name}/session.json"

Mac OS

temp_dir is set to $HOME/Library. The full path is:

;; This is the path on Mac
"$HOME/Library/emacs-porthole/{server_name}/session.json"

Linux

Linux is less predictable, so two methods are used.

temp_dir is set to the $XDG_RUNTIME_DIR environment variable if it exists. Normally, it will. If it does not, it is set to $HOME/tmp.

Thus, the full path is:

# This is the path on Linux
if exists("$XDG_RUNTIME_DIR"):
    "$XDG_RUNTIME_DIR/emacs-porthole/{server_name}/session.json"
else:
    "$HOME/tmp/emacs-porthole/{server_name}/session.json"

If neither $XDG_RUNTIME_DIR or $HOME exist, no session file is created.

Unknown Systems

Porthole will make an effort to run on unknown systems. If the system is unknown, it will use the same method as Linux.

What About Security?

By default, servers run on private ports. They are only available to localhost. Because of how Emacs handles TCP connections, any user will be able to connect to the server, but login credentials are only available to the current user.

Outside Localhost

If you want to expose your server on a public port, be careful.

None of the existing Emacs HTTP servers support HTTPS. The most secure form of authentication offered by any is Basic Authentication. In other words, login credentials have to be sent over the network in plain text. You should not allow direct connections to the server from outside localhost.

If you would like to publish the server over a wider network, the recommended way is to use an HTTPS proxy, such as Apache or Nginx. The proxy can forward external requests to the local RPC server. The emacs-web-server manual has a longer explanation with instructions.

Installation

It's easiest to install from MELPA. Make sure MELPA is in your list of repositories, then:

M-x package-install RET porthole RET

Once installed, require it with:

(require 'porthole)

List of Clients

Language	Link
Python	porthole-python-client

Do you have a client? Make a pull request to add it to the list!

FAQ

Can I run multiple servers? Yes. Porthole is designed to allow packages to build their own RPC system, without worrying about other packages.
Do you accept pull requests? Yes! If you like, you can open an issue first to discuss ideas.

jcaw/porthole