Org-clones

~~This package is still being developed and tested. Feedback and bug reports are welcome. Use at your own risk.~~ This package is abandoned. The ideas are being translated to a new package. I am keeping this up in the event anyone finds the approach used here interesting. Otherwise, please don’t plan to use this.

Clone Orgmode headings so they appear in multiple locations, and sync edits between clones.

Why clones?

Anyone who has made an outline has run into the problem that an entry belongs in two places. For example:

* Animals
** Dogs
** Cats
* Things with four legs that I don’t like very much
** Tables
** Dogs

Oh no. The Dogs header belongs in two places. Now suppose that the Dogs header contains everything you know about dogs.

You could handle this with links, so that you have a source Dog header and subsequent headers simply link back to it. I do not like that solution. I want to clone the header so that it appears in different places in an outline, including in different files. When I edit one of these clones, I want the changes to sync to all the other clones automatically.

Clones are a type of transclusion, meaning that text from one file appears in another place, with edits synced back to the original source. It is a prominent featuree of Leo Editor, see https://leoeditor.com/tutorial-pim.html, which I wish I could use but it’s not in Emacs so it is dead to me.

My use case is that I maintain one org file with ongoing research. This file is full of well-written and researched academic work, half-thoughts, notes, links, etc. Sometimes I need to use that research in a paper I am writing. When I do so, I inevitably edit and expand (or contract) the work I have already done. I want this new work—even though it is being written in a different file—to be synced back to my source research file. Before now, there did not appear to be a good and simply way to do this.

I do not use org-roam or any other package involving the Zettlekasten, or other, note taking method. My brain is simple. I like outlines and only outlines. Anything beyond a simple outline results in diministed productivity.

Terminology

I use the following teminology in the package:

Node: an entry in an org outline
Headline: the headline text of the node, but does not include the headline’s todo state or tags
Body: everything after the headline, planning line, and property drawers until the next node

Basics

Only the headline and body of a node are cloned.
Tags, TODO states, priorities, planning lines, property drawers, logbooks, and closing note logs, are not cloned and remain independent. You can change these properties on an individual basis without affecting any clones. (See here for a demonstration in a “busy” file.)
Anything in the body of a clone (e.g., text, source blocks, tabes, etc.) will be cloned. Only the items listed above aree ignored.
Child nodes of clones are not cloned. Only the node itself is cloned. If you want the children to be cloned, you must clone them separately.
Clones can be reordered, promoted, demoted, and appear anywhere in an org outline or in a different org file. All that matters is that org-id can find it.

Installation

There are no outside dependencies. Place org-clones.el in your load path and (require 'org-clones). You can then activate (org-clones-mode). If you create a clone without turning on the mode first, org-clones-mode will be enabled automatically.

In the alternative, this use-package declaration includes all customizable variables, commands, faces, and my own keymap, which you can easily customize:

(use-package org-clones
  :hook
  ;; Note: `org-clones-mode' will only activate
  ;; `cursor-sensor-mode' if there are cursor-sensor-properties
  ;; in the buffer.  Therefore, you can safely add `org-clones-mode'
  ;; to `orgmode-hook' even if you don't plan to use it in every file. 
  (orgmode . org-clones-mode)
  :custom
  (org-clones-commit-edit-shortcut "C-c C-c")
  (org-clones-abort-edit-shortcut  "C-c C-k")
  (org-clones-start-edit-shortcut  "C-c C-c")
  (org-clones-jump-to-next-clone-shortcut "n")
  (org-clones-clone-prefix-icon "◈ ")
  (org-clones-empty-body-string "[empty clone body]")
  (org-clones-empty-headling-string "[empty clone headline]")
  (org-clones-prompt-before-syncing nil)
  (org-clones-use-popup-prompt nil)
  :custom-face
  (org-clones-current-clone ((t (:background "orchid" :foreground "black"))))
  (org-clones-clone ((t (:background "black"))))
  :bind
  (("C-; C-c C-m" . org-clones-store-marker)
   ("C-; C-c C-c" . org-clones-create-clone-dwim)
   ("C-; C-c C-d" . org-clones-delete-this-clone)
   ("C-; C-c C-j" . org-clones-jump-to-clones)
   ("C-; C-c C-u" . org-clones-unclone-this-clone)))

Usage

Clone creation

Put the cursor at the place you want to create a heading, run org-clones-create-clone, which will prompt the user to select a source node with (org-goto) and create a new heading at point.
Run org-clones-store-marker, then place the point at a new location (which can be a different org file) and run org-clones-create-clone-from-marker.
Run org-clones-create-clone-dwim creates a clone from a stored marker if there is one, and otherwise prompts the user for the source.

Display

Clones are marked by placing an icon in front of the clone’s headline. This icon can be customized by changing org-clones-clone-prefix-icon. The headline and body of a cloned node receive the org-clones-clone face.

When the cursor moves into the headline or body of a cloned node, an overlay is applied which includes the face org-clones-current-clone. At that point, the text in that field becomes read-only, and the user must press C-c C-c to begin to edit the clone. (C-c C-c continues to work in the usual orgmode way so long as the cursor is not inside a headline field.)

Viewing clones

To cycle through clones of the current node, run org-clones-jump-to-clones. Then, press n to jump from one clone to another in a loop. Exit with C-g or any other key.

Editing clones

When the cursor enters a cloned headline or body, the text becomes read only. To edit the text, type C-c C-c.
To complete an edit, type C-c C-c again. To discard the edit, type C-c C-k.
After the edit is completed, all clones will be updated automatically. (If you want an additional prompt before syncing, set org-clones-prompt-before-sync to non-nil.)

Uncloning a clone

If you do not want a clone to be synced, run org-clones-unclone-this-clone. The node’s ID will removed from all other clones, and the node’s :ORG-CLONES: property will be set to nil. The node will not otherwise be affected.

Caveats

Org-clones does not (currently) check for conflicts before syncing clones. Sync at your own risk.
Org-clones relies on cursor-sensor-mode. I have not profiled to see what type of slowdown one might experience in a large file due to cursor-sensor-mode. My files are not large enough for this to be a concern.
Org-clones currently relies on org-id. Org-id sometimes has problems finding the location of an id, especially in a file that has just been created. Before you blame org-clones for a clone not syncing, make sure the file you are using appears in org-id-locations. Make use of org-id-update-id-locations if you must. You can test whether org-id is working as it should by manually trying (org-id-goto "INSERT ID HERE"). If that does not work, org-clones will not work. I find that saving the file and creating a few ids with (org-id-get-create) in some dummy headers, and re-saving the file, eventually solves the problem. (I have used org-id for years before noticing this issue and only discovered it when testing this package. Perhaps you will not encouter it.)
If org-clones cannot find a clone, it does not remove the clone from the clone list automatically (due to the issues with org-id, supra, or other issues involving multiple files/computers).
If you try to create a new node while editing the body of a clone, you are asking for trouble. Org-clones will be confused, and make a mess of everything. I will figure out a good way to prevent this in the future. For now, don’t do it.

Custom variables, faces, and commands

Most of this is laid out above, but just in case:

Faces

Face	Usage
org-clones-current-clone	Applied to the headline or body of a clone, depending on whether the point is within the headline or body
org-clones-clone	Applied to the headline and body of every clone, regardless of whether the point is on the clone

Custom Variables

Variable	Behavior	Default value
`org-clones-commit-edit-shortcut`	Shortcut to commit an edit to a clone and sync all clones	“C-c C-c”
`org-clones-abort-edit-shortcut`	Shortcut to abort an edit and return the clone to its previos state	“C-c C-k”
`org-clones-start-edit-shortcut`	Shortcut to start editing a clone, when the cursor is in a cloned region	“C-c C-c”
`org-clones-jump-to-next-clone-shortcut`	Shortcut to cycle to the next clone after running `(org-clones-cycle-through-clones)`	“n”
`org-clones-clone-prefix-icon`	Icon which precedes the headline of any cloned node	“◈ ”
`org-clones-empty-body-string`	You’re not allowed to have a blank body in a clone. If you clone a node without a body, use this place holder	“[empty clone body]”
`org-clones-empty-headling-string`	I don’t know why anyone would clone a node without a headline, but in case you try, use this place holder	“[empty clone headline]”
`org-clones-prompt-before-syncing`	Do you want an extra warning before syncing clones?	nil
`org-clones-use-popup-prompt`	If you do want an extra warning, do you want it in the minibuffer (default) or a pop up window?	nil

Commands

Org-clones provides the following interactive commands:

Command	Effect
`org-clones-create-clone`	Create a clone of the node at point, directly below the current node.
`org-clones-store-marker`	Store the current mode to create a clone in a different place
`org-clones-create-clone-from-marker`	After storing a node with `org-clones-store-marker`, create a clone of that node at point
`org-clones-create-clone-dwim`	Create a clone from the stored marker if one is stored; otherwise, prompt the user for the source node

How it works

Defining a headline

Org-clones automatically moves and progress cookie to the and of a headline, before the tags. It also assumes that the {{{results}}} of an inline source block will appear at the end of a heading, but before the tags and before any progress cookies. With those adjustment, the format of a headline is:

Todo Priority-cookie Comment Headline-text Inline-results Progress-cookie Tags

The only thing org-clones will sync is the headline-text.

Functionality

Clones are tracked via the Orgmode property :ORG-CLONES: which contains a list of IDs which correspond to other cloned nodes.
A cursor-sensor-function property is placed on each headline and body of each node.
When the cursor enters that field, org-clones places a transient overlay over the field to alert the user that they are on a cloned node.
Org-clones also makes the field read-only. This prevents inadvertent edits. Because clones only become read-only when the cursor is within the field, you can still kill and yank headlines, etc., without running into issues with the text being read only.
The transient overlay has a keymap which uses org-clones-start-edit-shortcut, bound to C-c C-c by default.
Once the edit mode is invoked, the read-only text property is removed, the header-line appears to remind the user they are editing a clone and showing the shortcuts to commit or abandon the edit. These shortcuts are set with org-clones-start-edit-shortcut (C-c C-c by default) and org-clones-abort-edit-shortcut (C-c C-k by default).
When the user terminates the edit, the read-only text properties are replaced, the header-line is reset to its previous value, and the transient overlay is replaced. Other variables (recording the state of the node before the edit, etc.) are reset to nil). If the user has committed the edit, all other clones are synced automatically.
When the cursor exits a cloned field without edits, the transient overlay (and its read-only property) is removed.

Other transclusion efforts

Here are other Emacs transclusion efforts (or discussions of such efforts):

https://github.com/alphapapa/transclusion-in-emacs

https://github.com/justintaft/emacs-transclusion

https://github.com/gregdetre/emacs-freex

Change log

[2020-09-03 Thu] Add org-capture hook to capture and sync edits to a clone via org-capture
[2020-09-02 Wed] Fix progress cookies and unrelated issue concerning org-clones improperly placing the body of a parent node inside a child node when syncing clones

legalnonsense/org-clones