Improve caching options (e.g. disabling to allow consistent side-effects)

OS/platform

Linux

Emacs version and provenance

Emacs 29.2, compiled from scratch

Org version and provenance

9.6.15, bundled with Emacs

org-ql package version and provenance

v0.8.5 (shipped with doom-emacs)

Description

RANT: It's not really obvious that org-ql-select :action gets cached by org-ql and IMHO :action is a bit a misleading name for a function that is supposed to be pure, :map would have been a better choice.

FEATURE REQUEST: it would be great to have a parameter to avoid the :action being cached, without having to empty the whole cache.

Etc.

No response

This is covered in the documentation. Please see https://github.com/alphapapa/org-ql#caching as well as https://github.com/alphapapa/org-ql#function-org-ql-select, which has extensive examples.

By the way, you might want to work on your delivery. This report lacked a greeting of any kind; instead the first thing I'm hit with as the lone, volunteer/free-time developer, is "RANT", which doesn't seem very friendly.

I'm sorry for the wording, there was absolutely no intent to be rude, I was just trying to be terse. I'll try to keep more in mind the point of view of volunteer maintainers in the future.

The caching is indeed extensively documented on github but it isn't cited in the function documentation:

org-ql/org-ql.el

Lines 336 to 375 in d5269bb

    
           (cl-defun org-ql-select (buffers-or-files query &key action narrow sort) 
        
             "Return items matching QUERY in BUFFERS-OR-FILES. 
        
           BUFFERS-OR-FILES is a file or buffer, a list of files and/or 
        
           buffers, or a function which returns such a list. 
        
           QUERY is an `org-ql' query sexp (quoted, since this is a 
        
           function). 
        
           ACTION is a function which is called on each matching entry with 
        
           point at the beginning of its heading.  It may be: 
        
           - `element' or nil: Equivalent to `org-element-headline-parser'. 
        
           - `element-with-markers': Equivalent to calling 
        
             `org-element-headline-parser', with markers added using 
        
             `org-ql--add-markers'.  Suitable for formatting with 
        
             `org-ql-view--format-element', allowing insertion into an Org 
        
             Agenda-like buffer. 
        
           - A sexp, which will be byte-compiled into a lambda function. 
        
           - A function symbol. 
        
           If NARROW is non-nil, buffers are not widened (the default is to 
        
           widen and search the entire buffer). 
        
           SORT is either nil, in which case items are not sorted; or one or 
        
           a list of defined `org-ql' sorting methods (`date', `deadline', 
        
           `scheduled', `closed', `todo', `priority', `reverse', or `random'); or a 
        
           user-defined comparator function that accepts two items as 
        
           arguments and returns nil or non-nil.  Sorting methods are 
        
           applied in the order given (i.e. later methods override earlier 
        
           ones), and `reverse' may be used more than once. 
        
           For example, `(date priority)' would present items with the 
        
           highest priority first, and within each priority the oldest items 
        
           would appear first.  In contrast, `(date reverse priority)' would 
        
           also present items with the highest priority first, but within 
        
           each priority the newest items would appear first."

this ticked wasn't supposed to be a bug report anyway, I just opened this feature request because the github documentation says "Note: Future improvements will allow the cache to be more easily disabled or cleared" but I didn't see this improvement tracked.

Anyone interested in developing or using this feature could reference this ticket to track the development.

No problem. I appreciate concise communication as well.

It hasn't been a priority to work on the caching (other than ensuring that it works properly). I don't mind an issue to track it.

In the meantime, you should be able to work around it in your own code by binding org-ql-cache to an empty hash table around your calls to org-ql functions.

as a workaround, I'm doing something like this (I only care about the side-effect and not about the returned value)

  (let ((results (org-ql-select (org-agenda-files)
                   '(and (or (deadline :on today) (scheduled :on today) (ts-active :on today)))
                   :action 'element-with-markers)))
    (save-excursion
      (dolist (heading-res results)
        (let ((marker (plist-get (nth 1 heading-res ) :org-marker)))
          (set-buffer (marker-buffer marker))
          (goto-char (marker-position marker))
          ;;do the action here...
          nil))))

As I said, this should be a sufficient workaround:

(let ((org-ql-cache (make-hash-table)))
  (org-ql-select (org-agenda-files)
    '(and (or (deadline :on today) (scheduled :on today) (ts-active :on today)))
    :action '(DO-ACTION)))

Also note that the cache is invalidated with every change to a buffer, so the cache may not be as inconvenient as you think.

	(cl-defun org-ql-select (buffers-or-files query &key action narrow sort)
	"Return items matching QUERY in BUFFERS-OR-FILES.

	BUFFERS-OR-FILES is a file or buffer, a list of files and/or
	buffers, or a function which returns such a list.

	QUERY is an `org-ql' query sexp (quoted, since this is a
	function).

	ACTION is a function which is called on each matching entry with
	point at the beginning of its heading. It may be:

	- `element' or nil: Equivalent to `org-element-headline-parser'.

	- `element-with-markers': Equivalent to calling
	`org-element-headline-parser', with markers added using
	`org-ql--add-markers'. Suitable for formatting with
	`org-ql-view--format-element', allowing insertion into an Org
	Agenda-like buffer.

	- A sexp, which will be byte-compiled into a lambda function.

	- A function symbol.

	If NARROW is non-nil, buffers are not widened (the default is to
	widen and search the entire buffer).

	SORT is either nil, in which case items are not sorted; or one or
	a list of defined `org-ql' sorting methods (`date', `deadline',
	`scheduled', `closed', `todo', `priority', `reverse', or `random'); or a
	user-defined comparator function that accepts two items as
	arguments and returns nil or non-nil. Sorting methods are
	applied in the order given (i.e. later methods override earlier
	ones), and `reverse' may be used more than once.

	For example, `(date priority)' would present items with the
	highest priority first, and within each priority the oldest items
	would appear first. In contrast, `(date reverse priority)' would
	also present items with the highest priority first, but within
	each priority the newest items would appear first."