alphapapa/org-ql

Org 9.7's deferred element property resolution breaks dynamic blocks

jezcope opened this issue · 4 comments

jezcope commented

OS/platform

Linux

Emacs version and provenance

emacs 29.3 (PGTK) installed from nixpkgs

Emacs command

emacs

Org version and provenance

org mode 9.7, installed via straight with doom emacs

org-ql package version and provenance

org-ql 0.8.6 (c937098), installed via straight with doom emacs

Actions taken

Create the following minimal example file:

* Dynamic block

#+begin: org-ql :query "todo:"
#+end:

* Tasks

** TODO First thing
** DONE Thing the second
** DOING Thing 3

Place the point on the dynamic block and press C-c C-x C-u.

Observed results

A corrupted table is inserted into the dynamic blog:

* Dynamic block

#+begin: org-ql :query "todo:"
| Heading | Todo | P |   |   |
|---------+------+---+---+---|
| [[P        | ][P    | ]]  |   |   |
| [[sks][sks]]     |      |   |   |   |
#+end:

* Tasks

** TODO First thing
** DONE Thing the second
** DOING Thing 3

In this example, no messages are shown in the echo area other than Updating dynamic block ‘org-ql’ at line 3...done. Depending on the content of the file, errors may occur. The one I ran into was "Empty link" from org-ql-search--org-make-link-string at org-ql-search.el#L334.

Expected results

Dynamic block is correctly filled with the matching tasks:

* Dynamic block

#+begin: org-ql :query "todo:"
| Heading                      | Todo  | P |
|------------------------------+-------+---|
| [[First thing][First thing]] | TODO  |   |
| [[Thing 3][Thing 3]]         | DOING |   |
#+end:

* Tasks

** TODO First thing
** DONE Thing the second
** DOING Thing 3

Backtrace

No response

Etc.

I ran into this completely incidentally while doing a minimal implementation of #151. The cause seems to be this sequence of events:

  1. User runs org-dblock-update
  2. Org then deletes the content of the dynamic block, per https://orgmode.org/manual/Dynamic-Blocks.html
  3. org-dblock-write:org-ql obtains a list of elements using org-ql-query: note that these are stored with references to character positions in the buffer, and some element properties are deferred until requested
  4. org-dblock-write:org-ql inserts the table header, which changes the character positions of all text following it
  5. org-dblock-write:org-ql loops over the elements fetching properties using character positions which have been invalidated by (4), with unpredictable results

The fix seems to be to build the table as a string first and then insert it all together, so that the character positions used in the element structures remain valid. Alternatively, org-element-properties-resolve could be called on each element at some point before the first call to insert. I have implemented a quick fix so I'm happy to make a PR.

alphapapa commented

Indeed, I can confirm that this happens when using Org 9.7, but not with Org 9.6. So this is some of the fallout from the deferred-element-properties change. I've been wondering how many of these kinds of bugs are going to get reported on my packages now.

It's additionally complicated by the fact that we must also support older Org versions that don't defer property resolution. @yantar92 What is the recommended solution to this problem?

I have implemented a quick fix so I'm happy to make a PR.

You're welcome to propose something, but please see https://github.com/alphapapa/org-ql?tab=readme-ov-file#copyright-assignment

  • 👍1
yantar92 commented
alphapapa commented

@yantar92 Thank you.

alphapapa commented

This should be fixed on v0.8.7 and master now. Thanks.