Embedding quartodoc outputs

[cpcloud]

In the ibis documentation we have many places where we want to insert quartodoc output inside of another non-quartodoc-generated document.

It's currently possible to do this with the quartodoc API, but it's quite challenging, and easy to produce API docs that don't match the properly quartodoc generated pages. For example, interproject links are not rendered when using the API as we are in ibis.

Here's a tiny Python module we've built to help us with this task: https://github.com/ibis-project/ibis/blob/master/docs/backends/_utils.py

I would love a way to embed API docs in a qmd that doesn't require me to write so much code. Perhaps there's a world where something like this exists?

arbitrary.qmd:

... prose ...

```{quartodoc}
# yaml specification for API docs
```

... more prose ...

[machow]

I'd love to be able to figure something like this out for quartodoc! Right now, the big challenge is orchestration between quartodoc and quarto.

Constraints

The big challenge, I think, comes from these constraints:

• quarto's native cross-references only work for books, so we made an interlinks quarto filter.
• quarto filters only support lua
• as a result, we often need to produce intermediate artifacts when quartodoc build is run
  • e.g. the doc qmds themselves
  • e.g. _sidebar.yml tells quarto how to generate the API sidebar
  • e.g. objects.{json,txt} is used by the interlinks filter to produce cross-references

There is also one important caveat for quarto filters:

• Pages are rendered in a single pass. This is when filters are run, so things like the interlinks filter need all cross-reference data up front.

Potential solutions

docstring stubs and the includes shortcode

In the quartodoc config in _quarto.yml, we add a section of doc partials to generate. These are rendered docstrings, that can be added to a qmd, using the includes shortcode (e.g. {{< include api/_MySqlBackend.qmd >}}).

• problem: we won't know where MySqlBackend documentation ends up living, for interlink purposes (it would need to be specified in the config? afaik there's no good workaround, due to the single pass rendering)

Maybe something like this?

quartodoc:
  package: quartodoc
  sections: ...
  stubs:
    # these get output to api/_stubs.qmd or something
    # you can include them in a doc using {{< include api/_stubs.qmd#quartodoc.MdRenderer >}}
    - contents:
      # contents here is specified in the same way it would be in the "sections:" block
      - get_object
      - name: MdRenderer
        members: []


# specify where in the site the stubs will end up
# this is necessary because quarto websites render in a single pass, so there is
# no way it can know how to cross-reference, unless it is specified up front
# (note that quarto books are an exception, and take 2 passes to support cross-reference)
interlinks:
  extra:
    # could probably omit the .qmd
    "quartodoc.MdRenderer": "path_to_where_it_was_inserted.qmd" 

In conclusion...

I'm not in love with the proposal above, but hopefully it helps with some of the constraints / dynamics at play 😓