zcysxy/obsidian-pandoc-filters

Pandoc filters and an academic workflow for obsidian

title	author	type	created	modified	output	sup	state
Obsidian Pandoc	zcysxy	tool	2023-07-20 08:47:11 -0700	2023-10-30 22:14:24 -0700	pdf_document defaults pdf	[[Obsidian]] [[Pandoc]]	done

Pandoc Filters and an Academic Workflow for Obsidian

Note

• This workflow focuses on markdown -> latex -> pdf. There are many great plugins for other pipelines, such as Obsidian Webpage Export.
• Basic knowledge of pandoc, YAML, etc., is assumed.

Imagine your professor gives you a LaTeX template and asks you to scribe a lecture note,¹ while you have already taken the note in Obsidian with handy features like callouts and block links, and even advanced plugins like Pseudocode and TikZ. Spend another year to type it again or do some tedious manual conversion? 😩 Never! With one command within Obsidian and you are ready to submit! 🥳

²

This repository includes

• A workflow that helps you seamlessly convert your Obsidian notes to camera-ready PDFs
• Pandoc filters that transform Obsidian Markdown for pandoc conversion
  • callout.lua transforms Obsidian callouts into LaTeX environments and color boxes with labeling support
  • link.lua transforms wikilinks into LaTeX references, supporting heading links and block links with alias
  • codeblock.lua parses codeblocks for plugins, including TikZ and Pseudocode
  • image.lua parses image captions and attributes, supporting Obsidian image alias
  • shift_headings.lua shifts heading levels to avoid duplicate H1 titles, supporting customizable shift levels
  • div.lua transforms custom pandoc divs
  • preamble.lua loads your reusable preamble file, e.g., preamble.sty for obsidian-latex
• A defaults file that configures all the pandoc options and meta variables
  • ❗ Please substitute the placeholders in the defaults file with your own configurations
• Some templates that control the look of the generated PDF

Other recommended programs

• Other filters
  • cross-ref: Pandoc filter for cross-references
  • mermaid-filter: Pandoc filter for Mermaid diagrams
• Pandoc wrappers
  • panrun or my fork: read pandoc arguments from the note fromtmatter
• Obisidian plugin
  • obsidian-shellcommands

The workflow

1. Configure pandoc files

1. Install pandoc and recommended programs that suit your needs.
2. Copy the three folders filters, templates, and defaults to the directory you choose, e.g., obsidian_vault/config/pandoc/.
3. Customize the defaults/pdf.yaml file to configure your pandoc options and replace the placeholders.

2. Write the note and frontmatter

When you compose the note, it is a good practice to keep the syntax simple and compatible with pandoc and other Markdown programs. Add meta variables in the frontmatter (properties) that you want pandoc and your template to pick up, e.g.,

---
title: Obsidian Pandoc
author: zcysxy
uni: Obsidian University # this is a variable specified in templates/scribe.tex
---

3. Setup the shell command

Now, you are ready to generate the PDF outside Obsidian, by running the following shell command

pandoc --defaults obsidian_vault/config/pandoc/defaults/pdf.yaml note.md

There are many plugins that can help you run shell commands within Obsidian, e.g., obsidian-shellcommands My obsidian-shellcommands command looks like this:

4. Advanced configurations in the frontmatter

Under the current setup, to change a template, you can either change the template configuration in the defaults file or modify the shell command, adding the option --template. Both methods are not ideal and break the automation. What if we can specify the template in the frontmatter? However, this is a limitation of the vanilla pandoc (Issue#4627). In the issue, several programs are mentioned to solve this problem, e.g., my fork of panrun. ³ With panrun, we can specify the pandoc arguments like template in the frontmatter:

---
output:
  pdf_document:
    pandoc_args:
    defaults: pdf
    template: scribe.tex
---

And the shell command becomes

panrun note.md --data-dir obsidian_vault/config/pandoc/

Therefore, the actual obsidian-shellcommands command I am using looks like this:

In the future, perhaps we can

• Add more filters for
  • beamer
  • embeds
  • literature note links as citations
  • Obsidian comments
• Integrate with obsidian-pandoc-templates and obsidian-enhancing-export
• Directly read the preamble.sty file used by obsidian-latex

Customization guide

WIP

Footnotes

1. Here is a genuine compliment from my professor on my docs generated using this workflow. ↩

2. Interested in the note in the figure? Check out our new preprint Riemannian Adaptive Regularized Newton Methods with Hölder Continuous Hessians 🎉 ↩

3. This fork allows configurations in the frontmatter to override the defaults file, which is not supported in the original panrun. ↩