/polygot

Multi-format Static blog generator with almost only POSIX shell

Primary LanguageRustGNU Affero General Public License v3.0AGPL-3.0

Description

A static-site blog/website generator designed for the international, modern web avoiding JavaScript without sacrificing interactivity.

This project consists primarily of three parts:

  • A simple build/make script for building from 'source' to 'public', automating SASS/LASS/sh/etc. (Checkout entr for file watching and Falkon automatic reload (hot reload))

  • A Rust backend for compiling posts, and a simple CLI script for managing posts

  • A starter website showcasing the CSS for spoilers, responsive (adapts to phone/computer/tablet), drop down menus, etc.

Why another static-site generator?

When researching better ways to taking notes or wanting to compile knowledge, I found that Office software come and go; only plain text stands the test of time and can truly follow you to every device. Although WYSIWYG editors are the norm, searching just a little deeper leads to Markdown as striking the appropriate balance between distraction-free simplicity and just enough formating.

However, with uniformed opinion on the rise, the lack of meaningful support for citations kills Markdown for me. (Also tables, including external files, etc.) In its place, there is Asciidoctor, Commonmark, LaTeX, Groff, SILE, raw HTML, MediaWiki markup, Org-mode, RMarkdown, Scribble…​ The search for a good markup language is time-consuming and intractable; I still have not found a markup language I am satisfied with.

Thus, I wanted something that could deal with any markup language. This precludes most static-site generators. Also, as speaker of more than one language, I would like my blog platform to support me writing in multiple languages from a single file. And thus this project was born.

Along with being a site/blog generator, the starter site commits to no JavaScript unless absolutely necessary. JavaScript is unnecessary for many sites; moreover this project demonstrates it. Following this theme of building up from the basics, I purposefully chose to be very minimal on depending on external crates.

Speed is also a large concern for me. LaTeX and Bibtex are unacceptably slow for me. I have not yet decided on a solution for bibliographies.

Instructions

Nothing other than POSIX shell is truly required.

  • Asciidoctor/Pandoc/Emacs/etc. depending on what markup language you want to use

  • Pygments: This is what I currently using for code syntax highlighting (sudo gem install pygments.rb or gem install --user-install pygments.rb, will need python installed for this)

  • POSIX shell: pretty much any Linux/Mac distribution. On Windows you can get this via WSL or via bash by install git (gives you the bash shell)

If you want to be able to compile

  • cmake: Required for building FileTime dependency, this allows me to set last-modified time cross-platform.

  • rust: stable is fine

To use this, you just need to git clone. I plan to include the compiled binary as part of the git, but until then, you must run make.sh build-rust first.

From within a shell/console (dash, bash, fish, etc.), you interact with blog.sh and make.sh. blog.sh opens files with $EDITOR by default (typically set by .bashrc); you are expected to customise this command to your use case.

Design Goals

Reader-facing

  • Static site generation

  • Tags

  • Automatic RSS/Atom feed generation from blog posts (@TODO)

  • Table of contents in sidebar

  • All features supported without JavaScript (except scrollspy, @TODO)

  • Linking between related posts into series (currently sorts by date created) (WIP)

Writer-facing

  • Able to use any markup format you wish (md, adoc, org, etc.). Supporting a file format requires implementing a simple API in config/api

  • Use shellscript as a templating markup language

  • Specify by output file locations via frontmatter, e.g. "{year}/{month}{day}/{filename}"

  • Write multilingual posts with languages side by side in the source files for easy comparison.

  • Support for shared navigation bar and footer etc. See usage of config/combine.sh in build.sh and

  • Builds sass/lass out of the box. Compilation is quite easy to customise (edit build.sh)

Why not just use a Makefile

Two of the subcommands of the rust portion of the code are dedicated to comparing and setting the 'last updated' metadata of files.⁠[1][2] This and the entirety of build.sh duplicate Makefile functionality.⁠[3] I do this because:

  • The original design of using primarily POSIX is to make this easily customisable and portable. Although POSIX shellscript is a bad language, it requires no setup on POSIX machines (and just installing git-scm on Windows). As this is a CLI tool, there is no need to learn any other language to customise this.

  • Ideally we be writing build scripts in real programming languages (not even POSIX shell), not adding needless complication with more syntax to learn. Again, one less language to learn.

  • I may be wrong, but there is no way to really recursively traverse a directory tree and compile all the files in said tree.

  • Shellscript serves as a reasonable templating language. More reason to stay with shellscript as the only cognitive burden

Customisation

Although this static site generator is tailored for customisation via shellscript, you can.

To support a file extension, it needs to implement three subcommands, see rust/fileapi.rs for where it is implemented or see config/api for examples.

Writing blog source files

In posts, to associate a section to a language, you must have api_set_lang[4] in a comment.⁠[5] e.g. in Asciidoctor (comments are //):

Defaults to an ALL view.
So this shows up in 'en', 'jp', and 'zh'

//api_set_lang: en jp
This only shows up in 'en' and 'jp'

// api_set_lang: ALL zh
This will be in all views. ('en', 'jp', 'zh')

There are two special languages: * and ALL, both set the following to text to be included in every valid language view of the post. Tags are whitespace delimited. There are a couple of characters blacklisted. See TAG_BLACKLIST in rust/helpers.rs

Frontmatter

Frontmatter is expected to be extracted via the user-defined file extension api Frontmatter is expected to be of the format {key}:{value}. A newline is the row delimiter (can have blank lines).

e.g. This a sample output of adoc frontmatter

title:        My First Blog Post
author:       John Doe
date-created: Tue, 22 Sep 2020 23:59:22 +0800
date-updated: Tue, 22 Sep 2020 23:59:22 +0800
tags:         travelling news work

The tags specified here are used to tag. They obey the same rules as language tags in the above section. Note: Values are trimmed (leading and trailing whitespace are removed).

Output location

TODO: write this

See also

  • Raph Levien’s pulldown-cmark' (written in Rust! Also CommonMark is pretty cool! The better Markdown)

  • Eric Radman’s entr, a CLI tool for watching for file changes

  • John MacFarlane’s Pandoc

  • The 'shell' branch of this project for more hardcore POSIX shell. This started as pure POSIX shell project

Similar static-site blog generators

TODO: Review links


1. This could have also been implemented with touch -m which is POSIX, but some platforms do not come with touch out of the box.
2. "POSIX.1-2017: Utilities". IEEE and The Open Group. 2017. Last Accessed: 2020-09-18.
3. "How does make know which files to update" Stack Exchange, Inc. Last Accessed: 2020-09-18.
4. api_set_lang is defined in rust/post.rs
5. Comments are provided by the relevant user-defined file-extension api as found in config/api