Mischback/mischback.de

Build pipeline

Opened this issue · 2 comments

Mischback commented

Create an actual build pipeline:

Stages

  • Staging
    • CSS assets -> included in #126
      • introduce logic for a development build -> included in #126
      • provide the actual cache busting setup -> included in #126
        • compilation of stylesheet from theme/mischback/_src/style/style.scss into theme/mischback/static/style.css
        • run sphinx build to generate the HTML output (in .build); this step will place the stylesheet in .build/_static/style.css
        • run PostCSS against .build/_static/style.css
          • using the HTML output as input for purgecss
          • autoprefixer
          • cssnano
        • run PostHTML to perform the actual cache busting
        • run a custom Python script to perform the actual cache busting --> #126

Dependencies

Reference

  • colorizer, which was one of the sidetracks that significantly impacted building this website, has the additional benefit of having implemented a working cache busting in its v2 branch.
  • Related: #61
Mischback commented

Build Pipeline

Stage 0: Development

  • the most recent code lives in the development branch
  • development happens in dedicated feature branches, which are then used in a pull request against the development branch
  • PRs are processed with a dedicated workflow through Github Actions:
    • linting (several different linters for the different languages, e.g. black for Python, stylelint and prettier for SCSS/SASS)
    • building the project includes several stages, including compilation of SCSS/SASS and TS/JS sources and running sphinx to actually create the static HTML output. Management of the actual build process is handled by the project's Makefile.
    • validating the created build artifact runs several HTML validation tools against the (generated) output to ensure semantically correct HTML. It does NOT check, if everything is working!
  • Please note: The website's images are not part of the repository. Their existence is not checked! The (generated) build artifact of the building step is not ready for deployment!.
  • Focus of this stage and the associated workflow is code quality.
  • MISSING / TODO:
    • Tests
      • Unit Tests for Python code, especially the utility scripts; try to provide unit tests for the (custom) sphinx extensions aswell
      • Unit Tests for TS/JS code (how to unit test a codebase with target browser?)
      • Integration Test for (compiled) JS code (at least target major browsers)
    • Validation of (compiled) CSS stylesheet

Stage 1: Staging

  • the current release is always the last commit of the main branch
  • staging is initiated by performing a PR against main
  • PRs are processed with a dedicated workflow in Github Actions:
    • linting as described above; this is only a precaution, result should not deviate from development branch.
    • building: This step is a heavy deviation from Stage 0:
      • obtain image sources (tbd)
      • create responsive images
      • create (minified) CSS stylesheet
      • create (minified) JS
      • run sphinx build process
      • cache busting
      • tidy HTML sources
    • validating: This step is a deviation from Stage 0:
      • (still) perform HTML validation as in Stage 0
        • include checks for image sources!
      • perform CSS validation
      • check for moved documents
        • obtain current live sitemap.xml
        • launch build artifact in a webserver (local/Docker) and try to access documents by their live URI
        • This should provide warnings, but does not FAIL the build
      • ensure no 404
        • launch build artifact in a webserver (local/Docker) and scrape the whole site (obviously don't follow external links; they are covered already by sphinx linkcheck and possibly by the HTML validators).
      • Integration Tests / End-to-End Tests
        • Selenium, Playwright

Stage 2: Release

  • a workflow is triggered for a commit to main branch; the commit should be a merge commit from a staging branch
  • it is an extension to the workflow described in Stage 1:
    • pick up the artifact from building and apply maximum brotli and gzip compression to everything
    • re-upload as the final artifact
Mischback commented

Resources