lbarraga/vsc_user_docs

VSC user training material and documentation

UGent HPC - pdf2wiki

• UGent HPC - pdf2wiki
  • Introduction
  • Requirements
  • Configuration
  • Build
  • Run website locally
  • Add new OS
  • Macros
    • Markdown macros
    • Python macros
      • Example

Introduction

UGent HPC currently has user manuals in LaTex. Sources can be found here: https://github.com/hpcugent/vsc_user_docs Pdf versions can be found here: https://www.ugent.be/hpc/en/support/documentation.htm

There are different variants for each VSC site and for each operating system.

OS selection

The OS selection is stored in localStorage and kept for , and can be bypassed by going to any index pages (i.e. one without the OS), and adding the hash #force_select_OS (case-sensitive).

Build

You can use the build.sh wrapper script to take of the steps below, all options including --help are passed to the build.py tool.

Make sure to run export INSTALLDEPS=1 before building for the first time. This will install the dependencies in a subdir pypkgs, and then build the docs. You typically only need to set this variable once, unless some of the dependencies are updated.

Using export WEBSERVER=1 before building will first build the docs, and then start a local webserver that provides the docs (and if supported by the OS, open a new browser tab with the page loaded).

Dependencies

Python version 3.6 and greater is required.

Install requirements by running:

python -m pip install -r requirements.txt

Install custom plugin, which is present in this repository (at least for now):

python -m pip install -e custom_plugin

Install computational macros - Pyton macros:

python -m pip install -e computational_macros

Build

Usage:

python build.py [options]

Options:

-l, --skip-docs           Build only landing page. Skip building documentation pages.
-d, --skip-landing-page   Build only documentation pages. Skip building landing page.
--ignore-errors           Ignore errors in partial mkdocs builds and continue with build process.
-v                        Enable verbosity.

Without options, it will build all documentation defined in build_config.yml and also landing page.

By default, the main build script fails and clean the build directory if any of the partial builds fails. To override this behaviour and continue even if some documentation will not be built correctly, use the option --ignore-errors described above.

python build.py

In directory ./build/ there will be built landing page called HPC for prettier url and site-OS hierarchy. According to config, there will be folder structure with documentation content.

Run website locally

Move to root directory of your static website:

cd build

Run simple HTTP server:

python -m http.server --cgi 8000

Visit localhost:8000 and start looking around your documentation.

Check dependencies

Get piprot (no or outdated package on pypi (oh the irony))

cd /tmp/
git clone https://github.com/sesh/piprot.git
cd piprot
python setup.py install --user

In this repo, do

piprot requirements.txt

Change as you see fit and try it.

Configuration

All .yml files in the mkdocs are supposedly generated and removed if unknown.

Every site has 3 generated configuration yamls in mkdocs dir one for each site and OS combination. Template is in config/templates dir named hpc.template. Naming convention of generated files is like this: hpc_Antwerpen_Linux.yml.

Configuration for OS picker utility is generated in mkdocs dir one for each site. Template is in config/templates dir named os_pick.template. Naming convention of generated files is like: os_pick_Antwerpen.yml.

Common constants are defined in constants.yml.

Configuration for landing page is defined in landing_page.yml.

Configuration for documentation building script build.py is in build.yml.

When editing content, only specific site-OS yaml could be affected. When adding or removing new site or OS except for site-OS yamls also other yamls are affected.

Add new OS

You might want to add a new OS or divide Linux to some distros. You need to follow these steps: 0. Let's pretend you want add Arch Linux distro to Gent site.

1. Add the OS in build.yml

In fact, this way you can add anything what you want. The build system will treat it as some OS.

Macros

Markdown macros

You can create macros in markdown as mkdocs documentation specifies. On top of that there is a support for Python macros described in following section.

Python macros

You can write Python script and use the output as content in the markdown. This feature is implemented as variable injection. That brings some specific steps to follow. The restrictions or rules are valid for current version and can vary in the future.

1. All scripts are placed in module computational_macros in package scripts.
2. Each script should contain exactly one method with the same name as the file. (Of course without the .py file extension.)
3. The method should return string object with desired output. This will be stored in the variable with again the same name. You have to consider markdown and HTML formatting!
4. Each script is automatically loaded, so you may want to add some prefix to prevent existing variables conflicts, respectively overriding.

You can return anything from Python macro, but it should make sense in context of mkdocs usage. For example, you can generate some JavaScript code which will provide some interactive stuff in target document page.

Implementation

You can find implementation in custom_plugin.py. It is single method called gen_content_from_macros()

Example

See example scripts in folder scripts.

See usage in file account.md.
Built page can be access only by knowing its location and that is: <server_name>/Antwerpen/Linux/HPC/examples/Antwerpen/Linux/HPC/account

Tips

Sitemaps

To generate correct sitemap data, you need to set the site_url in the hpc.template and os_pick.template. You must also make sure that any nested sites (like HPC, where every mkdocs.yml config generates a single sitemap) have merged sitemap data. For HPC, this is already taken care of in the build.py script.

Redirects

There is support for the mkdocs redirects module, but in case of HPC docs, you need to set the redirect in both the hpc.template and the os_pick.template and make sure the redirect plugin is configured after the ugent one.