Switch to MyST Parser

[jpmckinney]

We're currently using my fork of recommonmark and a specific commit of commonmark.py (until > 0.9.1 is released), because the released versions don't work with Sphinx.

https://myst-parser.readthedocs.io/en/latest/index.html was built in 2020, and seems to work. It uses https://pypi.org/project/markdown-it-py/ instead of commonmark.py. We can see if anyone has written about their switch.

[jpmckinney]

Follow-up: Switch from recommonmark to MyST-Parser in:

• ocds-babel open-contracting/ocds-babel#13 (waiting on executablebooks/mdformat#129)
• extension_registry.py open-contracting/extension_registry.py#31 (waiting for ocds-babel)
• extension-explorer: need to replace commonmark with markdown-it-py

[jpmckinney]

Steps from open-contracting/standard#1213

• requirements
  • Remove commonmark and recommonmark dependencies
  • Use Sphinx >= 3.5.2
  • Add myst-parser and mdit-py-plugins
• conf.py
  • Remove recommonmark
  • Add myst_parser extension
  • Set myst_enable_extensions = []
• Markdown
  • Change orphan pages to front matter (404.md)
  • Use {download} role or a tag if there are warnings
  • Convert raw HTML where possible (remove markdown=1)
  • Change all table directives to use :widths: auto
  • Change eval_rst to {eval-rst} (covered by script open-contracting/standard#1247 (comment))
  • Use {csv-table} instead of eval_rst with codelisttable (open-contracting/standard#1247 (comment))
  • Change {eval-rst} to {some-directive} style (open-contracting/standard#1247 (comment))

Done for:

• standard (need to upgrade MyST-Parser)
• standard_profile_template
• infrastructure open-contracting/infrastructure#302
• european-union
• government-procurement-agreement
• public-private-partnerships (switch RST to Markdown tables)

[jpmckinney]

Should also fix links to headings. Our options:

• https://myst-parser.readthedocs.io/en/latest/using/syntax-optional.html#auto-generated-header-anchors: Best option. Automatic headings and Markdown syntax.
• https://myst-parser.readthedocs.io/en/latest/using/howto.html#automatically-create-targets-for-section-headers: This syntax is less familiar to our technical writers.
• https://myst-parser.readthedocs.io/en/latest/using/syntax.html#targets-and-cross-referencing: Headings are not automatic. Can use Markdown syntax.

General process, matchind *.md files:

1. Check no unusual link formats: \] +\(
2. Check no fragments with non-/-terminated paths: \]+\((?!http)\S+[^/]#
3. Perform substitution: \]+\((?!http)\.\./(\S+)/# for ]($1.md#
4. Build docs and fix warnings

Done for:

• standard
• standard_profile_template
• infrastructure open-contracting/infrastructure#302
• european-union
• government-procurement-agreement
• public-private-partnerships (skipped: uses too many anchors in tables like #release-schema.json,,contracts/0/agreedMetrics)

[jpmckinney]

All done!