Bundled Spec

[quartzjer]

It would be great to have a "make pdf" that can turn the v3 markdown files into a combined spec. cc @temas

[bachp]

I have some experience with mkdocs but it only outputs html.
Sphinx allows to produce some nice pdfs but it is using reStructuredText and not markdown.

Then there is Pandoc that allows conversion between many formats, but I have not used it much.

[bachp]

I did some experiment with sphinx.
To do this I converted from Markdown to restructured text using Pandoc.
Then is used Sphinx to create a part of the spec.

Results:
PDF from Sphinx using Latex
PDF from Sphinx using rst2pdf

The sphinx project could also be uploaded to Read the Docs.

[quartzjer]

That turned out better than I expected, very cool!

Is there a way to automate this, perhaps make a v3/build/ dir with a Makefile, so that I could help tweak/improve it?

[bachp]

Sure I can add that.
However I think there are still some issues when converting from Markdown to reStructuredText like links between documents that are not correctly working.
So in the long therm if we want to go the sphinx route it might be worth to switch to rst. It also has some more advanced features like glossary term definitions which would address the JSON vs keyword problem mentioned in #128.

[quartzjer]

poke @temas to chime in too, since he went through a markdown->pdf project last year and should have some practical thoughts :)

[temas]

Just a note, I'll drop my workflow in here tomorrow.

[temas]

So, I just did a book over the summer and the process used was such:

• Original text was written in markdown, as the lowest common denominator agreement with the other author.
• MD was converted to HTML with a focus on the HTML being compatible with HTMLBook.
• Custom CSS was created for the look and design of the book
• The HTML and CSS was processed using Prince to generate the final forms.

I honestly can't remember what tool we used to do md -> html because we only had to do this step once in our case. We tweaked from the HTML after that. The biggest time sink is the CSS to get it to a style that you like. Overall it was a very pleasant process.

In our use case here we'd need some good focus on the conversion process md->html because we don't want to touch the HTML intermediate. This is definitely something I can put some time on unless @bachp is interested on running with it. I'm not married to any particular process either, just want to get it done.

[bachp]

I didn't know HTMLBook, but it looks interesting.
The reason I had a look at sphinx is because there were a number of nice documentations written with it (python, cmake, readthedocs) and it is able to produce many output formats (html, latex,pdf,epub,...). However I didn't use it peronally for more than my experiments. So I'm open.

Some problems I see, mabe you @temas can tell how you soved that with HTMLBook.

• Crossreferences did not work correctly after the conversion, because they referred to .md filey
• Organization of files so that it makes sense to read then in a linear fashion (book)
• Limited features of markdown (no glossary, no term definition)

[quartzjer]

If any of the md->html tools support the [foo][] footnote style references better I'd be happy to start using a different syntax to make linking easier?

[bachp]

Pandoc is able to understand the footnote syntax.

I will try to implement a solution using the following pipeline:

                                                   /==> Browsable on Read the Docs
Markdown ==[Pandoc]==> reStructuredText ==[Sphinx] ===> PDF document
                                                   \==> EPUB or Plaintext

I belive this would cover the most common use cases: Online and searchable on Read the Docs and offline as a PDF.
You can then tell me if this would satisfy the need

[quartzjer]

Sounds cool, I expect we'll have to do some significant reorg/formatting cleanup of all the markdown before v3 stable just for these kinds of purposes, so feel free to share anything you learn that will make it easier along the way, thanks!

[bachp]

Here are my first results you can have a look at it here.

The resulting Read the Docs.
The resulting PDF.

There are some open issues that still need to be solved:

1. Links between documents are not working. Sphinx uses special rST directives :doc: for that :-(
2. I was not able to do the Mardkdown -> reStructuredText conversion on Read the docs. (That's why I checked in the resulting .md.rst files)
3. The files need to be structured in a toctree

In order to solve issue 1. there is some post processing necessary after the conversion.
Issue 2 I don't know how to solve. One possibility would be to not use read the docs build the html on travis-ci and host it somewhere else, but that's a lot less comfortable.

[bachp]

Just to make sure. What is the expectation for this issue?
Just to create a PDF? Or also have the flexibility to have a alternative more approachable form of the documentation then just browsing github?

[quartzjer]

I'd like to keep the source docs all in markdown, but if there was an automated way to build "prettier" static html as well as a pdf from the markdown that would be amazing, I just don't know if it's really possible or easy, or what we have to do to the markdown to make it easier too.

I chatted w/ temas and he has some thoughts as soon as he's through c# debugging :)