Use with Literate.jl (Feature request?)

[briochemc]

Is it possible to use LiveServer.jl with Literate.jl? I guess it would need a similar function to the servedocs function (serveliterate?). Unless it can already be done by having Documenter.jl run Literate.markdown with documentation=true in the make.jl file? If so is there an example of that workflow somewhere?

I am looking for an efficient workflow to write documentation (and notebooks and tests using Literate), and it would be great if I could edit the source file on a half of my screen and see how the generated markdown/docs look like at each save. So I am open to any general advice on editing and generating those if you have any! 🙂

[fredrikekre]

It should just work, if you build the files in the make.jl file.

Edit: You need to have the source-files in /docs/src or they will not be watched. It would be nice with a way to add additional files to watch for servedocs.

[fredrikekre]

Test repo: https://github.com/fredrikekre/LiterateLive.jl But it seems to get stuck in an infinite loop repeatedly calling the callback function so something is wrong.

Edit: Updated the repo, had to only trigger the callback for source-files to avoid the infinite loop.

[asprionj]

Hi guys, I don't know Literate in detail. Depending on the inner workings, including LiveServer to serve the resulting web pages could be fairly easy. However, we should separate concerns:

• LiveServer serves static files and updates the view when these files change.
• Markdown converters (such as some static-site generators) convert markdown files to web pages (i.e. HTML, CSS, and possibly image files). They can hook into LiveServer to have their generated site served to a browser and live updated, see for example JuDoc.
• Literate-programming tools / markdown generators. These have code or some other (usually text) format as input and create markdown files.

Literate belongs to the last group, right? Or can it also directly produce the corresponding web page? @fredrikekre : does your test repo work now? I think to keep concerns separated, this functionality should become part of Literate, not LiveServer.

[fredrikekre]

Yea it works, and I don't think any extra functionality is needed anywhere. Well, a fixed servedocs of course, see #69. And would be nice if there was a way to include extra sources to watch, if you keep things outside of docs/src. Maybe a kwarg to servedocs.

[asprionj]

Do you mean specific files? Those could be just added to the docwatcher. If you mean entire directories to watch as well... they would have to be passed forward to scan_docs! (or a new, separate add_files_in_directory function) to include all files in those directories in the watcher.

[fredrikekre]

Yea, maybe just a kwarg with an array of files to watch, in addition to what scan_docs! finds in the src directory.

[asprionj]

I'm really short on time right now; could you add it and test it on your use case? Just add something along the lines of:

function servedocs(; verbose::Bool=false, filelist=[])
    # ...
    foreach(filepath -> watch_file!(docwatcher, filepath), filelist)
    # ...
end

[tlienart]

Hey all, this is now implemented and merged into master.

I've basically adapted Fredrik's prototype, fixed a few things so that you can have a mix of document pages with and without literate and made sure that you could have your literate script both in or out of docs/src depending on how you want to organise your files.

@briochemc would you mind reading the corresponding (short) doc page & maybe trying the example at the end and give feedback? thanks!

Assuming it kind of fits the bill I'll tag a new minor version soon.

[briochemc]

Wow! 🎉 That looks great! I cannot test it right now but I will try soon 😃

One additional thing that would be nice is to have a flag to run (or not) the code bits in the documentation. (My examples take 10–20 minutes to run, so I avoid running them when I edit the text! 😄 Right now my solution was to manually comment/uncomment two Literate.markdown commands with one having the documenter = false keyword argument.)

[tlienart]

Thanks, feedback would be great :); I'll close this for now but feel free to re-open in the future with comments.

Re eval of code blocks, I think that's more an issue for Literate than for LiveServer :-). Amusingly I also happen to be working on eval for code blocks in JuDoc.

[briochemc]

Hey so after #88 I tried to reorganize the structure according to the LiveServer + Literate docs, but I could not get it to work... For some reason the changes and saves of the source files in the literate folder were not triggering updates.

Then I tried to actually follow the complete example in the docs, but I could not get it to work either... I think what would be great is a little repository of the complete example, if you have it somewhere?

[tlienart]

Thanks for reporting and apologies for the trouble, could you please use the following script and report whether it works? (it seems fine here afaict) I can then add that function to LiveServer to make it easier for users to get started with this.

using Pkg

const INDEX_MD = raw"""
    # Test
    A link to the [other page](/man/pg1.md)
    """
const PG1_JL  = raw"""
    # # Test literate
    # We can include some code like so:
    f(x) = x^5
    f(5)
    """
const MAKE_JL = raw"""
    using Documenter, Literate
    src = joinpath(@__DIR__, "src")
    lit = joinpath(@__DIR__, "literate")
    for (root, _, files) ∈ walkdir(lit), file ∈ files
        splitext(file)[2] == ".jl" || continue
        ipath = joinpath(root, file)
        opath = splitdir(replace(ipath, lit=>src))[1]
        Literate.markdown(ipath, opath)
    end
    makedocs(
        sitename = "testlit",
        pages = ["Home" => "index.md",
                 "Other page" => "man/pg1.md"]
        )
    """

function generate_servedocs_example(dirname="servedocs_example")
    isdir(dirname) && rm(dirname, recursive=true)
    mkdir(dirname)
    # folder structure
    src  = joinpath(dirname, "src")
    mkdir(src)
    write(joinpath(src, "$dirname.jl"), "module $dirname\n foo()=1\n end")
    docs = joinpath(dirname, "docs")
    mkdir(docs)
    src = joinpath(docs, "src")
    lit = joinpath(docs, "literate")
    man = joinpath(lit, "man")
    mkdir(src)
    mkdir(lit)
    mkdir(man)
    write(joinpath(src, "index.md"), INDEX_MD)
    write(joinpath(man, "pg1.jl"), PG1_JL)
    write(joinpath(docs, "make.jl"), MAKE_JL)
end

generate_servedocs_example()

then in your REPL cd to the dir for instance servedocs_example and

using LiveServer
servedocs(literate=joinpath("docs", "literate"))

assuming that works and unless you have additional feedback, I'll merge the corresponding PR #89

[briochemc]

This is great! Yes it worked!

The only thing that did not is a detail, the link from the index page to pg1. I replaced the line (live!) with

A link to the [other page](/man/pg1/)

and it worked after. This is great, although I still need to figure out what's wrong on my side :) I'll close this!

Oh and, BTW, there is absolutely no need for apologizing! You are doing all the hard work (Thank you!), I'm just commenting on it :)