Anything like literalinclude in Sphinx?

[pauleveritt]

I'd like to include syntax-highlighted code in my Markdown, but from a separate file. This plugin doesn't support that. But is there another approach I should use?

[singingwolfboy]

It's not too difficult to implement the functionality: open the file, read the contents, and pass that text to the highlight_code() method that already exists in the codebase. However, there are a lot of edge-cases to handle:

• What syntax are you going to use in your Markdown file? How many different options will it need to support?
• Do you expect the plugin to auto-detect the language based on the file extension, or should that be manually specified?
• Do you support only absolute paths to files, or relative paths as well? If relative paths, then what should the path be relative to?
• What happens if you try to include a file that doesn't exist on the filesystem, or can't be read due to lack of permissions?
• Do you want to support nested includes? If so, that gets even more complicated.

I think we need to determine the answers to some of these questions before implementing this feature. Honestly, I think the trickiest part will be determining what Markdown syntax to use -- Markdown isn't extensible the way that reStructuredText is, which makes Markdown simpler but less powerful. It was never really intended to do this sort of thing. (Which is not to say it can't, only that it may not be a good idea.)

[pauleveritt]

Thanks @singingwolfboy For the near term, I'll just fork the plugin and keep it in my project to get this project. As I learn Lektor though, I'd like to participate in doing it the right way...once I learn the right way. :)

For your bullets:

• I thought I'd try to keep the existing way of specifying the language
• Ditto
• I would probably do whatever Sphinx does, as I've been pleased with that
• That's a tough one, as I don't know Lektor's way of reporting build problems
• I've never done nesting in Sphinx

Here's what I was really curious about was, is there a Better Way? For example, chaining some filters where one filter reads the string from disk, then passes that string to the directive, which can't tell the difference between inline vs. included.

[singingwolfboy]

@pauleveritt: wait, I'm confused. What do you mean by "keep the existing way of specifying the language"? Here's an example of doing syntax highlighting inline:

```python
print("Hello, World!")
```

Can you show me an example of what you're thinking for doing syntax highlighting of an included file?

Also, you said you would "do whatever Sphinx does", when it comes to relative paths. In the Sphinx documentation, it says that literalinclude takes a file path as an argument that is "relative to the current file's path". Lektor's Markdown-rendering system is mistune, which doesn't have a concept of "this is the path of the file I'm currently rendering" -- it deals strictly with text. I'm not sure how you could figure out where to find a relative path while parsing.

[pauleveritt]

Here's an example. It uses html+jinja as the specifier for the string.

You're right about the relative path. Two weeks ago when I was looking, I was hoping that the plugin context had access to the current node's full path.

[singingwolfboy]

@pauleveritt: that looks like an example of the syntax for inline syntax highlighting. I'm asking what syntax you would use for specifying where to find the file to include.

[pauleveritt]

I think I would do something like this.

Again, this is only for the case of my own purposes. I realize others would want something that felt more natural, and I don't yet know what that means for Lektor.

[singingwolfboy]

Just in case your link goes down at some point in the future, I'm going to copy the example that you linked to into this comment:

```html+jinja
file:./some_file.py
```

And yes, if it's for your own purposes, you can do whatever you want. But that format isn't something that would work for a general-purpose framework -- for one thing, it means that you can't do inline syntax highlight for any content that begins with file:.

The basic point I'm trying to get across is that these are difficult questions, and if this is a feature that you want to see in Lektor, then someone needs to figure out the answers to these questions.

[pauleveritt]

You're right that, if this went into this plugin, all of those things would kick in. My nudge was just asking for advice during the learning process. I'm not trying to beg others to add a feature for me. :)

[singingwolfboy]

Gotcha! Well, I'm not sure how helpful I was in giving advice. Do you have specific questions that I might be able to help with?

[pauleveritt]

I first wanted confirmation that nothing like this exists, and I think I can say I received that. [wink]

Then, I wanted any ideas you have for me playing with this. Is there some filter construct to chain directives together?

Next, do you know if the plugin API and context will let me know the filename of the current Markdown file?

[singingwolfboy]

Unfortunately, I don't have anything useful for you. I don't know of any filtering construct for chaining directives -- in fact, "directives" aren't even a concept in Markdown, that's only in reStructuredText. As for the plugin API, you'd do better to check the mistune project -- but as I said, I don't think it's possible. There's a bit more documentation on the Lektor website, but I don't know anything more than that.

[pauleveritt]

I know you've closed this, but now that I have what I sketched out above working, I'd like to re-open the idea of me working towards putting it into this plugin, which would mean answer your points above.

I personally am ok with saying a delimiter like file: to signify that the body is in a filename, but I recognize that it might be surprising. I think there are solutions to that: require people to turn it on with a delimiter they put in the config, or packing it into the already-overloaded first line, where the lang is at.

My challenge now is that edits to code that are included, don't generate a rebuild. I see that the watcher notices that the file changed. I'd need to hook into that event. And then, if I want incremental, then I'll need to maintain some state in the plugin about what references what.

IMO, this is a very useful feature for people that write websites about code. Requiring your code to be pasted into Markdown whenever you change it, for me at least means I might stick with Sphinx+ablog.

If you're ok with me doing a branch here, and with coaxing me as I work through the build system and events, I'm willing to do it to your specs.