Proposal: "Include" directive

[LRitzdorf]

Description

This feature is not related to a problem, but does draw inspiration from a suggestion made in the YAGPDB support server:

Vloralys#3408
Suggestion: #1814
One centralized template file for all custom commands

Currently you can use define to create some templates for your custom commands, which make life easier by containing things that get repeated very often in a custom command. Unfortunately templates that are used in multiple commands are hard to maintain, because you need to copy&paste around every little change to each single command and hopefully not miss a single one. Therefor it would be easier to maintain one single file/source/thing which gets included into each single custom command automagically.

Example:

1. Manage YAGPDB
2. Click "Command Settings"
3. Click the tab "Templates" (new)
4. Have one (or multiple) textarea to fill in some define.
5. Reuse them in any Custom Command

Suggestion ID 528415266577842196, made on 05/15/2022

Solution

While it doesn't make much sense to implement this in Yag itself, file substitution (á la #include) would be quite doable for an editor plugin.

We would simply define our own "include" directive, and when the appropriate action is triggered, replace instances of the directive with the specified file. The resulting processed code would be saved to a new location (probably using the original filename, plus a suffix of some sort).

Code of Conduct

• I agree to follow this project's Code of Conduct.

[LRitzdorf]

Initial thoughts:

We could use syntax like {{ include myfile.yag }}, but this could cause confusion by implying that include is a valid CC template (which it definitely is not).

The alternative is to use some other special symbol to signal a processor directive, like #include, though this would have to be chosen carefully so as not to conflict with Discord-flavored Markdown or user/role/channel/emoji syntax.

Also, consideration must be given to the behavior of include-enabled CCs when run directly (that is, without substitution having taken place). Such a CC could:

• Fail to save at all, due to invalid syntax
  • {{ include myfile.yag }}
• Save successfully, but produce unexpected output
  • #include myfile.yag
• Save successfully, and ignore the effects of the include directive
  • {{/* include myfile.yag */}}

[l-zeuch]

Honestly, I'm not quite sure what to think of this suggestion; sure, it might be convenient, however I see a lot of shortcomings here.

You already mentioned storing the preprocessed code to a new file, perhaps with a fitting suffix. However, I can see that adding overhead during development: When I save the file and the preprocessor triggers, it has to scan the code for any such include directives and replace them accordingly, whilst traversing the whole project's tree searching for such header files.
Then, whenever I am ready to copy the code over, I'd have to locate my resulting file containing the code and copy that; even if we add such functionality to our :YagCopy command, that still has to locate the preprocessed code.

Then there's another issue regarding portability: Not all (in fact, only a handful) YAGPDB-users do use (Neo)vim to write their custom commands, and the vast majority uses VSCode instead. If we implement such one preprocessor for Neovim, any contributions to one such repository with headers and preprocessing will not work with VSCode and require manual intervention.

As you also already said, we'd have to find a way of signalling our software to replace this with something else -- finding some sensible approach that is

a) non-intrusive and feels natural to the YAGPDB-syntax
b) yet obviously not from YAGPDB
c) does not potentially clash with any other features like Discord-markdown or the core text/template syntax

is indeed quite hard, and I cannot think of anything that meets these criteria. I've been thinking however about utilising comments with a special syntax; similar to jo3-l/cc-minifier.

Lastly, we would have to find a way of implementing this; I highly doubt Vim script is capable of this task, so we'd have to either use the Python- or Lua-API provided by Neovim, which would bring up the issue of portability yet again.

Though surely convenient and I quite like the idea behind the original suggestion, this simply does not seem feasible to me, especially for a plugin that is rarely used. However, I am as always open to discuss this and find a common denominator, if we and the community wish to implement something along these lines.

[LRitzdorf]

Those are all certainly good points. Especially in light of the difficulty of actually implementing the replacement feature, I agree that it's probably best to hold off for now.

A point of clarification, though: I don't think the replacement operation would incur that much overhead, if we set our requirements properly. For example, if the include statement accepted a relative path from the location of the file containing it, there would be no need to search for files — we'd simply read in the target file, replacing the current line. This would have the same effect as running :cd relative/path V :r header.yag manually.

[l-zeuch]

Regarding setting requirements properly, I was thinking about it for a few hours yesterday.

Personally, I'd like to limit one such header file to 3 000 characters max (this is both arbitrary and also a reference to old YAGPDB custom commands), plus any actual custom command code must be inside a define action. The header file may also contain comments at any level of indentation.

The relative path is for sure worthwhile and should make it easy for us to implement. However, I can imagine that especially larger project repositories have some sort of "global" header, for example for having some sort of standard leading comment.

Our benefit here is that no such thing exists in YAGPDB, so we're basically free to specify the format to our liking, within reason.

Let me know what you think.

[LRitzdorf]

Those are relevant suggestions, though I do have some mixed feelings about them. On the one hand, requiring a define block probably makes sense for the wider YAGPDB user base, but a) that's not really our target audience, and b) it does take away some flexibility, while requiring more work on our end (i.e. we have to validate the included file). I was personally thinking of something closer to the C philosophy — you can include whatever you want, and if it breaks things, that's your problem. Plus, I think Vim users would be comfortable with this.

Regarding character limits, though, I do think we should (at least) add a warning if the final generated file is larger than the CC char limit.

Also, we'd previously discussed a configurable limit on include depth — 2 by default, I think? But override-able by the user, similar to the existing filetype autocommand overrides.

[l-zeuch]

I was personally thinking of something closer to the C philosophy — you can include whatever you want, and if it breaks things, that's your problem. Plus, I think Vim users would be comfortable with this.

Being closer to the C philosophy does indeed make sense, and -- as you said -- it's fair to assume Vim users would be comfortable with this approach.

Regarding character limits, though, I do think we should (at least) add a warning if the final generated file is larger than the CC char limit.

I agree, a warning shall (must) be given when the generated file exceeds the character limit, but I still strongly think we should impose some sort of limits on these header files. Even with 3 000 characters available it is quite easy to exceed the limit of
10 000 characters.

[LRitzdorf]

Maybe another warning, based on header size? Not to be a broken record, but I do think warnings are generally preferable to hard limits here.