"Multi-level" includes not working

[jmezach]

I have some existing documentation that uses PlantUML extensively which I'm trying to convert to using this extension as I'm trying to get these docs into Backstage. The existing docs do a whole bunch of includes which is not working well with plantuml-markdown. But running the PlantUML binary and passing the filepath of my .puml file(s) works just fine.

I've had a look around the code and I think this stems from the way that source file inclusion works currently. For example, let's say I add the following to my Markdown document:

```plantuml source="some-file.puml"

What happens now is that plantuml-markdown will try to look up this file (based on the base_dir configuration section), read its contents and then append the rest of the code-fence block to it. Then it passes that string to the plantuml binary on the standard input pipe in order to render the resulting diagram. Now let's assume that some-file.puml has the following:

!include some-other-file.puml

This no longer works now, because PlantUML doesn't have the context of where the original file came from, so it doesn't know where some-other-file.puml is, unless it happens to be in the current working directory (which isn't being said for the plantuml process as far as I can tell).

It would be nice if this would just work and I think there are two ways of fixing this. First of all we could just pass the filepath to the plantuml binary straight from the source attribute. That means we'd loose the ability to append whatever is in the fence block, although I'm not sure how useful that is in practice. Alternatively I guess we could rewrite this as a temporary .puml file ourselves with an include and whatever else is in the fence block and then point the plantuml binary to that file. Ideally that file would be next to the Markdown file from which it originated so that embedded includes would work. This would have the added bonus of being able to include a specific diagram from the .puml file by name (as there could be multiple).

I'd be more than happy to submit a PR for this if necessary.

[mikitex70]

Hi @jmezach, thank you for reporting this issue.
Passing the file path directly to plantuml can lead to inconsistencies if you are using a remote plantuml server (which cannot download files).
I think recursive inclusion is more feasible, I'll try over the weekend.

[jmezach]

@mikitex70 Thanks, that would be great. Perhaps it would even suffice to set the working directory for the plantuml process to the directory containing the Markdown file, but I'm not sure.

[jmezach]

@mikitex70 I just tried this by running cat /fully/qualified/path/to/some-file.puml | plantuml -p from the root of my repository and that fails, but if I run cat some-file.puml | plantuml -p from the same directory as where the .puml file is located it works just fine. So if we could set the working directory before running plantuml I think my issue would be solved.

And after doing some more experimentation I found out that echo "!include some-file.puml!full" | plantuml -p as long as I'm running this within the same folder as the .puml file. So I guess that if we take the source from the Markdown, prepend it with !include and pipe that to plantuml should work just fine and would also allow including specific diagrams from a .puml file if there is more than one. Combined with #88 I think I can make all of my existing docs just work with this setup.

Note that the above might introduce some security risks though if we just take whatever is in the source attribute and prepend some text to it and pipe that straight to PlantUML. Perhaps we might want to do some checks on the value of source, ie. check if it refers to an existing file (minus the !<diagram-id> postfix of course).

[mikitex70]

I've managed to use nested includes with a local plantuml using a custom plantuml start script like this (borrowed from Gentoo or Ubuntu, I don't remember):

#!/bin/sh
# PlantUML Launcher
#
# This script is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This script distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public
# License for more details.
# You should have received a copy of the GNU General Public
# License along with this library; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301,
# USA.
#
# Copyright (C) 2010 Ilya Paramonov <ivparamonov@gmail.com>
if [ -n "${JAVA_HOME}" ] && [ -x "${JAVA_HOME}/bin/java" ] ; then
    JAVA="${JAVA_HOME}/bin/java"
elif [ -x /usr/bin/java ] ; then
    JAVA=/usr/bin/java
else
    echo Cannot find JVM
    exit 1
fi

HEADLESS=
[ -z "$DISPLAY" ] && HEADLESS=-Djava.awt.headless=true

$JAVA -Djava.net.useSystemProxies=true $HEADLESS $JAVA_OPTIONS -jar $HOME/bin/plantuml-*.jar "$@"

Note the $JAVA_OPTIONS variable reference in the last line, my customization.
Now simply set the variable like this:

export JAVA_OPTIONS="-Dplantuml.include.path=/path/to/include/directory"

and if the script is found first in the path (save it in $HOME/.local/bin for instance), it will pass the include path to plantuml.
This method can be used also for setting the path of a configuration file (like requested in #88) meanwhile I make the changes in the plugin, changing the last line of the script to:

$JAVA -Djava.net.useSystemProxies=true $HEADLESS $JAVA_OPTIONS -jar $HOME/bin/plantuml-*.jar $PLANTUML_OPTIONS "$@"

and setting the config option:

export PLANTUML_OPTIONS="-config /path/to/config/file.cfg"

If you are using a server, customization may vary. I used the Docker image plantuml/plantuml-server:jetty and in my docker-compose.yml I have declared the environment variable "JAVA_OPTIONS=-Dplantuml.include.path=/mnt/includes" (but it is commented, I don't remember if it works or not).

[mikitex70]

Hi @jmezach, the multi-level / recursive inclusion works with remote server.
With local plantuml if works only if the base_dir is set to . (default value).
To use a different directory you must use the 3.9.0 version of the plugin AND configure the plugin with something like:

base_dir: path/to/include/folder
palntuml_cmd: java -Dplantuml.include.path=path/to/include/folder -jar path/to/plantuml.jar

[jmezach]

I just tried the new version, but unfortunately it seems that it is still not working as I expected it would work. But I think this ultimately stems from the fact that plantuml isn't being run from the same directory as where the Markdown file is located. I've had a look through the code, but it seems that that information isn't even available to this plugin as we're simply being handed a bunch of Markdown, but we have no idea from which source file that has been read. In fact, even Python Markdown doesn't know where the Markdown comes from, so it looks like there isn't much we can do about that.

[mikitex70]

What is your configuration?
The plantuml command is executed by the plantuml_markdown plugin, which is executed by the markdown_py command, or a tool that uses it, such as mkdocs.
So the current directory (the directory were plantuml searches for files) is the directory where the command (markdown_py or mkdocs) was executed.
If the markdown sources (and includes) are in the same directory all works fine.
If the directory for the includes is in another location, you need to configure the plugin by setting the configuration property base_dir with that path (better if it is an absolute path), AND set the plantuml_cmd option with something like java -Dplantuml.include.path=/path/to/includes/directory -jar plantuml.jar.
With these configurations it should work.