asciidoctor/asciidoctor-diagram

When I set the `outdir` attribute, the picture in the gernerated document is not correct.

diguage opened this issue ยท 37 comments

I have a Asciidoc as follows:

:outdir: target

== Test Diagram

plantuml::puml/class.puml[format=svg, alt="Class diagram", target="{outdir}"]

The class.puml is as follows:

class BlockProcessor
class DiagramBlock
class DitaaBlock
class PlantUmlBlock

BlockProcessor <|-- DiagramBlock
DiagramBlock <|-- DitaaBlock
DiagramBlock <|-- PlantUmlBlock

run asciidoctor -r asciidoctor-diagram sample.adoc, then it gernerates the UML diagram in target/, named target.svg.

But the path in html file is <img src="target.svg" alt="Class diagram" width="254" height="281">.


When I remove the target attribute, as follows:

:outdir: target

== Test Diagram

plantuml::puml/class.puml[format=svg, alt="Class diagram"]

The gernerated UML diagram in target/, named class.svg.

Then, path in html file is <img src="class.svg" alt="Class diagram" width="254" height="281">.

If you want to control where images are written, you should use the imagesoutdir attribute. If you set the target attribute, then you are changing the name of file that is written. If you don't set the target, the output file name is generated from the input file name.

See https://github.com/asciidoctor/asciidoctor-diagram#image-output-location

I set imagesoutdir attribute, the images was written to the folder. But the gernerated HTML document cound not refer the images with the correct path.The path is <img src="class.svg" alt="Class diagram">, does not include the imagesoutdir.

I forgot to clarify. The path used in the HTML is controlled by imagesdir. The imagesoutdir attribute tells Asciidoctor diagram where to write the file. The imagesdir attribute tells the processor what to put in the HTML.

Admittedly, this is not an ideal system yet. What's important is that we study the uses and think about how we might want to change it to make it more ideal. The design should evolve as we learn more about the use cases.

I found out that asciidoctor-mathematical works OK in this case, only setting imagesoutdir is enough.
When setting imagesdir also then asciidoctor-diagram starts working, but asciidoctor-mathematical applies both imagesoutdir and imagesdir to the final path and thus does not work correctly in this case.
It looks like asciidoctor-mathematical is wrong here, I submitted an issue there: asciidoctor/asciidoctor-mathematical#8

I think right solution would be that if filename is not specified explicitly, then asciidoctor-diagram generates the filename and saves it to imagesout dir, so in that case in html the path should be imagesoutdir/... But if filename is specified explicitly as target attribute then the path should be imagesdir/...

Today I came across the use case that I may not want to generate the files into the same directory that my other images are in, ex:

imagesoutdir == $(img_loc)/gen
imagesdir == $(img_loc)

However, I have no way to tell asciidoctor-diagram where to look for the generated files, and it will look in imagesdir with no way to change it for generated blocks.

This comes into play where we are using git lfs to keep track of our static files, but still want to ignore autogenerated ones. I think it would be great if it was possible to have the resultant image:: tag able to be pointed in more than one location (imagesdir). At this point, I have to have imagesdir be a duplicate of imagesoutdir and then specify my path from there for all of my non-autogenerated images (image::../static/{..}).

A basic solution would be the ability to use :imagesoutdir: for the path that the images:: tag for autogenerated images automatically searches, as that is where they are placed anyway.

mojca commented

Another user raising hand due to the same issue.

I would like to keep a clear separation between generated images and the ones provided by the user, else it becomes all too easy to overwrite the originals.

bric3 commented

Actually I found out that it even applies when SVGs are inlined.

[plantuml, "yet-another-activity-diagram-example",format="svg",opts="inline"]
----

Setting, something like --attribute imagesoutdir=./.asciidoctor and the current version 2.0.2 will emit this warning :

WARNING: <stdin>: SVG does not exist or cannot be read: /Users/bric3/private/repo/activity-diagram-example.svg

Like mentionned above setting imagesdir solves the issue for generated images, but ultimately broke the other images.

Is there a solution for this? I'm trying to get us to use asciidoc + plantuml + drawio for documentation. This will primarily be viewed on gitlab, but for writing the docs it will mostly be an editor like vscode with the asciidoc extension. I'm struggling to get the writing workflow working because of this issue of not wanting generated images next to static images to make sure version control works correctly. It also seems to keep generating new files which also creates a lot of clutter. Even if I inline the plantuml diagrams, it still creates a generated image for some reason. I'm not sure if it actually uses the generated image when using inlining, so if it doesn't, maybe that can just be turned off?

EDIT: And before you mention kroki, that is not a good solution to have to get every dev to setup a long running webserver on their machine, rather than just install a package.

ahus1 commented

@BrendanBall - as I understand it, the files you see appearing in your project's working directory are created when rendering the preview in the editor.

The AsciiDoc plugin for IntelliJ (when using the JavaFX or JCEF preview) will set the outdir and imagesoutdir to a temporary folder so that temporary images are kept out-of-sight. It post-processes the HTML to show generated images in the preview together with non-generated images. This prevents cluttering your working directory.

When using a different editor, you might try one of the following:

  • raise your issue with the AsciiDoc plugin support plugin you use with a minimal example
  • emulating the behavior of a temporary folder by setting attributes either in the plugin you use, or conditionally in the AsciiDoc source (see below for an example testing for GitHub). But I am afraid it will get messy once you have a mix of dynamic or static images.
ifdef::env-github[]
:imagesdir: foo/
endif::[]

A side node: if you see image files with randomly changing file names: these originate from diagrams without a diagram name. If no name is provided as second parameter (just after the diagram type), the file name is a hash (?) that changes whenever the contents of the diagram change. Adding a diagram might get you half-way as it allows you to add these diagram file names to a .gitignore file.

This is still pretty tricky to solve for all possible situations due to how Asciidoctor core handles images. In short:

  • The diagram extension must generate files on disk. There's no other way at the moment to pass an image reference to Asciidoctor core from a block processor.
  • Since we need to generate files we need to decide where to put them on disk. The current logic tries to service that location based on imagesoutdir or by resolving imagesdir against outdir.

An improvement on this system might be to add an attribute diagram-imagesdir (yet another one, I know it's not getting any easier) that allows you to specify a prefix for generated image. The idea would be that this is a relative directory that gets resolved wrt imagesdir. This would enable you to specify something like this:

:imagesdir: images
:diagram-imagesdir: gen
:imagesoutdir: images/gen

The resulting HTML for a diagram PNG with target target would then contain an <img> tag with src set to images/gen/diag.png.

Would that help? I'm open to other suggestions as well of course, but do keep in mind the restrictions mentioned above. The information passing options between core and extensions are limited.

I think Asciidoctor is lacking the concept of a build. If I wanted to publish the result of Asciidoctor, I would for e.g. want a public directory where the html etc gets put, which also means the images that were included in the doc should be copied to public/images and then the generated images could go there as well. Of course if you specify inline then it probably shouldn't end up there, so we probably need an intermediate build directory. We could probably hack this with a plugin that just copies any included images into imagesoutdir, ie. the same place as asciidoctor-diagram.

I think Asciidoctor is lacking the concept of a build.

You are absolutely correct. That's because Asciidoctor is not a build tool. It's a file processor. This is where the Maven, Gradle, and other such build plugins come into the picture. We're not in the business of writing build tools.

Asciidoctor is the only thing that knows which files are dependencies though.
So do you agree the best way to do this right now is to create a plugin asciidoctor-images to copy the images to imagesoutdir?

The Gradle plugin knows how to copy the dependencies, even the generated images.

ok, I'm using the vscode asciidoctor plugin for live preview, specifying a custom command asciidoctor -r asciidoctor-diagram, so I'm not sure that'll help

No, Gradle will not help you for the IDE. The IDE has to take on this responsibility (as the IntelliJ plugin does). If you are outside of the IDE, then you'd use a build tool like Gradle. Either way, Asciidoctor is not a build tool.

So to clarify my point, you should seek for the vscode plugin to match the behavior of the IntelliJ plugin. (In general, we want the IDEs to align on behaviors such as this, which will be a focus of the AsciiDoc Working Group).

bric3 commented

@pepijnve In reply to your #110 (comment), I also wanted to have the ability to inline diagrams in the rendered document.

Currently I'm forced to declare opts=inline, as diagram-svg-type=inline don't work for me, I'm not sure why #247 (comment).

Anyway I would like to avoid mixing relgular images in imagesdir with generated diagrams that need to be inlined, those should go in a temporary folder of our choice, so this could be useful to be able to indicate a folder outside imagesdir too.

@bric3 inlining images (either SVG or PNG) is handled by the backend, the diagram extension doesn't play a part in this. opts=inline is the way to get embedded images in your HTML.

If you want the images to get generated in some other directory you need to specify the imagesoutdir attribute. If specified, that's where the diagram extension will write images.

bric3 commented

@pepijnve Sorry I wasn't clear, but if the given parameter are passed

--base-dir /Users/bric3/private/bric3.github.io/content/drafts
-a outdir=/Users/bric3/private/bric3.github.io/public/drafts/exploring-java-native-memory
-a imagesoutdir=./asciidoctor

Then the basedir will receive the generated SVG files, while the outdir/imagesoutdir will receive the *.svg.cache files.
But then if you set imagesdir, the generated files can get mixed with regular images.

So I wanted a way to have a specific imagesdir for generated resoruces only.

Human error is still possible as I have been playing this for a while to get it working reasonably well.


In regard of inlining I understand inlining is done by asciidoctor backend. But asciidoctor added this diagram-svg-type in version 2.0.0 : https://github.com/asciidoctor/asciidoctor-diagram/blob/master/CHANGELOG.adoc#200. So I expected to avoid repeating the opts=inline.

In short this test don't work for me :

it "should respect the diagram-svg-type attribute when format is set to 'svg'" do
doc = <<-eos
= Hello, PlantUML!
:diagram-svg-type: inline
Doc Writer <doc@example.com>
== First Section
[plantuml, format="svg"]
----
User -> (Start)
User --> (Use the application) : Label
:Main Admin: ---> (Use the application) : Another label
----
eos
d = load_asciidoc doc
expect(d).to_not be_nil
b = d.find { |bl| bl.context == :image }
expect(b).to_not be_nil
expect(b.content_model).to eq :empty
target = b.attributes['target']
expect(target).to_not be_nil
expect(target).to match(/\.svg/)
expect(File.exist?(target)).to be true
expect(b.attributes['opts']).to eq('inline')
expect(b.attributes['width']).to_not be_nil
expect(b.attributes['height']).to_not be_nil
end
when passing to the cli -a diagram-svg-type=inline

@bric3 ok, I'll try to reproduce this. That doesn't sound like the expected behaviour.

Regarding svg-type seems like I already forgot the changes I made . Oops. From looking at the code it seems like what you're describing should work indeed. I'll have to run that through the debugger as well.

@bric3 the svg-type issue should be fixed by aa9969c already.

bric3 commented

@pepijnve Ah ok it's in the 2.0.3 which isn't yet published.

That's correct. You can try that out already if you like by pointing to the git master branch in your Gemfile. I'll probably make a new release, but I need some more time to test everything.

@bric3 I can't quite reproduce the output results you're seeing.

If I start from this situation
image

and then run asciidoctor with the options

-r asciidoctor-diagram
--base-dir /Users/pepijn/Projects/asciidoctor-diagram/temp
-a outdir=/Users/pepijn/Projects/asciidoctor-diagram/temp/out
-a imagesoutdir=/Users/pepijn/Projects/asciidoctor-diagram/temp/asciidoctor
-o out/index.html
/Users/pepijn/Projects/asciidoctor-diagram/temp/temp.adoc

I get this
image

The generated SVG file goes into <imagesoutdir>, the cache files go into <outdir>/.asciidoctor/diagram and the output html file gets written to <basedir>/out/index.html. I'm not quite sure why you would be seeing different behaviour; this behaviour matches exactly with the code.

bric3 commented

Hi, I worked on some reproducer yesterday, but home life took over before posting the comment ;)

Here's some reproducer, the commands are inspired by hugo (currently 0.75) integration, which is using the stdin / stdout to process the document (then hugo integrate the asciidoc rendered output in the site HTML layout).

So here's my test set :

$ tree -a
.
โ”œโ”€โ”€ content
โ”‚   โ”œโ”€โ”€ blacksad.jpeg
โ”‚   โ”œโ”€โ”€ images
โ”‚   โ””โ”€โ”€ sample.adoc
โ””โ”€โ”€ public
$ cat content/smaple.adoc
= Hello!

== First Section

[plantuml, format="svg"]
----
A -> B
----

image:blacksad.jpg[width="50%"]

Suppose the content directory is in git, I don't want to put generated images in the content folder, but somewhere else in a temporary folder. The public folder for example is exluced from git. And should receive only the generated SVG.

In order to showcase the reproducer, let's use what Hugo sends as a base (without images attributes)

basic Hugo command
$ rm -rf content/.asciidoctor public/.asciidoctor public/sample.html
$ cat content/sample.adoc | asciidoctor -r asciidoctor-diagram \
 -a library=asciidoctor-ruby \
 --base-dir=/Users/bric3/private/asciidiagram-reproducer/content \
 -a outdir=/Users/bric3/private/asciidiagram-reproducer/public \
 --trace \
 - | tee > public/sample.html | grep -A10 "_first_section" \
 && tree -a
<h2 id="_first_section">First Section</h2>
<div class="sectionbody">
<div class="imageblock">
<div class="content">
<img src="diag-5c51c4688d042deefec4631ff5a14ef4.svg" alt="diag 5c51c4688d042deefec4631ff5a14ef4" width="79" height="112">
</div>
</div>
<div class="paragraph">
<p><span class="image"><img src="blacksad.jpg" alt="blacksad" width="50%"></span></p>
</div>
</div>
.
โ”œโ”€โ”€ content
โ”‚   โ”œโ”€โ”€ blacksad.jpeg
โ”‚   โ””โ”€โ”€ sample.adoc
โ””โ”€โ”€ public
    โ””โ”€โ”€ diag-fc61bd93cf6e30262fe2f8768a9862de.svg
The issue appear when using `imagesdir` only as a relative folder.
$ rm -rf content/.asciidoctor public/.asciidoctor public/sample.html
$ cat content/sample.adoc | asciidoctor -r asciidoctor-diagram \
 -a library=asciidoctor-ruby \
 --base-dir=/Users/bric3/private/asciidiagram-reproducer/content \
 -a outdir=/Users/bric3/private/asciidiagram-reproducer/public \
 -a imagesdir=.asciidoctor  \
 --trace \
 - | tee > public/sample.html | grep -A10 "_first_section" \
 && tree -a
<h2 id="_first_section">First Section</h2>
<div class="sectionbody">
<div class="imageblock">
<div class="content">
<img src=".asciidoctor/diag-5c51c4688d042deefec4631ff5a14ef4.svg" alt="diag 5c51c4688d042deefec4631ff5a14ef4" width="79" height="112">
</div>
</div>
<div class="paragraph">
<p><span class="image"><img src=".asciidoctor/blacksad.jpg" alt="blacksad" width="50%"></span></p>
</div>
</div>
.
โ”œโ”€โ”€ content
โ”‚   โ”œโ”€โ”€ blacksad.jpeg
โ”‚   โ””โ”€โ”€ sample.adoc
โ””โ”€โ”€ public
    โ”œโ”€โ”€ .asciidoctor
    โ”‚   โ”œโ”€โ”€ diag-5c51c4688d042deefec4631ff5a14ef4.svg
    โ”‚   โ””โ”€โ”€ diagram
    โ”‚       โ””โ”€โ”€ diag-5c51c4688d042deefec4631ff5a14ef4.svg.cache
    โ””โ”€โ”€ sample.html
With `imagesoutdir` only as a relative folder, the `.svg.cache` files are in the output folder, and the `.svg` are generated in the _content_ (basedir) folder
$ rm -rf content/.asciidoctor public/.asciidoctor public/sample.html
$ cat content/sample.adoc | asciidoctor -r asciidoctor-diagram \
 -a library=asciidoctor-ruby \
 --base-dir=/Users/bric3/private/asciidiagram-reproducer/content \
 -a outdir=/Users/bric3/private/asciidiagram-reproducer/public \
 -a imagesoutdir=.asciidoctor \
 --trace \
 - | tee > public/sample.html | grep -A10 "_first_section" \
 && tree -a
<h2 id="_first_section">First Section</h2>
<div class="sectionbody">
<div class="imageblock">
<div class="content">
<img src="diag-5c51c4688d042deefec4631ff5a14ef4.svg" alt="diag 5c51c4688d042deefec4631ff5a14ef4" width="79" height="112">
</div>
</div>
<div class="paragraph">
<p><span class="image"><img src="blacksad.jpg" alt="blacksad" width="50%"></span></p>
</div>
</div>
.
โ”œโ”€โ”€ content
โ”‚   โ”œโ”€โ”€ .asciidoctor
โ”‚   โ”‚   โ””โ”€โ”€ diag-5c51c4688d042deefec4631ff5a14ef4.svg
โ”‚   โ”œโ”€โ”€ blacksad.jpeg
โ”‚   โ””โ”€โ”€ sample.adoc
โ””โ”€โ”€ public
    โ”œโ”€โ”€ .asciidoctor
    โ”‚   โ””โ”€โ”€ diagram
    โ”‚       โ””โ”€โ”€ diag-5c51c4688d042deefec4631ff5a14ef4.svg.cache
    โ””โ”€โ”€ sample.html

Notice however that the paths to the image and the diagram, are the same. Which is not the case on the filesystem even in the content folder. Indeed that makes sense since imagesoutdir only tells where diagram are to be generated, which leads to the usage of imagesdir again.

With both `imagesdir` and `imagesoutdir`, it's the same as `imagesoutdir` alone

In the above example we saw that diagrams are generated in the content folder, and in order to refer to them in particular for svg inlining, we need to set the imagesdir to the same value.

$ rm -rf content/.asciidoctor public/.asciidoctor public/sample.html
$ cat content/sample.adoc | asciidoctor -r asciidoctor-diagram -a library=asciidoctor-ruby --base-dir=/Users/bric3/private/asciidiagram-reproducer/content -a outdir=/Users/bric3/private/asciidiagram-reproducer/public -a imagesdir=.asciidoctor -a imagesoutdir=.asciidoctor --trace - | tee > public/sample.html | grep -A10 "_first_section" && tree -a
<h2 id="_first_section">First Section</h2>
<div class="sectionbody">
<div class="imageblock">
<div class="content">
<img src=".asciidoctor/diag-5c51c4688d042deefec4631ff5a14ef4.svg" alt="diag 5c51c4688d042deefec4631ff5a14ef4" width="79" height="112">
</div>
</div>
<div class="paragraph">
<p><span class="image"><img src=".asciidoctor/blacksad.jpg" alt="blacksad" width="50%"></span></p>
</div>
</div>
.
โ”œโ”€โ”€ content
โ”‚   โ”œโ”€โ”€ .asciidoctor
โ”‚   โ”‚   โ””โ”€โ”€ diag-5c51c4688d042deefec4631ff5a14ef4.svg
โ”‚   โ”œโ”€โ”€ blacksad.jpeg
โ”‚   โ””โ”€โ”€ sample.adoc
โ””โ”€โ”€ public
    โ”œโ”€โ”€ .asciidoctor
    โ”‚   โ””โ”€โ”€ diagram
    โ”‚       โ””โ”€โ”€ diag-5c51c4688d042deefec4631ff5a14ef4.svg.cache
    โ””โ”€โ”€ sample.html

While this fixes the issue for diagrams, the imagesdir value also affects the path of the regular local image.

Ideally there should a configurable location for the .svg.cache (ditaa, svgbob, plantuml etc.), and the should be a different imagesdir for generated diagrams.


I'm not sure I can install asciidoctor-diagram using a git ref, I'm using gem install ....

Ideally there should a configurable location for the .svg.cache

That's an easy one to add. I'll add support for a diagram-cachedir attribute to override the current behaviour.

and the should be a different imagesdir for generated diagrams

That's kind of what imagesoutdir is for. Having a distinct attribute for generated images with the semantics of imagesdir is not currently possible I'm afraid. There's no good way for an extension to pass all the necessary bits of information to the backend converters. Or at least not that I know of. @mojavelinux has anything changed in this area in the meantime?

Regarding the relative paths, those are resolved wrt the base-dir value you provide. So specifying imagesoutdir as a relative path and then setting base-dir to some directory means your images are going to end up in <base-dir>/<imagesoutdir>. If that's not what you want you need to specify an absolute path for imagesoutdir.

bric3 commented

If that's not what you want you need to specify an absolute path for imagesoutdir.

That's what I was afraid of, currently this cannot be done easily with Hugo. I cannot set this argument to an absolute path in the config. Otherwise the build would either fail locally or on the build system.

Having a distinct attribute for generated images with the semantics of imagesdir is not currently possible I'm afraid.

OK, that's probably something intersting to add at some point I think. Not to mention that I need imagesdir and imagesoutdir to the same value if I want inlining to work. Currently I'm working around this by using a path from the root for regular images, putting them in a separate location (i.e. not in the same folder as the document). So I can tweak imagesdir for asciidoc to fetch the generated SVG at the righ location:

= Hello!

== First Section

[plantuml, format="svg"]
----
A -> B
----

// absolute path, to avoid `imagesdir` interference
image:/assets/images/blacksad.jpg[width="50%"]

Unfortunately during document edition the regular image is not found with this path by editors, but this an acceptable inconvenient as diagrams are properly rendered.

@bric3 FYI, I just built and published 2.0.3 which should fix the svg-type issue.

bric3 commented

@pepijnve Excellent, that works great. Thank you very much!

@mojavelinux has anything changed in this area in the meantime?

The imagesdir attribute can now be set directly on the image node in the API. That means that you could rewrite the imagesdir for a specific image so it points to where the image was generated.

This change still be turn out to be insufficient, but we can keep iterating. (Also note that, in the future, it will be possible to store the image data itself on the node, but we're not there yet).

Perfect. I'll add support for a diagram-imagesdir attribute then and set imagesdir on the generated nodes to either diagram-imagesdir or imagesdir in that order of precedence. That should cover quite a few cases already I think.

๐Ÿค” that initial plan might be too simplistic. If I can set imagesdir on the nodes then there might not even be a need for user input at all. I'll try to see if I can figure out what the value of a per node imagesdir value should be based on where the image file and output file are being written respectively.

This was added quite some time ago in asciidoctor 1.5.8 (asciidoctor/asciidoctor#2779) so I don't think there's a need to add version checks, backwards compatibility workarounds, etc.

For the adventurous: I've added some code that tries to set the correct imagesdir value per node. You need to set the diagram-autoimagesdir attribute to enable this.