Formatting Docker Compose files removes the YAML document end

Question

Formatting Docker Compose files removes the YAML document end

ferrarimarco opened this issue 2 years ago · 17 comments

Hi all!

I noticed that formatting a Docker Compose file strips away the YAML end document marker (...). This doesn't happen if you manually set a Docker Compose file as a YAML file, and format it.

Steps to reproduce:

Create a Docker Compose file named compose.yaml (although any other file name recognized by the dockercompose language definition would work).
Ensure that VS Code recognizes the file compose.yaml file as a Docker Compose file (look for the Compose language mode in the status bar (bottom right).
Paste the following content in the file:
```
---
services:
  test-service-1:
...
```
Save the file.
Format the file (if you don't have "Format on Save" enabled).

The resulting, formatted file is:

---
services:
  test-service-1:

While I was expecting:

---
services:
  test-service-1:
...

Is this behavior something I can configure? Or is it a bug?

Thanks!

Answer 1 · 2022-03-07T13:56:29.000Z

I'll take a look but I suspect it's a bug. I'll transfer this to the language service repo.

Answer 2 · 2022-03-09T08:30:13.000Z

Thanks @bwateratmsft !

Answer 3 · 2022-03-09T17:38:46.000Z

@ferrarimarco Do you know if Docker Compose actually supports multiple YAML documents in one file? I've tried it out but it just seems to use only the first document and ignore everything else.

Answer 4 · 2022-03-10T07:55:40.000Z

The compose file spec supports merging multiple compose files. The spec doesn't say anything about multiple YAML documents in the same file. So I'd say that having multiple YAML documents in the same file is not a use case that Docker Compose supports, which is consistent with your test.

I guess this makes this a formatting bug only, rather than one that causes a loss of functionality.

I tried to have a look at where compose-language-server sets YAML formatting options it uses the yaml package to provide this feature, if I got the code correctly.

There's one interesting option to play:directivesEndMarker

From the yaml package docs

Whether the document should always include a directives-end marker --- at its start, even if it includes no directives.

Unfortunately, I found no mentions of the document end marker ... in the yaml package docs or code.

Answer 5 · 2022-03-10T14:57:44.000Z

Yeah, I'd noticed the same about directivesEndMarker. Apparently since the ... isn't strictly necessary it isn't being included at all.

Answer 6 · 2022-03-10T15:24:34.000Z

Yup. Compose file are valid yaml files. Start and end markers can be there even if you have a single yaml document in a yaml file, according to the yaml spec. Given that both markers are optional, the formatter shouldn't mandate either, nor strip any marker if it's there. Maybe this is due more to a limitation of the yaml library that the formatter relies on, rather than an explicit design choice? Also, I was surprised about the name of that option, given that `---` is actually the yaml document start marker, not the end marker.

Answer 7 · 2022-03-10T17:46:21.000Z

The formatter works by parsing the YAML document and then rewriting it using the yaml library. It will only act if there are no YAML syntax errors in the document; if there are syntax errors, it is more likely to harm than help so it's better to do nothing until the user fixes them.

While the --- and ... are valid in a single-document YAML file--why use them? Compose apparently doesn't support multi-document files, thus rendering those directives optional.

Answer 8 · 2022-03-10T18:05:26.000Z

My use case is that I'm also parsing those yamls with other tools that use the document start and end markers to delimit those documents.

Answer 9 · 2022-03-10T18:08:03.000Z

Anyway, I think the formatter shouldn't alter that part of the document, as the "plain" yaml formatter is doing. It shouldn't enforce either choice, given that those are optional parts of the yaml spec. Or, at least, make it configurable.

Answer 10 · 2022-03-10T19:04:00.000Z

That makes sense. Can you open an issue at https://github.com/eemeli/yaml? It makes sense to me that the toString function should emit the ... at the end if it was present in the input, or at least provide an option.

Answer 11 · 2022-03-10T19:26:27.000Z

Done: eemeli/yaml#368

Answer 12 · 2022-03-20T17:15:01.000Z

@bwateratmsft eemeli/yaml#368 was merged.

Answer 13 · 2022-04-04T10:36:06.000Z

@bwateratmsft Hi! It appears that the above PR made through the current eemeli/yaml release, so the necessary API should now be available.

In the version that compose-language-service ships now, the document start marker is not stripped if it's there, nor it's added if it's not there. I'd expect the same for the next version, but I'm not sure if that applies to the doc end marker.

Answer 14 · 2022-04-06T14:48:35.000Z

It looks like yaml@2.0.0 just shipped so we'll switch over to that.

Answer 15 · 2022-04-08T18:34:57.000Z

Thanks! I'll have a look as soon as a new release of the extension is available.

Answer 16 · 2022-04-08T18:37:32.000Z

This is the related PR btw: microsoft/vscode-docker#3485

Answer 17 · 2022-04-08T20:18:05.000Z

Glad to help! If you want to test it sooner, and you have Node.js installed, you can try out just the compose language service with the following steps:

Clone this repo
npm install from the repo root
cd src/test/clientExtension, and npm install in there too
Choose the launch config "Client + Server" in VSCode, and hit F5. This will launch an extension host, which activates the language server alone.

I use this approach for testing, since it's a lot quicker than packaging, installing the package in the Docker extension, and building and running that.