YAML exports with strings in certain formats are interpreted as datetimes
lytedev opened this issue · 9 comments
What version of CUE are you using (cue version)?
$ cue version cue version 0.4.0 linux/amd64
Does this issue reproduce with the latest release?
It looks like I'm using the latest release.
What did you do?
echo 'service: port_mappings: ["2222:22"]' | cue eval - --out yaml | yq
What did you expect to see?
echo 'service: port_mappings: ["2222:22"]' | cue eval - --out yaml | yq
{
"service": {
"port_mappings": [
"2222:22"
]
}
}
What did you see instead?
{
"service": {
"port_mappings": [
133342
]
}
}
Reasoning
YAML supports some date/time formats, and the unquoted 2222:22 is interpreted as a base60 float. So any string in this format with a port on the right less than 60 will be interpreted this way.
Cue should either wrap all strings in quotes or detect this format and wrap it in quotes to protect you.
Possibly related: go-yaml/yaml#34
My workaround for this, for anybody who cares, was to output JSON (cue ... --out json) which any YAML deserializer should handle just fine.
This seems more like a problem with yq and Yaml 1.1 vs 1.2+ (looking through the go-yaml issue you linked
$ echo 'service: port_mappings: ["2222:22"]' | cue eval - --out yaml
service:
port_mappings:
- 2222:22
This looks to be valid YAML 1.2+
Also, which yq are you using
This one works fine https://github.com/mikefarah/yq/releases/tag/v4.9.6
$ echo 'service: port_mappings: ["2222:22"]' | cue eval - --out yaml | yq e -
service:
port_mappings:
- 2222:22
This also happens with docker-compose, to be clear. I just used yq as a quick way to demonstrate the problem. Also, if you refer to this comment on the linked issue, you can see that these YAML deserializers/parsers are following the spec for parsing what my Cue intended to be a string as a base-60 float: go-yaml/yaml#34 (comment)
It's been more than 10 years since YAML the 1.2 spec was written. Should the parsers be updated at this point? I'm not sure what CUE has said about how far back it supports various specs... not sure if there is a link anywhere. I only know for Go it is the same versions that the Go team supports.
Other libraries are specifically saying no to supporting the base60 nonsense
Note that CUE is using the yaml.v3 (https://github.com/cuelang/cue/blob/master/go.mod#L24) but may also have some custom yaml parsing around?
Are the python yq and docker-compose using the same python library which is supporting this? Is this primarily a python related thing? Looking at some other major languages, the top yaml libraries target v1.2 as well, some specifically calling out dropping the base60.
go-yaml has indeed never supported the float 60 representation as it's a bad idea (spec 1.2 confirms that), but it does marshal such values as quoted strings to prevent them from being interpreted incorrectly by old parsers. So it doesn't look like that specific output was generated by go-yaml v3.
Here is how it would handle it:
@niemeyer is the general (simple?) idea to quote any string which has a colon when emitting?
No, we check the actual syntax of base 60 floats from the 1.1 spec. This is also far from being the only edge case in that regard. See bools on 1.1 vs. 1.2, for instance.
This issue has been migrated to cue-lang/cue#1076.
For more details about CUE's migration to a new home, please see cue-lang/cue#1078.