hcl.Unmarshal adding spurious arrays
jantman opened this issue · 5 comments
I'm not entirely sure if this is a bug or not. However, I was using https://github.com/kvz/json2hcl in reverse mode to convert a HCL example to JSON, and stumbled on this...
HCL Template
Taken directly from https://vagrantcloud.com/help/vault/vsi/configuration
environment "aws" {
}
vault {
address = "https://vault.service.consul:8200"
mount_path = "auth/aws"
}
serve "file" {
path = "/ramdisk/vault-token"
}
Expected behavior
What should have happened?
{
"environment": {
"aws": {}
},
"serve": {
"file": {
"path": "/ramdisk/vault-token"
}
},
"vault": {
"address": "https://vault.service.consul:8200",
"mount_path": "auth/aws"
}
}
Actual behavior
An invalid configuration, according to VSI:
{
"environment": [
{
"aws": [
{}
]
}
],
"serve": [
{
"file": [
{
"path": "/ramdisk/vault-token"
}
]
}
],
"vault": [
{
"address": "https://vault.service.consul:8200",
"mount_path": "auth/aws"
}
]
}
This JSON has an extra level of arrays inside each top-level object, which is detected as invalid.
Steps to reproduce
I was using https://github.com/kvz/json2hcl for this, which is a CLI wrapper around hcl.
- Clone and build https://github.com/kvz/json2hcl
- Write the hcl template example to a file
cat example.hcl | json2hcl -reverse
References
This is the expected behavior (and the correct hcl to json equivalent) from what I can see.
The reason it is an array is because it's totally valid to specify environment "aws" {}
multiple times. It may not be valid in your use case, but it's valid HCL:
environment "aws" {
a = b
}
environment "aws" {
c = d
}
environment "aws" {
e = f
}
Maybe your application wants to merge those values, or maybe it wants to always use the one last-defined.
which is detected as invalid.
Could you elaborate on this a bit more? What makes it invalid? This seems like valid JSON to me.
I thought it looked right as well. Sorry about the vague "invalid"...
jantman@phoenix:pts/12:~/tmp$ ./vault-secure-intro -version
Vault Secure Introduction Client v0.1.0-alpha ('a730993ff7fdb24f97c7aeb7b4fb8db7aed3547a')
jantman@phoenix:pts/12:~/tmp$ cat example.hcl
environment "aws" {
}
vault {
address = "https://vault.service.consul:8200"
mount_path = "auth/aws"
}
serve "file" {
path = "/ramdisk/vault-token"
}
jantman@phoenix:pts/12:~/tmp$ ./vault-secure-intro -config=example.hcl
[ERR] server/file: error opening temp file at /ramdisk for writing: open /ramdisk/vault-token.tmp.030468548: no such file or directory
this is correct and expected; the config parsed right, but /ramdisk doesn't exist
jantman@phoenix:pts/12:~/tmp$ cat example.hcl | json2hcl -reverse > example.json
jantman@phoenix:pts/12:~/tmp$ cat example.json
{
"environment": [
{
"aws": [
{}
]
}
],
"serve": [
{
"file": [
{
"path": "/ramdisk/vault-token"
}
]
}
],
"vault": [
{
"address": "https://vault.service.consul:8200",
"mount_path": "auth/aws"
}
]
}
jantman@phoenix:pts/12:~/tmp$ ./vault-secure-intro -config=example.json
Error loading configuration: error parsing 'environment': environment type not specified
I suppose it's possible that the bug is actually in VSI, but my assumption was that since VSI uses the hcl library, this should work...
Looking at the above, it seems that VSI is choking on "environment" being an array instead of a hash... and I'm not sure if that's the vault of hcl or vsi. But this feels to me like a bug in HCL, if it's supposed to support JSON but the JSON generated from HCL isn't treated as equivalent by apps...
Hmm yea - this is going to be a tough one, since the mapping between HCL<->JSON isn't 1:1. Specifically, there is more than one way to represent JSON in HCL and visa-versa.
Hey all,
I've had this same problem as I started using viper recently and tried out HCL for the first time--it's great, I love it!
It was super annoying that every map was being surrounded by a slice. I was playing around with the source and figured out a quick fix for it: lukevers@90f1452.
I haven't done any testing except for my config tests for an IRC bot I have which is setup like this and works perfectly with the patch I added:
type Config struct {
Servers map[string]Server
}
type Server struct {
Nick string
User string
Name string
Host string
Port int
TLS bool
Reconnect bool
Channels []string
Debug bool
Plugins map[string]struct {
Enabled bool
Folder string
Pattern string
Events []string
}
}
Just thought I'd point this out.
I understand that JSON <-> HCL isn't one-to-one, but it would be nice if a given HCL resulted in valid JSON for terraform. I wouldn't expect a mapping backwards would result in the same HCL, but right now the inability to get terraform compatible JSON from valid HCL is causing me major headaches.