clux/lq

allow in-place editing

Closed this issue · 3 comments

clux commented

when passing a file arg as the last argument we could support inline editing with -i ala python-yq's --in-place.

the idea is that we can improve this use-case:

-yq -y '.some.query.to.replace = "better value"' < big_path_to_generated.yaml > tmp && mv tmp > big_path_to_generated.yaml
+yq -iy '.some.query.to.replace = "better value"' big_path_to_generated.yaml
clux commented

this is a bit hairy because we now definitely need to know if we are in the file setup or not.

however, as soon as we add:

    #[arg(default_value = "false")]
    file: Option<PathBuf>,

to the arg parser, it breaks the leftover args (and the file is usually passed at the end so it guarantees we need the overflow -- thing passed)..

so either 1. we have to start enumerating jq args explicitly does (which i don't like because we end up with a much larger api):

    // ----- jq arguments

    /// Output strings without escapes and quotes
    #[arg(short, long, default_value = "false")]
    raw_output: bool,

    /// Compact instead of pretty-printed output
    #[arg(short = 'c', long, default_value = "false")]
    compact_output: bool

    ....

(which also does not have a nice way for us to re-serialize the args for the shellout - and clap does not allow flattening args structs)

...or 2. we do some hacks with mutable state to figure out early if we are in the file or stream case (which already isn't really working anyway #5)

kind of leaning towards some form of 1. because a) it's kind of what python-yq does (to some extent: 1, 2), and b) it allows us to solve #5 properly, and c) it avoids the need for -- altogether and allows full compat.

clux commented

actually, doing args explicit makes more sense because most of the jq args do not work in our context (passed through yaml anyway), in particular:

  • -C or colorising only really works if we implement it somehow (or the output happens to be straight from jq
  • so -M or monochrome, ascii ouput is probably also pointless
  • indent control (and how to indent) must be controlled specifically by us (unless output happens to be straight from jq) and serde_yaml does not support it (upstream issues exist)
  • streaming input makes a lot less sense for yaml since we don't have a newline delimiter type thing that makes sense in json land (people can just use jq for that and maybe pipe to yq --input=json in the future to allow converting in all directions)
  • similarly, input joining (-j), and json-sec (--seq)

but stuff we do support straight if we pass through:

  • [~] sorted keys (-S) works because jq does it, and serde_yaml preserves order (actually, seems something is forcing sorting no matter what - skipping)
  • -c compact
  • -r aka --raw-output and --raw-output0
  • input joining -j
  • -L for modules directory (it's ultimately jq doing that evaluation on modules)

and stuff we could possibly support (but currently don't want to - see follow-up below):

  • -s or slurp basically is the json equivalent of our yaml multidoc parsiing (but we have it done implicitly based on what's in the file - which i guess doesn't work if you have a stream, but should be fine for yaml or toml input)
  • slurping files (i.e. lift files into an object) - but you could also just trivially concatenate the yaml docs, ditto i guess for --rawfile
  • an argjson name value or arg name value equivalent, i guess arg name value for kv injection
  • -R read each line as a string, maybe, not actually sure what this would do
  • -n or --null-input. this is kind of meh. could just have people do <<< "~" or something

so will goof around a bit and try to at least do the passthrough bits and see how that ends up feeling.

EDIT: since of 0.8 we got the middle bits.

clux commented

Notes on slurp after some more thoughts about it:

  -s, --slurp               read all inputs into an array and use it as
                            the single input value;

works like this

$ cat slurp1
"health"
"liveness"
$ cat slurp2
"foo"
"bar"
$ jq . -s slurp1 slurp2
[
  "health",
  "liveness",
  "foo",
  "bar"
]

which can also be a budget way to concatenate strings

jq '.[]' -r -s slurp1 slurp2
health
liveness
foo
bar

i thought about integrating this, but it feels a bit pointless for this. it's basically a different json-like input format. if people need this they could slurp first with jq -s and then pipe to yq --input=json to convert that to other formats.