jqlang/jq

What should '. |= try . catch .' do in jq 1.6, and why is it different from jq 1.5?

Closed this issue · 9 comments

sshine commented

Describe the bug
Unless |= in combination with try-catch have changed semantics, this is probably a bug?

$ jq-1.5 '. |= try . catch .' <<< 1
1

$ jq-1.6 '. |= try . catch .' <<< 1
{
  "__jq": 0
}

Update: I understand from #658 that {"__jq": 0} is some internal control mechanism.

To Reproduce
See above or visit https://jqplay.org/s/M_RpdNHvHF for jq 1.6.

Expected behavior
Unless I'm mistaken, this should return 1 in both versions.

Note that neither . |= . nor try . catch . alone produce this effect.

Environment (please complete the following information):

Additional context
Discovered here:

https://stackoverflow.com/questions/58879588/jq-try-catch-goes-directly-to-catch-even-when-no-error

wtlangford commented

This... does seem incorrect. I'll see what I can figure out.

wtlangford commented

The behavior change would have been introduced here: bd7b48c

It's a weird interaction of nesting labels and a try/catch that trips up a bug in the way we generate code for the catch handler: https://github.com/stedolan/jq/blob/master/src/compile.c#L1041

The issue is that when we look for the internal errors jq uses to handle certain types of control flow jumps (in particular around labels), we intend to check for the type of the error to be an object, but instead we compare the error to "object". You can see this by running try error("object") catch ., which attempts to index into that string to test if we've got an internal error. That results in the internal error getting returned as the value to you.

I'll push a fix later tonight.

wtlangford commented

I pushed e74eab8, which fixes this. Thanks for the report!

pkoppstein commented

@wtlangford - Great work as always. Thanks.

I thought it was time to recreate an instance of jq "from scratch" and sure enough ran into some issues. After the first failure, I installed pipenv (brew install pipenv), and reran make, which failed as follows:

✔ Successfully created virtual environment! 
Virtualenv location: /Users/peter/.local/share/virtualenvs/docs-utiv-7IS
Traceback (most recent call last):
  File "build_manpage.py", line 4, in <module>
    from lxml import etree
ModuleNotFoundError: No module named 'lxml'
make[2]: *** [jq.1] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

I suppose I'm supposed to install somehow but pip3 says it's already installed:

pip3 install lxml
Requirement already satisfied: lxml in /usr/local/lib/python3.7/site-packages (4.4.1)

Maybe I need to dig deeper into the (revised?) installation instructions, but I'm disappointed that what used to be a painless process has seemingly become somewhat fraught. Am I missing something?

wtlangford commented

@pkoppstein- yeah, that's an issue I'm aware of. It also prevents people from doing brew install jq --HEAD.
Essentially, I didn't set up that whole ruby replacement the way I meant to. Can you try the branch in #1995 and see how it works for you? It changes a few things about how the build process works and only requires a python environment if you make changes to the manpage yml.

wtlangford commented
pkoppstein commented

@wtlangford: Whoops - I just let SourceTree clone the repo without checking which branch it was checking out.

So, after checking out the relevant version, things ran successfully for a while,
but then the dreaded 'lxml' dragon raised its fearsome head:

....
checking for Python dependencies... Warning: the environment variable LANG is not set!
We recommend setting this in ~/.profile (or equivalent) for proper expected behavior.
Creating a virtualenv for this project…
...
✔ Successfully created virtual environment!
...
  GEN      jq.1
Traceback (most recent call last):
  File "build_manpage.py", line 4, in <module>
    from lxml import etree
ModuleNotFoundError: No module named 'lxml'
make[2]: *** [jq.1] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
wtlangford commented
pkoppstein commented

@wtlangford - git show begins:

commit dd87c3b9d7acf4efaced0a8eccb534df82c6cf27 (HEAD -> fix-brew-head-build, origin/fix-brew-head-build)
Author: William Langford <wlangford@digitalocean.com>
Date:   Tue Oct 22 12:37:19 2019 -0400

    Rework pipenv requirement to be more relaxed
    
    Keep a cached copy of the man tests that we can use when no manpage
    changes are made. This allows automated systems that might not have
    easy access to a pipenv to build and run tests.