hledger-flow should exit with a non-zero error code if a journal construct invocation results in ExitFailure
lestephane opened this issue · 6 comments
Background
I have PDF statements from my bank which i convert using a construct
script (combination of pdftotext and awk script).
During the hledger-flow import
invocation, I see some error, and indeed the corresponding journal is never created.
Collecting input files...
Found 23 input files in 0.134708928s. Proceeding with import...
...
hledger: -:161:1:
|
161 | 2018-02-30 "Abschluss lt. Anlage 1 PN:905"
| ^^^^^^^^^^
well-formed but invalid date: 2018-2-30
2019-04-15 17:04:58.505274683 EEST hledger-flow End: executing 'girokonto/construct' on 'girokonto/1-in/2018/Kontoauszug_vom_28.02.2018_05.03.2018.pdf' ExitFailure 1 (0.082273261s)
...
Imported 23 journals in 2.203744436s
the problem is twofold:
- there is now a
YYYY-include.journal
file that contains the!include
of a file that does not exist on the filesystem in3-journal
- the
import
command did not return a non-zero exit code (despite the construct ExitFailure 1). So, one will only see a problem later (potentially days later), when loading the files with hledger. By then the user won't know what made his file disappear.
$ hledger-ui -f all-years.journal
...
in file included from /Finance/import/personal/bbbank/2018-include.journal,
/Finance/import/personal/bbbank/girokonto/2018-include.journal:4:97:
|
4 | !include 3-journal/2018/Kontoauszug_vom_28.02.2018_05.03.2018.journal
| ^
No existing files match pattern: 3-journal/2018/Kontoauszug_vom_28.02.2018_05.03.2018.journal
Another example of ExitFailure that did not cause hledger-flow to exit with a non-zero code (a parsing error in a .rules file)
hledger: user error (/Finance/import/personal/n26/checking/n26-checking.rules:23:1:
|
23 | else
| ^
unexpected 'e'
expecting blank or comment line, conditional block, directive, end of input, field assignment, or field name list
)
...
Imported 24 journals in 0.9541295s
$ echo $?
0
Nice catch! I'll look at this as soon as I can make some time.
I've made a change that I think should fix the issue, but I have only tested it briefly.
There should be a pre-release uploaded soon, if the build succeeds:
https://github.com/apauley/hledger-flow/releases
Otherwise you can build this using branch exit-on-err
This should be fix now, please let me know if you see any other issues
- I get a non zero exit code when a .rules file has a syntax error (stray 'else', invalid date)
- I get a non zero exit code when a
construct
script exits with a non-zero exit code.
As an optional usability improvement bonus, I would suggest adding a final one-line log statement like
command `<insert command line that failed here>` exited with code `<insert code here>`, aborting...
Just before hledger-flow exits.
@lestephane The error output in the latest release now looks something like this:
=== Begin Error: importing 'import/gawie/bogart/cheque/2-preprocessed/2016/123456789_2016-05-28.csv' using rules file 'import/gawie/bogart/cheque/bogart-cheque.rules' ===
External command:
/usr/bin/hledger print --rules-file import/gawie/bogart/cheque/bogart-cheque.rules --file import/gawie/bogart/cheque/2-preprocessed/2016/123456789_2016-05-28.csv --output-file import/gawie/bogart/cheque/3-journal/2016/123456789_2016-05-28.journal
Exit code 1
Error output:
hledger: error: could not parse "'29 Apr'" as a date using date format "YYYY/M/D", "YYYY-M-D" or "YYYY.M.D"
the CSV record is: "5", "1", "'29 Apr'", "#Monthly Bank Fee", "", "", "-599.06", "37394.41", ""
the date rule is: %3
the date-format is: unspecified
you may need to change your date rule, add a date-format rule, or change your skip rule
for m/d/y or d/m/y dates, use date-format %-m/%-d/%Y or date-format %-d/%-m/%Y
=== End Error: importing 'import/gawie/bogart/cheque/2-preprocessed/2016/123456789_2016-05-28.csv' using rules file 'import/gawie/bogart/cheque/bogart-cheque.rules' ===
Let me know if you can think of more usability improvements.