`flow test --output=json` does not strictly emit JSON data
Opened this issue · 1 comments
Instructions
Please fill out the template below to the best of your ability and include a label indicating which tool/service you were working with when you encountered the problem.
Problem
Logs outputted by flow test --output=json
are not captured in any way and are emitted to stdio as plaintext. This is problematic because applications which rely on this functionality will fail to parse the output from the Flow CLI.
Steps to Reproduce
Run the command with a test file that emits a log.
Acceptance Criteria
The CLI strictly emits JSON output to stdio. There are multiple solutions for this:
- Suppress logs entirely (probably the easy quick-fix that can be revisited later)
- Extend the output of
flow test
to include logs. It's likely this could introduce some breaking changes to the schema of the CLI output and break external consumers, however, this is a relatively new feature so this is probably acceptable in order to migrate to a more robust output format in the long-run. - Output logs directly to stdio in JSON format and stream multiple messages in realtime (this is similar to
flow emulator --log-format json
, however requires all consumers be equipped to handle multiple JSON objects in the output). It does, however, offer the benefit of instantly serving in-progress information without waiting for a terminal state.
Context
Currently cadence log()
calls break the VSCode Test Runner.
This relies on changes to the Cadence Test Framework (see onflow/cadence-tools#265 onflow/cadence-tools#267)
cadence-tools/test@v0.14.6
was released with a default suppressed logger. Simply upgrading to this version will prevent this error, however more work should be done so that there is no regression in the logs available (either use runner.Logs
or preserve logger behaviour in "text" output format)