Piping output of tailspin to another pager can results in panic
jurisevo opened this issue · 4 comments
The following uses case.
$ cat large_log_file.log | tspin | lessDo not scroll till EOF in less, but quit at some point before. This results in crash due to broken pipe.
The application panicked (crashed).
Message: failed printing to stdout: Broken pipe (os error 32)
Location: library/std/src/io/stdio.rs:1019
Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
The application panicked (crashed).
Message: Could not receive EOF signal from oneshot channel: RecvError(())
Location: /home/XXX/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tailspin-2.0.0/src/main.rs:48
Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
This is probably an unconventional use case, but I noticed that feeding large (>10MB) file directly to tspin gets slow, while in the above case it is pretty much instant.
tspin version 2.0.0
It is important to remember that while tailspin is fast, running many regexes is an inherently processor intensive task. It is not possible to "trick" tailspin to be faster by cating the output.
The reason it seems instant, is that tailspin highlights as much as it is able to before piping the result to less. In other words, you are missing logfile entries towards the end of the file.
As an alternative, you could run tspin large_log_file.log --follow to see the progress of highlighting in real time. Cancel and resume the tailing at any time, tailspin will finish the highlighting in the background.
That being said, there are actual ways to speed up the highlighting if performance is important. The first one is disabling highlighting groups that are less important to you.
Moving forward with tailspin, I would like to to parallelize the workload across several threads. This would speed things up, but only when opening files directly. I hope that this improvement can help on systems with large log files, but it would likely not be instant.
See #55, for more on performance considerations for tailspin.
Happy to explain! Will close this issue as a duplicate of #55 for now, but feel free to reopen if you have any more questions!
The same error.
$ journalctl -k | tspin | less
The application panicked (crashed).
Message: failed printing to stdout: Broken pipe (os error 32)
Location: library/std/src/io/stdio.rs:1118
Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
The application panicked (crashed).
Message: Could not receive EOF signal from oneshot channel: RecvError(())
Location: src/main.rs:49
Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.I want to search something in the log, but there is no option to do that.
Piped to less, but no hightlight, and panic when quit.