onecodex/needletail

Doesn't work with named pipes

Closed this issue · 1 comments

bovee commented

Building anything that uses fastx_cli and passing in a named pipe (e.g. <(cat test.fa)) will cause an Invalid("Bad starting byte") error. This happens because we read the first two bytes to determine if a gzip file has been passed in and then seek back to the beginning before doing our FASTA/Q check. Named pipe files (e.g. /dev/fd/63) don't throw any error when a seek attempt is made, but also don't do the seek so the FASTA/Q test is done on the third byte of the file.

It should be possible to fix this by only checking the first byte for gzip compatibility and then passing that byte into fastx_reader (we used to do this, but changed when we did the rewrite to eliminate seeking for reading from stdin).

bovee commented

As a note, named pipes still won't work with gziped files out of the box. You'll have to <(gunzip ...) inside your pipe if you want to do this (because we can't rewind to allow the GZDecoder to work after we read the magic bytes).