fastpack/fastpack

Refactor integration tests from Jest to pure shell

zindel opened this issue · 0 comments

Jest is a great testing tool, but seems to be an overkill for our case, it takes significant time to run the test suite as it grows and also requires some extra helpers to maintain.

Proposal:

  1. Have test/run.sh command with the interface ./run.sh [test1 test2 ...]
  2. Arguments are optional and may be either the directory containing all the tests or the shell file inside of it
  3. If no arguments are provided ./run.sh collects all the tests as following: ls test/*/*.sh
  4. As you can see from the previous point - the test is a Shell file in the specific directory. The expectation is that it contains the fpack invocation of some sort.
  5. Each test will be provided with env variable $FPACK containing something like ../../_build/bin/fpack --output=/path/to/output. That said tests should not specify output directory on their own.
  6. The name of the output directory is calculated from the location and filename of the test itself. For example, for the test in pack-css/dev.sh it will be ./dev/ given that CWD is pack-css/.
  7. Now, if the output directory does not exist - it is created and contains all the files which were emitted during successful packing + stdout.txt containing the stdout output of the fpack.
  8. If the output directory exists fpack is executed to a temporary location and then diff <output direction> <temporary location> is executed. If they are different - test is considered to be failing.
  9. It is fine if fpack exits with non-zero exit code. In this case output directory contains only one file stderr.txt
  10. If you need to update the test snapshots (output directories), run './update.sh' which has the same signature as run.sh and effectively is just a symlink.

Thoughts?