na4zagin3/satyrographos

Add testsuite modules

Opened this issue · 1 comments

Here are some ideas to support testing frameworks like https://github.com/zeptometer/satysfi-test

  • Add a new module type TestSuite whose name suffix is *-testsuite.
    • Any modules should not depend on *-testcase modules. This prevents circular dependency of test cases of a fundamental library and a popular testing framework.

We also need a way to tell test result to Satyrographos.

  • Prepare autogen library %result with a function that tells a result to Satyrographos
    • tell-result (tag : string) (value : string) : unit
  • Add a new build command (assert-eq-result <job-name> <tag> <expected-value>)
    • This compares result stored

Internally, tell-result stores the value in crossref entry satyrographos-result-<crossref-name> in file <job-name>.satysfi-aux and assert-eq-result gets it and compare the value with <expected-value>.

This will mitigate this situation:

  • backward-incompatible enumitem is released:
  • some libraries use enumitem to verify their packages; thus they should be removed from the snapshots.

na4zagin3/satyrographos-repo#412