/quickcheck

Automatic testing of MoonBit programs

Apache License 2.0Apache-2.0

MoonBit QuickCheck

QuickCheck is a library for random testing of program properties. The programmer provides a specification / theorem of the program, in the form of properties which functions should satisfy, and QuickCheck then tests that the properties hold in a large number of randomly generated cases.

Demo

Check file lib/qc_demo.mbt for usages. The following example reports an error of @json.stringify in core.

test "json generator" {
  quick_check(
    fn(jv : JValue) {
      match @result.wrap1(f=@json.parse, @json.stringify(jv)) {
        Err(_) => false
        Ok(jv2) => jv == jv2
      }
    },
  ).print()
}

Output:

[test-0]: FAIL
  There exist Object({"\x0c": True}) (State={seed: 1941323925062528825, gamma: 16934044424796929712}) such that condition is false (after 0 shrink(s))
    Distribution: 
      1.0: trivial

Features

MoonBit QuickCheck brings many modern academic ideas into industrial practice. Note that all the ideas used in this project have been listed in references.

Testing Strategy

  • Randomized property test: Run the test for N random elements drawn from the argument type.
    • Similar to Haskell QuickCheck
  • Exhaustive specialized property test: Run the test for each element of a subset of the argument type.
    • Similar to SmallCheck and Feat

Data Generation

  • Functional Enumeration
  • Falsify (with free shrinkers)
  • Random Generation
    • Currently use the SplitMix algorithms

Property Verification

  • Boolean Property
  • Test Invariants
  • Operation Invariance
  • Testable Trait
  • Existential Quantification

Shrinking

  • Linear Shrinking
  • Integrated Shrinking
  • Internal Shrinking

References