Global idea

Check is a personal project for now. It is very raw and undocumented at this stage.

It's an exploration at writing a test framework that addresses some of jest's shortcomings (imho):

when multiple tests fail in a block, you only see the first one
the automatic hoisting of imports and mocks
- this makes it difficult to write you own helpers on top of jest
the need to use beforeEach / afterEach (detailed explanation here)
- which result in the need to use convoluted syntax when you need to access something inside the beforeEach
missing matchers for React-based use-cases (probably other frameworks as well)
- when testing a component, it's difficult to test for only some props
...

setup

install dependencies by running yarn

create your .env file by copying .env.sample details here

try it out

To try it out, clone the repository, have a look at files in the src/example folder: example.ts & test files.

You can first run the test using jest with yarn jest → 3 tests will fail, 1 because it's supposed to, and 2 because of how jest execute the tests. ❌

You can then run the test using this new framework with yarn check → 1 test will fail (it's supposed to) and all other will pass. ✔

what's included

CLI (all arguments are optional)

argument	default value	meaning
configuration file path	`check.config.json`	cf. below
`--watch`	false	will watch for file changes to re-trigger tests

Configuration file (optional)

parameter	default value	meaning
`testFilesPattern`	`*/.test.ts`	the pattern to get the test files to run
`watchFilesPattern`	`*/`	(watch mode only) the pattern to get the files to watch
`watchFilesIgnored`	`['node_modules', '.git', '.swc', 'dist', 'out']`	(watch mode only) files or folder not to watch

Watch mode

In watch mode, tests are re-run every time you save a file.

If only one test file is run, you'll see this file full test tree (and individual status) in your terminal

If more than one test file is run, you'll see only the global status for each test file

In both cases, you'll see details for failing tests (see examples below)

You also have access to additional functionalities as described in your terminal:

Ran all tests.

press <f> to filter
press <ctrl> + <c> to exit

pressing ctrl + c will stop the watch mode and exit
pressing f will allow you to enter a filter on which tests to run. You'll see a live list of files affected by your filter.
- once you press enter, only those tests will be run (still on watch mode)
- you can later press f again to change that filter (entering nothing runs all tests)

Test files & running them

general

in test files
- accepts describe and it / test syntax
- standard matcher are available:
  - toEqual matcher (uses lodash)
  - not function
- accepts spy definition & exposes related spy matchers (cf. section below)
prints the result in the console (using chalk)
- if only one test file, prints global result (PASS/FAIL), detailed tests tree & errors
- if multiple test files, prints global result (PASS/FAIL) & errors for each file (no detailed tests tree)
- in both cases, in case of errors, check continues to run tests and show all tests that are failing, not just the first one

spy

you can defined a spy by specifying the module & declaration you want to spy:

const spyExistsSync = spy('fs', 'existsSync')

optionally, you can add typing to your spy function:

import { type existsSync } from 'fs';

const spyExistsSync = spy<typeof existsSync>('fs', 'existsSync');

Regular matchers don't work on spy function, instead, they have their dedicated set of spy matchers. So far:

toBeCalled matcher (will compare the number in parameter with the spy function inner counter)
- first argument is the number of calls to match
- second argument (optional) is an array of parameters to match (match, not equal)
- if both are provided, it will succeed if the function has been called X times with arguments matching those provided

A note about typings:

check's types are smart enough so that depending on the parameter passed to the it/test function:

if this parameter is a spy functions, only spy matchers are available (not standard matchers)

if this parameter is not a spy function, only standard matchers are available (not spy matchers)

See dedicated doc for more advanced information about the spy function.

example of test outputs

multiple files	single file (PASS)	single file (FAIL)

check / jest comparison example

check	jest

output tests in the written order	reorders tests (strict hierarchical order)
all tests pass ✔	one test fail ❌

how it works (for now)

description

the whole thing is written in Typescript and runs using ts-node (with the currently experimental @swc transpiler for performance)
it works on Typescript test files (usually xxxx.test.ts)
each file is handled in its own thread using node's worker_thread
first, the parser (cf. src/parser) will:
- read the file (it must be utf-8)
- transpile it (using Typescript) in CommonJS & the Latest configuration from ts
- get the transpiled file tree (using Typescript's createSourceFile)
- parse it into JSON using the project's own parser (output in out/****-result.json if WRITE_DEBUG_FILES is set)
then, the runner will get the parser result and:
- transform it into runs (output in out/****-runs.json if WRITE_DEBUG_FILES is set)
  - a hierarchical representation of the test suites
  - with code & tests being grouped in an array at the test level
  - this array contains the whole code & test you pass through to go to that test → they will be executed this way
- execute those runs
  - using the node vm
  - using a brand new context for each run to avoid side-effects
  - output a result in the console (cf. images above)

Diagram

This diagram describes how tests are running without the watch mode.

TBD: describe how watch mode works internally

flowchart TD
  A[get test files] --> B(("one worker_thread\n per file"))
  B --> |test file 1| C1["read (utf-8)"]
  B --> |test file 2| C2["read (utf-8)"]
  subgraph parser
    direction LR
    subgraph file 1
      C1 --> D1[transpile\nusing Typescript]
      D1 --> E1["parse file tree\ninto a Line[]"]
    end
    subgraph file 2
      C2 --> D2[transpile\nusing Typescript]
      D2 --> E2["parse file tree\ninto a Line[]"]
    end
  end
  subgraph runner
    subgraph file 1
      E1 --> F1["transform\ninto Run[]"]
      F1 --> G1A["execute\nRun 1"]
      F1 --> G1B["..."]
      F1 --> G1C["execute\nRun n"]
      G1A --> H1["aggregate\nRun[] results"]
      G1B --> H1["aggregate\nRun[] results"]
      G1C --> H1["aggregate\nRun[] results"]
    end
    subgraph file 2
      E2 --> F2["transform\ninto Run[]"]
      F2 --> G2A["execute\nRun 1"]
      F2 --> G2B["..."]
      F2 --> G2C["execute\nRun n"]
      G2A --> H2["aggregate\nRun[] results"]
      G2B --> H2["aggregate\nRun[] results"]
      G2C --> H2["aggregate\nRun[] results"]
    end
  end
  H1 --> W[aggregate results]
  H2 --> W[aggregate results]
  W --> X{success}
  X --> |Yes| Y[Exit with status 0]
  X --> |No| Z[Exit with status 1]

.env

name	meaning
`WRITE_DEBUG_FILES`	`true` → write debug files (`result.json` & `runs.json`) in the `out` folder

e-krebs/check