Esc

Testing

jacket test builds the @test procedures once and runs each in its own process. jacket test covers filters, coverage and the test library.

Build the tests once, then run each in its own process (-os/-cpu as in jaic; flags after -- go to jaic)

jacket test [OPTIONS] [FILTER]... [-- <FLAGS>...]

Arguments

ArgumentMeaning
[FILTER]...Run the tests whose name contains one of these; test/subtest runs one subtest
[FLAGS]...Flags for jaic, after --

Options

OptionMeaning
--exactFilters are whole names, not parts of names
-E, --filter-expr <EXPR>Run the tests a filter expression selects, as module(Sort) and not kind(property); with FILTERs, the tests both select (predicates: test, module, file, kind, all, none)
--listPrint the tests (and benchmarks) instead of running them
--list-targetWith --list: discover the selected target's compile-time cases instead of the host's
--rootsPrint the test roots (the folders [test] paths expands to, narrowed by the current directory), one per line, and nothing else
--depsWith --list: print the source files the test programs read (what jaic -deps prints for each), instead of the tests
--changed <FILE>Run only the tests a change to FILE can affect: those whose programs read it (as --list --deps shows), the tests of a root it adds an unlisted file to, and tests whose program does not compile. Repeat it for each changed file; it applies before --partition
--coverageReport which lines of the code the tests ran (unit tests, golden programs and doc examples), and write them as lcov
--coverage-out <FILE>Where --coverage writes its report [default: target/lcov.info, or target/coverage.html]
--coverage-format <COVERAGE_FORMAT>The report's format Possible values: - lcov: lcov's text format, with line counts and procedures - html: One self-contained page: modules, files and the source with the count of each line [default: lcov]
--coverage-by <COVERAGE_BY>What a row of the printed table is Possible values: - file - module: The nearest folder with a module.jai [default: file]
--coverage-baseline <FILE>Fail when a module (or the total) covers fewer lines than this recorded baseline by more than --coverage-margin
--coverage-margin <POINTS>Percentage points a module may lose against --coverage-baseline [default: 0.5]
--coverage-record <FILE>Record this run's lines per module as a baseline
--coverage-summary <FILE>Add a markdown table of the modules to this file ($GITHUB_STEP_SUMMARY)
--coverage-include <DIR>Also report the code below this folder (repeatable); by default the test roots and the folders of -import_dir
--watchRun again whenever a .jai file of the tests or the code they import changes
-j, --jobs <N>Tests run at once, as jaic's -j [default: the machine's threads]
--timeout <SECONDS>Seconds before a test without @timeout is killed; 0 for no limit [default: 120]
--retries <N>Run a failed test again up to N times; one that then passes is FLAKY
--fail-fastStart no more tests after the first failure
--no-fail-fastRun every test even when a profile sets fail_fast
--no-captureShow the tests' output as it is written (runs one test at a time)
--no-isolationRun a root's tests in one process instead of one process each
-q, --quietPrint only the tests that fail, time out or are flaky, then the summary
--format <FORMAT>human text, or json: one event per line on standard output Possible values: - human - json: One JSON object per event on standard output [default: human]
--junit <FILE>Also write the results as JUnit XML
--partition <SPEC>Run one part of the tests: hash:K/N (by name) or count:K/N (by position)
--interpRun the tests in jaic's interpreter
--os <OS>Target OS, as jaic's -os (also spelt -os) [possible values: linux, windows, macos, wasm]
--cpu <CPU>Target CPU, as jaic's -cpu (also spelt -cpu)
--runner <COMMAND>Command that runs test programs built for another target (node tools/wasi_run.ts)
--needs <LIST>Capabilities this machine has, for tests with @needs (network,gpu)
--profile <NAME>Use [test.profile.NAME] from jacket.toml
--path <DIR>Test the tests/ of these folders instead of the project's
--docRun only the doc tests: the jai code blocks in a library's doc comments
--check-onlyOnly type-check the doc examples, without building or running them: with --os, the examples for that target from any machine
--no-docLeave the doc tests out
--keep-tempKeep every test's temporary folder (failed tests keep theirs anyway)
--acceptWrite what golden and compile-fail tests printed into their expected files, and keep the snapshots that differ or are new
--reviewGo through the pending .snap.new snapshots one by one (only lists them without a terminal)
--seed <N>The seed property tests start from [default: random for each run, shown with a failure]
-h, --helpPrint help (see a summary with '-h')