Skip to content
Bynk

Run your tests

Goal: run a project’s suite and integration blocks, read pass/fail, and — in your editor — click straight from a failing expectation to the line that failed.

Once you’ve written some tests, run them with bynkc test:

Terminal window
bynkc test            # from the project root

bynkc test compiles the project (every generated tests/*.test.ts module included), then runs the aggregated runner under Node — type-checking as it goes, so a type error stops you with the usual diagnostics. It exits non-zero if any case fails. You need node and tsc (or tsx) on your PATH; check with bynk doctor --only test.

commerce.money:
  ✓ accepts positive
  ✗ deliberate failure
    expect total == 900
      expected: total == 900
      actual:   950 == 900
    at tests/commerce/money.bynk:8:12


1 passed, 1 failed.

A failed expect reports the predicate and — for a top-level comparison — its expected-vs-actual operands, plus the path:line:col of the line that failed, for both unit and integration tests.

Generative failures: seeds, shrinking, and --seed

Section titled “Generative failures: seeds, shrinking, and --seed”

A property runs its for all body over many generated inputs. When one fails, the report tells you how many cases ran, the run’s root seed, and a shrunk counterexample — the smallest inputs that still fail — with a copy-paste line to reproduce it:

commerce.money › more discount, never a higher price
  property failed after 41 cases (seed 0x5f3a)
  shrunk counterexample:  p = 100, a = 10, b = 20
  expect discount(p, b) <= discount(p, a)
  reproduce: bynkc test commerce/money.bynk --seed 0x5f3a

Each run draws a fresh random seed, printed only on failure. Thread it back with bynkc test --seed <hex> (e.g. --seed 0x5f3a) to make the run reproduce byte-for-byte — the same generated inputs, the same shrink, every time. This is what the reproduce line pastes for you, so a flaky-looking property becomes a deterministic one you can debug.

Machine-readable results: --format json

Section titled “Machine-readable results: --format json”

For CI and tooling, --format json emits a single, stable JSON document instead of the human lines:

Terminal window
bynkc test --format json
{
  "passed": 10,
  "failed": 1,
  "suites": [
    {
      "name": "commerce.money",
      "kind": "unit",
      "cases": [
        {"name": "accepts positive", "outcome": "pass"},
        {"name": "deliberate failure", "outcome": "fail",
         "message": "expect total == 900\n  expected: total == 900\n  actual:   950 == 900\n  at tests/commerce/money.bynk:8:12",
         "location": {"path": "tests/commerce/money.bynk", "line": 8, "col": 12}}
      ]
    }
  ]
}

Each suite carries a kind ("unit" or "integration") and a clean name; a failing case carries a message and a structured location for click-through. The exit code is unchanged — 0 only if the project compiled and every case passed.

There are three shapes a consumer should handle, distinguished by error:

  • a normal run — suites present, no error (it may still have failed > 0);
  • a compile failure — no suites; error.kind is "compile" and error.diagnostics carries the path:line:col: severity[category]: message lines (the same shape the editor’s problem-matcher reads);
  • a crashed run — the suites observed before the crash, plus error.kind "runtime" with the captured stderr.

The exit code always follows the runner’s own process status, so a crash is never reported as success.

Coverage: --coverage

Section titled “Coverage: --coverage”

--coverage runs the suite as normal, then reports statement/line coverage attributed to your .bynk source — not the generated TypeScript:

Terminal window
bynkc test --coverage
✓ 12 passed  (rate-limiter)


Coverage
  src/limiter.bynk    ▓▓▓▓▓▓··   86%   (42/49 lines)
  src/decide.bynk     ▓▓▓▓▓▓▓▓  100%   (11/11 lines)
  src/entry.bynk      ▓▓▓▓▓···   71%   (20/28 lines)  uncovered: 34-38, 51
  ─────────────────────────────────────────────────
  total                          84%   (73/88 lines)

Coverage is collected out-of-band by the runtime (V8’s NODE_V8_COVERAGE) and remapped through the same source maps the debugger trusts, so the generated .ts/.js layer is invisible: a covered line is a .bynk line. Generated glue with no source origin (codec wrappers, capability-injection) is out-of-scope — never counted as uncovered user code — and the measured set excludes your tests/ tree and the workers scaffold an integration suite stands up, so what you see is the source you are actually testing.

--coverage needs the tsc → node run: it is incompatible with --inspect and --no-run, and requires tsc and node on your PATH (the tsx fallback is not supported for coverage). Any of those combinations fails with an actionable message rather than silently producing wrong numbers. This increment reports line/statement coverage; per-branch coverage is a planned follow-up.

With --format json, the same numbers arrive as a coverage block instead of the table — for a CI artifact:

{
  "passed": 12,
  "failed": 0,
  "suites": [ /* … */ ],
  "coverage": {
    "covered": 73,
    "lines": 88,
    "percent": 84,
    "files": [
      {"path": "src/limiter.bynk", "covered": 42, "lines": 49, "percent": 86,
       "uncovered": []},
      {"path": "src/entry.bynk", "covered": 20, "lines": 28, "percent": 71,
       "uncovered": [34, 35, 36, 37, 38, 51]}
    ]
  }
}

The block is present only for a --coverage run that attributed lines; a normal run omits it, so an existing consumer’s document is byte-for-byte unchanged. Every path is a project-relative .bynk file, and uncovered lists the 1-based source lines that never ran.

In the editor: the Test Explorer

Section titled “In the editor: the Test Explorer”

The VS Code extension consumes that JSON surface directly. Open the Testing view (the beaker icon): the tree populates by discovery — bynkc test --no-run --format json lists your suites and cases without running them, so each test links to its .bynk line before you run anything (use the Refresh control to re-discover after edits). Run from the tree, or invoke Bynk: Run Tests from the command palette; results then show inline, a failing expectation links to its .bynk line, and a compile failure lands in the Problems panel exactly as bynkc check does. The extension resolves bynkc the same way the check task does — the bynk.compilerPath setting, else bynkc on PATH.

Section titled “Related”