IF Commons GitHub

Test

0001Draft

0001-test · owner if · domain commons

Registration

FR-001checkpassing

Tests MUST register through test, which MUST run on node:test and assert with node:assert/strict under short plain names (assert, assertEquals, assertStringIncludes, ...); proves(requirements, name, fn) MUST name each test for the requirements it proves and keep them for a report. The same test files MUST pass unchanged under bun test and node --test.

Enforced (check): .github: cd commons/test && bun test suite/ — test.test.ts: the assertions and proves; the same tests pass under bun test and node --test in CI

Proved by: test/suite/test.test.ts

Written in: test/src/test.ts

Tests: passed: suite/test.test.ts each assertion passes on what it describes and throws on what it does not; passed: suite/test.test.ts proves names the test for its requirements and keeps them; reviewed keeps the reason

Cited by: Test, test, Test evidence

FR-002checkpassing

test MUST give only (a test run alone where any test is marked so: node --test --test-only runs it of itself, and on bun test the other tests of the file skip themselves), steps (t.step(name, fn) as subtests of node:test, recorded in the test's evidence as commons 0008-test-evidence records a test), and assertAlmostEquals, assertNotMatch and assertNotStrictEquals. Resource sanitizers and per-test permissions MUST NOT be given, as Node and Bun have neither: a test closes what it opens, by using and await using.

Enforced (check): .github: cd commons/test && bun test suite/ — test.test.ts: only, steps and the three asserts

Proved by: test/suite/test.test.ts

Tests: passed: suite/test.test.ts a failed step fails its test with its error; each step is a span under its test's span and; passed: suite/test.test.ts a test marked only runs alone on this runner, and the others of its file are recorded as s; passed: suite/test.test.ts assertAlmostEquals, assertNotMatch and assertNotStrictEquals keep @std/assert's meaning; passed: suite/test.test.ts each step is a subtest, awaited, and recorded in its test's evidence with its own steps; t

Cited by: test-report, Test, record, test

FR-003checkpassing

test MUST give the(value, message?), which fails as assertExists does where the value is null or undefined and otherwise gives the value back narrowed to NonNullable, so that a test reads the(map.get(k)).name; assertExists MUST keep its form as an assertion.

Enforced (check): .github: cd commons/test && bun test suite/ — test.test.ts: the gives back the value narrowed, and fails on null and undefined

Proved by: test/suite/test.test.ts

Tests: passed: suite/test.test.ts the gives the value back narrowed, and fails on null and undefined

The record of a test

FR-004checkpassing

test MUST record every test it registers, by its own code and not a runner's reporter: its id (the SHA-256 of its file and name), its file, its name, the requirements it proves, its risk (high or not-high, given as an option and not-high by default, after the Computer Software Assurance guidance's high process risk), its start and end time, its outcome (pass | fail | skip | todo | cancelled), a skip's or todo's reason, a failure's message, expected and actual values (redacted by commons 0006-signals FR-006 and cut to 4096 characters), and the attempt number.

Enforced (check): .github: cd commons/test && bun test suite/ — test.test.ts: a test's record with its outcome, values, risk and attempt

Proved by: test/suite/test.test.ts

Written in: test/src/record.ts

Tests: passed: suite/test.test.ts each test is recorded in memory: id, file from the caller, requirements, risk, times, outc

Cited by: Test evidence

FR-005checkno test run

Each test MUST be recorded as a test span and an OpenTelemetry log record, with the test.case.name, test.case.result.status (pass or fail, as OpenTelemetry defines it), test.suite.name (the file) and test.suite.run.status conventions and the ifc.test.* attributes for the rest; its parent MUST be the run's span from TRACEPARENT (commons 0006-signals FR-017).

Enforced (check): .github: cd commons/test && bun test suite/ — test.test.ts: a test span and log record with the test.* conventions

Cited by: Test evidence

FR-006checkno test run

The resource of the record MUST be the installation record: process.runtime.name and process.runtime.version, os.type, host.arch, vcs.ref.head.revision where it is known, ifc.test.lockfile.sha256 and ifc.test.pins.sha256 where given, and ifc.test.role (the role that ran it, ci or developer by default). It MUST NOT carry a person's name, user name, e-mail address or host name; an attribute of the conventions that names a person (user.*, enduser.*, host.name) MUST be refused.

Enforced (check): .github: cd commons/test && bun test suite/ — test.test.ts: the installation record, and a person's name refused

Cited by: Test evidence

The package

FR-007checkno test run

The library MUST be published to npm as compiled modules with their declarations, one entry for the package (., the registration and the assertions) and one for the record (./record, the pure model of a test's record and its OpenTelemetry form), built, inspected and packed by tools/pack.ts (commons 0032-libraries-by-concern FR-005, FR-013), and it MUST depend only on core, crypto, system and telemetry, so that it imports nothing of the evidence's reports, ledger or runner.

Enforced (check): .github: cd commons/test && bun run verify — the package is built, inspected and packed by tools/pack.ts; tests/commons-packages.test.ts installs the tarball, runs every entry on Node 26 and Bun and type-checks a consumer

Written in: test/about.ts, test/library.ts, test/tools/check.ts, test/tools/pack.ts

Acceptance scenarios

  1. Given a test file that imports proves and the assertions from this package, When it runs under bun test and under node --test, Then it passes unchanged on both, per FR-001.
  2. Given a test marked only, When its file runs, Then it runs alone and the others of the file are recorded as skipped, per FR-002.
  3. Given a run with IFC_TEST_RECORD set, When a test ends, Then its span and log record are appended to that file, on the installation record, per FR-004 to FR-006.
  4. Given the package is installed in an empty project, When Node 26 and Bun import its entries and a consumer type-checks them, Then they work, per FR-007.

Out of scope

  • The run of a suite (runtime/test-run, ifc test), the renderings of a record (TAP, JUnit, CTRF, SARIF, OSCAL, the completion report), the evidence ledger and its signing: the quality system's, which depend on this library, not the reverse (commons 0008-test-evidence FR-005 to FR-014).

Edge cases

  • A person's name in the installation record's role: refused, per FR-006.
  • A test that has not ended when the process exits: recorded as cancelled, per commons 0008-test-evidence FR-004.
  • bun test --only: runs no test of node:test at all, as Bun's node:test never marks one; give bun test no --only, per FR-002.

Assumptions

  • node:test is the registration both runners implement.

Open questions

  • None.

Key entities

  • Test - a registered body with the requirements it proves and its risk.
  • Record - the span and the log record of a test, on the installation record.
  • Installation record - the resource that says where a test ran: runtime, system, architecture, revision, role.

Success criteria

SC-001success criterion

One import gives a harness its assertions and the evidence of its runs.

Review & acceptance checklist

  • [x] Every requirement is testable (MUST / MUST NOT), not aspirational
  • [x] No company fact is asserted here
  • [x] Every open item is marked, not silently decided
  • [x] Confidential: no sensitive fact written as a literal

Cited by

test-report, Test, record, test, Test evidence

Titles, headings, exports, tables, columns and requirements. Esc closes.