testRunnamespacenamespace testRunA test run as evidence (commons 0001-quality-evidence FR-002 to FR-004, FR-008, FR-009): runTests starts one trace with a test run span, runs bun test or node --test (one or both in turn) with TRACEPARENT and IFC_TEST_RECORD set, passes each runner's own output on, gathers every process's records into one OTLP JSON file (<out>/<trace id>.otlp.json) with the run's span, applies the exit rule, renders the reports asked for from the record as written, appends the record's digest to an evidence ledger and, where a key is given, signs an in-toto Test Result statement of it in a DSSE envelope whose signature is entered in the ledger with its meaning and role. signRun signs a record later (a review, an approval), and verifyRun checks a record against the ledger and the envelopes. A runtime that is not installed is reported and fails the run. test-cli.ts is the command line.
import { runTests } from "./test-run.ts";
const r = await runTests({ runtimes: ["node"], paths: ["tests"], out: "test-results", reports: ["junit", "md"] });
if (r.kind === "ok") console.log(r.value.recordPath, r.value.code);testRunnamespacenamespace testRuntestRun.Runtimetypeno test imports ittype Runtime = "bun" | "node";A runtime whose runner a run uses.
testRun.RUNTIMESconstno test imports itconst RUNTIMES: readonly Runtime[]Every runtime.
testRun.ReportFormattypeno test imports ittype ReportFormat = "tap" | "junit" | "ctrf" | "sarif" | "md" | "html" | "oscal";A report a run renders from its record (FR-005, FR-006).
testRun.REPORT_FORMATSconstno test imports itconst REPORT_FORMATS: readonly ReportFormat[]Every report format.
testRun.reportNamefunctionno test imports itfunction reportName(traceId: string, format: ReportFormat): stringThe file name of a report of a run.
testRun.SigningKeytypeno test imports ittype SigningKey = CryptoKey | string;A key to sign with: a Web Crypto private key, or the path of a PKCS#8 PEM file of an Ed25519 or ECDSA P-256 key.
testRun.SignOptionstypeno test imports ittype SignOptions = { readonly key: SigningKey; readonly meaning: Meaning; readonly role: string; readonly keyid?: string };A signature to make: the key, its meaning, the role that signs, and the key's id in the envelope.
testRun.RunOptionstypeno test imports ittype RunOptions = {
/** The runners, in turn (the runtime this code runs on by default). */
readonly runtimes?: read… (1431 characters)type RunOptions = {
/** The runners, in turn (the runtime this code runs on by default). */
readonly runtimes?: readonly Runtime[];
/** Test files and folders (each `*.test.ts`, `*.test.mts`, `*.test.js` or `*.test.mjs` under a folder); `tests` by default. */
readonly paths?: readonly string[];
/** Where the record and the reports go (`test-results` by default). */
readonly out?: string;
/** The working folder of the runners (this process's by default). */
readonly cwd?: string;
/** A file of requirement ids, or a spec-kit enforcement register (FR-003). */
readonly requirements?: string;
/** Keep only the requirements of this specification (`0005-typescript-practice`). */
readonly spec?: string;
/** A file of reviewed requirements: the requirement, a tab and the reason, one per line. */
readonly reviewed?: string;
/** The evidence ledger (JSON lines) the record's digest is appended to (FR-008). */
readonly ledger?: string;
readonly sign?: SignOptions;
readonly reports?: readonly ReportFormat[];
/** Whether the runners' output is passed on (`inherit`, the default) or dropped (`null`). */
readonly output?: "inherit" | "null";
/** Variables added to the runners' environment. */
readonly env?: Readonly<Record<string, string>>;
/** The executable of a runtime, where it is not the one on `PATH`. */
readonly executables?: Readonly<Partial<Record<Runtime, string>>>;
};How to run a suite.
testRun.RunResulttypeno test imports ittype RunResult = {
/** 0 where the run passed, 1 where it failed (FR-003). */
readonly code: 0 | 1;
readonly recordPath: string;
/** The SHA-256 of the record's bytes (FR-004). */
readonly digest: Sha256;
readonly run: TestRun;
readonly reports: readonly string[];
readonly ledger?: readonly LedgerEntry[];
readonly envelopePath?: string;
};What a run did.
testRun.testFilesfunctionno test imports itfunction testFiles(paths: readonly string[], cwd: string): Promise<Result<readonly string[], string>>The test files of some paths: a file as given, a folder as the test files under it, each absolute and once, sorted.
testRun.readLedgerfunctionno test imports itfunction readLedger(path: string): Promise<Result<readonly LedgerEntry[], string>>A ledger read from its file, empty where there is none.
testRun.loadPrivateKeyfunctionno test imports itfunction loadPrivateKey(pathOrPem: string, extractable?: boolean): Promise<Result<CryptoKey, string>>A private key from a PKCS#8 PEM file (or its text) of an Ed25519 or ECDSA P-256 key.
testRun.loadPublicKeyfunctionno test imports itfunction loadPublicKey(pathOrPem: string): Promise<Result<CryptoKey, string>>A public key from an SPKI PEM file (or its text), or the public half of a PKCS#8 private key, of Ed25519 or ECDSA P-256.
testRun.writeReportsfunctionno test imports itfunction writeReports(run: TestRun, out: string, formats: readonly ReportFormat[], extra?: { readonly digest?: Sha256 | undefined; readonly ledger?: readonly LedgerEntry[] | undefined; }): Promise<readonly string[]>The reports asked for, rendered from the record as read back, written to out; their paths.
testRun.runTestsfunctionno test imports itfunction runTests(options?: RunOptions): Promise<Result<RunResult, string>>A suite run as evidence (FR-002 to FR-004, FR-008): every runner in turn, one record, the exit rule, the reports, the ledger and the signature asked for. An error is something that kept the run from being recorded (a path that does not exist, a list that cannot be read); a run that fails is ok with code 1.
testRun.loadRecordfunctionno test imports itfunction loadRecord(path: string): Promise<Result<{ readonly run: TestRun; readonly digest: Sha256; }, string>>A record read from its file, with its digest.
testRun.signRunfunctionno test imports itfunction signRun(options: { readonly record: string; readonly ledger: string; readonly sign: SignOptions; }): Promise<Result<{ readonly envelopePath: string; readonly ledger: readonly LedgerEntry[]; }, string>>A record signed after its run (a review, an approval): the envelope beside it, and its signature in the ledger (FR-008, FR-010).
testRun.verifyRunfunctionno test imports itfunction verifyRun(options: { readonly record: string; readonly ledger: string; readonly envelopes?: readonly string[] | undefined; readonly publicKey?: string | CryptoKey | undefined; readonly keyid?: string | undefined; readonly requireReview?: boolean | undefined; }): Promise<Result<Verification & { readonly digest: Sha256; }, readonly string[]>>A record checked (FR-009, FR-010): its digest in the ledger, the ledger's chain whole, each envelope verified with the key and naming the digest, and, with requireReview, a signature of meaning reviewed in the ledger. Each problem as a sentence.
Runtimetypetype Runtime = "bun" | "node";A runtime whose runner a run uses.
RUNTIMESconstno test imports itconst RUNTIMES: readonly Runtime[]Every runtime.
ReportFormattypeno test imports ittype ReportFormat = "tap" | "junit" | "ctrf" | "sarif" | "md" | "html" | "oscal";A report a run renders from its record (FR-005, FR-006).
REPORT_FORMATSconstconst REPORT_FORMATS: readonly ReportFormat[]Every report format.
reportNamefunctionno test imports itfunction reportName(traceId: string, format: ReportFormat): stringThe file name of a report of a run.
SigningKeytypeno test imports ittype SigningKey = CryptoKey | string;A key to sign with: a Web Crypto private key, or the path of a PKCS#8 PEM file of an Ed25519 or ECDSA P-256 key.
SignOptionstypeno test imports ittype SignOptions = { readonly key: SigningKey; readonly meaning: Meaning; readonly role: string; readonly keyid?: string };A signature to make: the key, its meaning, the role that signs, and the key's id in the envelope.
RunOptionstypeno test imports ittype RunOptions = {
/** The runners, in turn (the runtime this code runs on by default). */
readonly runtimes?: read… (1431 characters)type RunOptions = {
/** The runners, in turn (the runtime this code runs on by default). */
readonly runtimes?: readonly Runtime[];
/** Test files and folders (each `*.test.ts`, `*.test.mts`, `*.test.js` or `*.test.mjs` under a folder); `tests` by default. */
readonly paths?: readonly string[];
/** Where the record and the reports go (`test-results` by default). */
readonly out?: string;
/** The working folder of the runners (this process's by default). */
readonly cwd?: string;
/** A file of requirement ids, or a spec-kit enforcement register (FR-003). */
readonly requirements?: string;
/** Keep only the requirements of this specification (`0005-typescript-practice`). */
readonly spec?: string;
/** A file of reviewed requirements: the requirement, a tab and the reason, one per line. */
readonly reviewed?: string;
/** The evidence ledger (JSON lines) the record's digest is appended to (FR-008). */
readonly ledger?: string;
readonly sign?: SignOptions;
readonly reports?: readonly ReportFormat[];
/** Whether the runners' output is passed on (`inherit`, the default) or dropped (`null`). */
readonly output?: "inherit" | "null";
/** Variables added to the runners' environment. */
readonly env?: Readonly<Record<string, string>>;
/** The executable of a runtime, where it is not the one on `PATH`. */
readonly executables?: Readonly<Partial<Record<Runtime, string>>>;
};How to run a suite.
testFilesfunctionno test imports itfunction testFiles(paths: readonly string[], cwd: string): Promise<Result<readonly string[], string>>The test files of some paths: a file as given, a folder as the test files under it, each absolute and once, sorted.
loadPrivateKeyfunctionno test imports itfunction loadPrivateKey(pathOrPem: string, extractable?: boolean): Promise<Result<CryptoKey, string>>A private key from a PKCS#8 PEM file (or its text) of an Ed25519 or ECDSA P-256 key.
loadPublicKeyfunctionno test imports itfunction loadPublicKey(pathOrPem: string): Promise<Result<CryptoKey, string>>A public key from an SPKI PEM file (or its text), or the public half of a PKCS#8 private key, of Ed25519 or ECDSA P-256.
writeReportsfunctionfunction writeReports(run: TestRun, out: string, formats: readonly ReportFormat[], extra?: { readonly digest?: Sha256 | undefined; readonly ledger?: readonly LedgerEntry[] | undefined; }): Promise<readonly string[]>The reports asked for, rendered from the record as read back, written to out; their paths.
runTestsfunctionfunction runTests(options?: RunOptions): Promise<Result<RunResult, string>>A suite run as evidence (FR-002 to FR-004, FR-008): every runner in turn, one record, the exit rule, the reports, the ledger and the signature asked for. An error is something that kept the run from being recorded (a path that does not exist, a list that cannot be read); a run that fails is ok with code 1.
loadRecordfunctionno test imports itfunction loadRecord(path: string): Promise<Result<{ readonly run: TestRun; readonly digest: Sha256; }, string>>A record read from its file, with its digest.
signRunfunctionfunction signRun(options: { readonly record: string; readonly ledger: string; readonly sign: SignOptions; }): Promise<Result<{ readonly envelopePath: string; readonly ledger: readonly LedgerEntry[]; }, string>>A record signed after its run (a review, an approval): the envelope beside it, and its signature in the ledger (FR-008, FR-010).
verifyRunfunctionfunction verifyRun(options: { readonly record: string; readonly ledger: string; readonly envelopes?: readonly string[] | undefined; readonly publicKey?: string | CryptoKey | undefined; readonly keyid?: string | undefined; readonly requireReview?: boolean | undefined; }): Promise<Result<Verification & { readonly digest: Sha256; }, readonly string[]>>A record checked (FR-009, FR-010): its digest in the ledger, the ledger's chain whole, each envelope verified with the key and naming the digest, and, with requireReview, a signature of meaning reviewed in the ledger. Each problem as a sentence.