Outcometypeno test imports ittype Outcome = "pass" | "fail" | "skip" | "todo" | "cancelled";What became of a test (FR-001).
The capture half of test evidence (commons 0008-test-evidence FR-001 to FR-004), kept apart from the reports so that every test process loads only this: the model of a test case and of an installation, a test case as its test span and log record with the test.* conventions and the ifc.test.* attributes (testCaseSpan, testCaseLog), the mark a test leaves as it starts (testStartLog), so that one that never ends is closed as cancelled (closeOrphans), the installation record as a resource (installationFromEnv, which refuses a role that names a person, and isPersonAttribute), and the OTLP JSON lines a test process appends to IFC_TEST_RECORD (recordLine, decodeLines). ./test-report.ts reads a run's record and renders it. A test's steps (commons 0001-test FR-002) are its steps, named by its span's ifc.test.steps, each a test step span whose parent is the test's span or the step it is in (testStepSpan).
import { installationFromEnv, installationResource, recordLine, testCaseSpan } from "./record.ts";
const { installation } = installationFromEnv({ runtime: "node", runtimeVersion: "26.0.0", os: "linux", arch: "x86_64" }, { CI: "true" });
// recordLine(installationResource(installation), { spans: [testCaseSpan(test, context)] }) is one line of IFC_TEST_RECORDOutcometypeno test imports ittype Outcome = "pass" | "fail" | "skip" | "todo" | "cancelled";What became of a test (FR-001).
OUTCOMESconstno test imports itconst OUTCOMES: readonly Outcome[]Every outcome, in the order reports list them.
Risktypeno test imports ittype Risk = "high" | "not-high";A test's risk, after the Computer Software Assurance guidance's high process risk: high or not-high (FR-001).
isOutcomefunctionno test imports itfunction isOutcome(text: string): text is OutcomeWhether a text is an outcome.
isRiskfunctionno test imports itfunction isRisk(text: string): text is RiskWhether a text is a risk.
StepOutcometypeno test imports ittype StepOutcome = "pass" | "fail" | "skip";isStepOutcomefunctionno test imports itfunction isStepOutcome(text: string): text is StepOutcomeWhether a text is a step's outcome.
TestSteptypeno test imports ittype TestStep = {
readonly name: string;
readonly start: bigint;
readonly end: bigint;
readonly outcome: StepOutcome;
readonly message?: string;
readonly steps?: readonly TestStep[];
};TestCasetypeno test imports ittype TestCase = {
/** The SHA-256 of its file, a NUL and its name. */
readonly id: string;
/** Its file, relative … (895 characters)type TestCase = {
/** The SHA-256 of its file, a NUL and its name. */
readonly id: string;
/** Its file, relative to the working folder, with `/`. */
readonly file: string;
readonly name: string;
readonly requirements: readonly string[];
readonly risk: Risk;
/** The quality characteristic it proves, in the words of ISO/IEC 25010 (`ifc.test.characteristic`, 0021-ifqs FR-012); absent where it names none. */
readonly characteristic?: string;
readonly start: bigint;
readonly end: bigint;
readonly outcome: Outcome;
/** A skip's or a todo's reason, or why it was cancelled. */
readonly reason?: string;
readonly message?: string;
readonly expected?: string;
readonly actual?: string;
/** From 1. */
readonly attempt: number;
/** Its steps, in the order they ended; absent where it has none (0001-test FR-002). */
readonly steps?: readonly TestStep[];
};One test as it ran once (FR-001): times in nanoseconds since the Unix epoch; text already redacted and cut.
Installationtypeno test imports ittype Installation = {
readonly runtime: string;
readonly runtimeVersion: string;
readonly os: string;
readonly arch: string;
readonly revision?: string;
readonly lockfileSha256?: string;
readonly pinsSha256?: string;
readonly role: string;
};The installation record: the resource of a test process or of the run (FR-003).
Reviewtypeno test imports ittype Review = { readonly requirement: string; readonly reason: string };A requirement declared proven by review rather than by a test, with the reason.
RunnerOutcometypeno test imports ittype RunnerOutcome =
{ readonly kind: "exited"; readonly runtime: string; readonly code: number } | { readonly kind: "missing"; readonly runtime: string };What one runner did: the runtime, its exit status, or that it is not installed.
TEST_SCOPEconstno test imports itconst TEST_SCOPE: ScopeThe instrumentation scope of test records.
TEST_SPANconstno test imports itconst TEST_SPAN: "test"The name of a test's span.
STEP_SPANconstconst STEP_SPAN: "test step"RUN_SPANconstno test imports itconst RUN_SPAN: "test run"The name of a run's span.
EVENTSconstno test imports itconst EVENTS: { readonly case: "ifc.test.case"; readonly start: "ifc.test.start"; readonly review: "ifc.test.review"; readonly runner: "ifc.test.runner"; readonly warning: "ifc.test.warning"; }The event names of the log records of a test record.
TEXT_LIMITconstno test imports itconst TEXT_LIMIT: 4096The most characters a message, an expected or an actual value keeps (FR-001).
cutfunctionno test imports itfunction cut(text: string): stringA text cut to TEXT_LIMIT characters (without splitting a surrogate pair).
testIdfunctionno test imports itfunction testId(file: string, name: string): Promise<Sha256>A test's id: the SHA-256 of its file, a NUL and its name (FR-001).
recordPathfunctionno test imports itfunction recordPath(path: string): stringA file path as a record names it: \ as /, without a leading ./.
isPersonAttributefunctionno test imports itfunction isPersonAttribute(key: string): booleanWhether a resource attribute names a person or a machine of one, and is refused (FR-003): user.*, enduser.*, host.name.
hostArchfunctionno test imports itfunction hostArch(arch: string): stringA machine's architecture as OpenTelemetry's host.arch names it (amd64, arm64, ...).
installationFromEnvfunctionno test imports itfunction installationFromEnv(self: { readonly runtime: string; readonly runtimeVersion: string; readonly os: string; readonly arch: string; }, env: Readonly<Record<string, string | undefined>>): { readonly installation: Installation; readonly warnings: readonly string[]; }The installation record of a process, from what the runtime says of itself and the environment given as a record (the kernel reads no environment): IFC_TEST_REVISION or GITHUB_SHA, IFC_TEST_LOCKFILE_SHA256, IFC_TEST_PINS_SHA256, and IFC_TEST_ROLE (ci where CI is set, else developer). A role that is not a role (an e-mail address, a name with capitals or spaces) is refused with a warning and the default used, and so is a digest that is not a SHA-256.
installationResourcefunctionno test imports itfunction installationResource(i: Installation): ResourceThe resource of an installation record (FR-003).
attributeTextfunctionno test imports itfunction attributeText(a: Readonly<Record<string, AttributeValue>>, key: string): string | undefinedAn attribute's value where it is a string.
attributeTextsfunctionno test imports itfunction attributeTexts(a: Readonly<Record<string, AttributeValue>>, key: string): readonly string[]An attribute's value where it is an array of strings, else none.
attributeIntegerfunctionno test imports itfunction attributeInteger(a: Readonly<Record<string, AttributeValue>>, key: string): number | undefinedAn attribute's value where it is an integer.
installationOffunctionfunction installationOf(r: Resource): InstallationThe installation record a resource holds; unknown fields are unknown.
TestContexttypeno test imports ittype TestContext = { readonly traceId: TraceId; readonly spanId: SpanId; readonly parentSpanId?: SpanId };Where a test's signals belong: the trace, the test's own span id, and the run's span as its parent where there is one.
stepPathsfunctionno test imports itfunction stepPaths(steps: readonly TestStep[], within?: readonly string[]): string[]The steps of a test, each by its path (a step within another is outer > inner), in their order with each step's own steps first.
testStepSpanfunctionno test imports itfunction testStepSpan(step: TestStep, test: Pick<TestCase, "file" | "id" | "name">, ctx: TestContext): SpanDatatestCaseSpanfunctionno test imports itfunction testCaseSpan(c: TestCase, ctx: TestContext): SpanDataA test as its test span (FR-002): the conventions, the ifc.test.* attributes, and an error status where it failed or was cancelled.
testCaseLogfunctionno test imports itfunction testCaseLog(c: TestCase, ctx: TestContext): LogRecordA test as its log record (FR-002): the same attributes, the name as its body, at ERROR where it failed or was cancelled.
testStartLogfunctionno test imports itfunction testStartLog(c: Omit<TestCase, "end" | "outcome">, ctx: TestContext): LogRecordThe mark a test leaves as it starts (FR-004): its span id, so that a test whose span never comes is closed as cancelled.
reviewLogfunctionno test imports itfunction reviewLog(r: Review, time: bigint, traceId?: TraceId | undefined): LogRecordA review declared in a test file or a list (ifc.test.review).
warningLogfunctionno test imports itfunction warningLog(message: string, time: bigint): LogRecordA warning of a test process or a run (ifc.test.warning), such as a role refused.
runnerLogfunctionno test imports itfunction runnerLog(r: RunnerOutcome, time: bigint): LogRecordWhat one runner did (ifc.test.runner).
recordLinefunctionno test imports itfunction recordLine(resource: Resource, signal: { readonly spans: readonly SpanData[]; } | { readonly records: readonly LogRecord[]; }): stringOne line of IFC_TEST_RECORD: an OTLP JSON traces or logs export request of one resource, without a line feed (FR-004).
RecordErrortypeno test imports ittype RecordError =
| { readonly kind: "json"; readonly line: number; readonly reason: string }
| { readonly kind: "otlp"; readonly line: number; readonly error: DecodeError }
| { readonly kind: "person"; readonly key: string }
| { readonly kind: "test"; readonly spanId: string; readonly reason: string }
| { readonly kind: "no-run"; readonly reason: string };Why the lines of a test record or a run's record could not be read.
formatRecordErrorfunctionno test imports itfunction formatRecordError(e: RecordError): stringA record error as one sentence.
Signalstypeno test imports ittype Signals = { readonly traces: readonly TraceBatch[]; readonly logs: readonly LogBatch[] };The signals of one or more export requests.
decodeRecordDocumentfunctionno test imports itfunction decodeRecordDocument(value: unknown, line: number): Result<Signals, RecordError>The traces and logs of one OTLP JSON document (an export request, or a run's record holding both), as line line.
decodeLinesfunctionfunction decodeLines(input: string): Result<Signals & { readonly warnings: readonly string[]; }, RecordError>The signals of IFC_TEST_RECORD's lines, each an OTLP JSON export request (FR-004). A last line cut short (a process killed while it wrote) is left out with a warning; any other line that is not one is an error.
mergeBatchesfunctionno test imports itfunction mergeBatches(signals: Signals): SignalsBatches of one resource and scope joined into one, in the order they first came.
closeOrphansfunctionno test imports itfunction closeOrphans(signals: Signals, at: bigint): SignalsThe traces with a cancelled test span for each test that started and has no span (FR-004): its process died, or exited, before the test ended. Its end is at, the time the run saw its process end.