IF Commons GitHub

record

Written to: 1 requirement: 1 proved, 0 partly, 0 failing, 0 with no test recorded. 0001-test FR-004

The capture half of test evidence (commons 0008-test-evidence FR-001 to FR-004), kept apart from the reports so that every test process loads only this: the model of a test case and of an installation, a test case as its test span and log record with the test.* conventions and the ifc.test.* attributes (testCaseSpan, testCaseLog), the mark a test leaves as it starts (testStartLog), so that one that never ends is closed as cancelled (closeOrphans), the installation record as a resource (installationFromEnv, which refuses a role that names a person, and isPersonAttribute), and the OTLP JSON lines a test process appends to IFC_TEST_RECORD (recordLine, decodeLines). ./test-report.ts reads a run's record and renders it. A test's steps (commons 0001-test FR-002) are its steps, named by its span's ifc.test.steps, each a test step span whose parent is the test's span or the step it is in (testStepSpan).

import { installationFromEnv, installationResource, recordLine, testCaseSpan } from "./record.ts";
const { installation } = installationFromEnv({ runtime: "node", runtimeVersion: "26.0.0", os: "linux", arch: "x86_64" }, { CI: "true" });
// recordLine(installationResource(installation), { spans: [testCaseSpan(test, context)] }) is one line of IFC_TEST_RECORD

Source: commons/test/src/record.ts

3 of 46 exports are imported by a test of the run. A test that imports an export may not call it or check what it returns.

Outcometypeno test imports it
type Outcome = "pass" | "fail" | "skip" | "todo" | "cancelled";

What became of a test (FR-001).

OUTCOMESconstno test imports it
const OUTCOMES: readonly Outcome[]

Every outcome, in the order reports list them.

Risktypeno test imports it
type Risk = "high" | "not-high";

A test's risk, after the Computer Software Assurance guidance's high process risk: high or not-high (FR-001).

isOutcomefunctionno test imports it
function isOutcome(text: string): text is Outcome

Whether a text is an outcome.

isRiskfunctionno test imports it
function isRisk(text: string): text is Risk

Whether a text is a risk.

StepOutcometypeno test imports it
type StepOutcome = "pass" | "fail" | "skip";

What became of a step (0001-test FR-002): it passed, it failed, or its runner did not run it.

isStepOutcomefunctionno test imports it
function isStepOutcome(text: string): text is StepOutcome

Whether a text is a step's outcome.

TestSteptypeno test imports it
type TestStep = {
  readonly name: string;
  readonly start: bigint;
  readonly end: bigint;
  readonly outcome: StepOutcome;
  readonly message?: string;
  readonly steps?: readonly TestStep[];
};

One step of a test as it ran (0001-test FR-002): its name, times, outcome, a failure's message, and its own steps.

TestCasetypeno test imports it
type TestCase = { /** The SHA-256 of its file, a NUL and its name. */ readonly id: string; /** Its file, relative … (895 characters)
type TestCase = {
  /** The SHA-256 of its file, a NUL and its name. */
  readonly id: string;
  /** Its file, relative to the working folder, with `/`. */
  readonly file: string;
  readonly name: string;
  readonly requirements: readonly string[];
  readonly risk: Risk;
  /** The quality characteristic it proves, in the words of ISO/IEC 25010 (`ifc.test.characteristic`, 0021-ifqs FR-012); absent where it names none. */
  readonly characteristic?: string;
  readonly start: bigint;
  readonly end: bigint;
  readonly outcome: Outcome;
  /** A skip's or a todo's reason, or why it was cancelled. */
  readonly reason?: string;
  readonly message?: string;
  readonly expected?: string;
  readonly actual?: string;
  /** From 1. */
  readonly attempt: number;
  /** Its steps, in the order they ended; absent where it has none (0001-test FR-002). */
  readonly steps?: readonly TestStep[];
};

One test as it ran once (FR-001): times in nanoseconds since the Unix epoch; text already redacted and cut.

Installationtypeno test imports it
type Installation = {
  readonly runtime: string;
  readonly runtimeVersion: string;
  readonly os: string;
  readonly arch: string;
  readonly revision?: string;
  readonly lockfileSha256?: string;
  readonly pinsSha256?: string;
  readonly role: string;
};

The installation record: the resource of a test process or of the run (FR-003).

Reviewtypeno test imports it
type Review = { readonly requirement: string; readonly reason: string };

A requirement declared proven by review rather than by a test, with the reason.

RunnerOutcometypeno test imports it
type RunnerOutcome =
  { readonly kind: "exited"; readonly runtime: string; readonly code: number } | { readonly kind: "missing"; readonly runtime: string };

What one runner did: the runtime, its exit status, or that it is not installed.

TEST_SCOPEconstno test imports it
const TEST_SCOPE: Scope

The instrumentation scope of test records.

TEST_SPANconstno test imports it
const TEST_SPAN: "test"

The name of a test's span.

STEP_SPANconst
const STEP_SPAN: "test step"

The name of a step's span, a child of its test's span or of the step it is in (0001-test FR-002).

Imported by 15 tests of the run (12 passed, 0 failed): the library's results

RUN_SPANconstno test imports it
const RUN_SPAN: "test run"

The name of a run's span.

EVENTSconstno test imports it
const EVENTS: { readonly case: "ifc.test.case"; readonly start: "ifc.test.start"; readonly review: "ifc.test.review"; readonly runner: "ifc.test.runner"; readonly warning: "ifc.test.warning"; }

The event names of the log records of a test record.

TEXT_LIMITconstno test imports it
const TEXT_LIMIT: 4096

The most characters a message, an expected or an actual value keeps (FR-001).

cutfunctionno test imports it
function cut(text: string): string

A text cut to TEXT_LIMIT characters (without splitting a surrogate pair).

testIdfunctionno test imports it
function testId(file: string, name: string): Promise<Sha256>

A test's id: the SHA-256 of its file, a NUL and its name (FR-001).

recordPathfunctionno test imports it
function recordPath(path: string): string

A file path as a record names it: \ as /, without a leading ./.

isPersonAttributefunctionno test imports it
function isPersonAttribute(key: string): boolean

Whether a resource attribute names a person or a machine of one, and is refused (FR-003): user.*, enduser.*, host.name.

hostArchfunctionno test imports it
function hostArch(arch: string): string

A machine's architecture as OpenTelemetry's host.arch names it (amd64, arm64, ...).

installationFromEnvfunctionno test imports it
function installationFromEnv(self: { readonly runtime: string; readonly runtimeVersion: string; readonly os: string; readonly arch: string; }, env: Readonly<Record<string, string | undefined>>): { readonly installation: Installation; readonly warnings: readonly string[]; }

The installation record of a process, from what the runtime says of itself and the environment given as a record (the kernel reads no environment): IFC_TEST_REVISION or GITHUB_SHA, IFC_TEST_LOCKFILE_SHA256, IFC_TEST_PINS_SHA256, and IFC_TEST_ROLE (ci where CI is set, else developer). A role that is not a role (an e-mail address, a name with capitals or spaces) is refused with a warning and the default used, and so is a digest that is not a SHA-256.

installationResourcefunctionno test imports it
function installationResource(i: Installation): Resource

The resource of an installation record (FR-003).

attributeTextfunctionno test imports it
function attributeText(a: Readonly<Record<string, AttributeValue>>, key: string): string | undefined

An attribute's value where it is a string.

attributeTextsfunctionno test imports it
function attributeTexts(a: Readonly<Record<string, AttributeValue>>, key: string): readonly string[]

An attribute's value where it is an array of strings, else none.

attributeIntegerfunctionno test imports it
function attributeInteger(a: Readonly<Record<string, AttributeValue>>, key: string): number | undefined

An attribute's value where it is an integer.

installationOffunction
function installationOf(r: Resource): Installation

The installation record a resource holds; unknown fields are unknown.

Imported by 15 tests of the run (12 passed, 0 failed): the library's results

TestContexttypeno test imports it
type TestContext = { readonly traceId: TraceId; readonly spanId: SpanId; readonly parentSpanId?: SpanId };

Where a test's signals belong: the trace, the test's own span id, and the run's span as its parent where there is one.

stepPathsfunctionno test imports it
function stepPaths(steps: readonly TestStep[], within?: readonly string[]): string[]

The steps of a test, each by its path (a step within another is outer > inner), in their order with each step's own steps first.

testStepSpanfunctionno test imports it
function testStepSpan(step: TestStep, test: Pick<TestCase, "file" | "id" | "name">, ctx: TestContext): SpanData

A step as its test step span (0001-test FR-002): a child of ctx.parentSpanId (its test's span or the step it is in), with its test's id and file, its name, outcome and a failure's message, and an error status where it failed.

testCaseSpanfunctionno test imports it
function testCaseSpan(c: TestCase, ctx: TestContext): SpanData

A test as its test span (FR-002): the conventions, the ifc.test.* attributes, and an error status where it failed or was cancelled.

testCaseLogfunctionno test imports it
function testCaseLog(c: TestCase, ctx: TestContext): LogRecord

A test as its log record (FR-002): the same attributes, the name as its body, at ERROR where it failed or was cancelled.

testStartLogfunctionno test imports it
function testStartLog(c: Omit<TestCase, "end" | "outcome">, ctx: TestContext): LogRecord

The mark a test leaves as it starts (FR-004): its span id, so that a test whose span never comes is closed as cancelled.

reviewLogfunctionno test imports it
function reviewLog(r: Review, time: bigint, traceId?: TraceId | undefined): LogRecord

A review declared in a test file or a list (ifc.test.review).

warningLogfunctionno test imports it
function warningLog(message: string, time: bigint): LogRecord

A warning of a test process or a run (ifc.test.warning), such as a role refused.

runnerLogfunctionno test imports it
function runnerLog(r: RunnerOutcome, time: bigint): LogRecord

What one runner did (ifc.test.runner).

recordLinefunctionno test imports it
function recordLine(resource: Resource, signal: { readonly spans: readonly SpanData[]; } | { readonly records: readonly LogRecord[]; }): string

One line of IFC_TEST_RECORD: an OTLP JSON traces or logs export request of one resource, without a line feed (FR-004).

RecordErrortypeno test imports it
type RecordError =
  | { readonly kind: "json"; readonly line: number; readonly reason: string }
  | { readonly kind: "otlp"; readonly line: number; readonly error: DecodeError }
  | { readonly kind: "person"; readonly key: string }
  | { readonly kind: "test"; readonly spanId: string; readonly reason: string }
  | { readonly kind: "no-run"; readonly reason: string };

Why the lines of a test record or a run's record could not be read.

formatRecordErrorfunctionno test imports it
function formatRecordError(e: RecordError): string

A record error as one sentence.

Signalstypeno test imports it
type Signals = { readonly traces: readonly TraceBatch[]; readonly logs: readonly LogBatch[] };

The signals of one or more export requests.

decodeRecordDocumentfunctionno test imports it
function decodeRecordDocument(value: unknown, line: number): Result<Signals, RecordError>

The traces and logs of one OTLP JSON document (an export request, or a run's record holding both), as line line.

decodeLinesfunction
function decodeLines(input: string): Result<Signals & { readonly warnings: readonly string[]; }, RecordError>

The signals of IFC_TEST_RECORD's lines, each an OTLP JSON export request (FR-004). A last line cut short (a process killed while it wrote) is left out with a warning; any other line that is not one is an error.

Imported by 15 tests of the run (12 passed, 0 failed): the library's results

mergeBatchesfunctionno test imports it
function mergeBatches(signals: Signals): Signals

Batches of one resource and scope joined into one, in the order they first came.

closeOrphansfunctionno test imports it
function closeOrphans(signals: Signals, at: bigint): Signals

The traces with a cancelled test span for each test that started and has no span (FR-004): its process died, or exited, before the test ended. Its end is at, the time the run saw its process end.

Titles, headings, exports, tables, columns and requirements. Esc closes.