IF Commons GitHub

test-report

Test evidence (commons 0008-test-evidence): the record of a test run as OpenTelemetry and every report rendered from it, purely.

  • Capture (FR-001 to FR-001) is ./test-record.ts: the model of a test case and an installation, and the signals a test process writes.
  • The run (FR-002 to FR-004): the exit rule (judge), the run's test run span and the one OTLP JSON document of a run (assembleRecord), read back only through the OTLP decoders of 0006-signals FR-004 into a typed model (readRecord). The requirement lists a run is held to: ids one per line, or a spec-kit enforcement register's rows of mechanism check (parseRequirementList), and requirements declared reviewed (parseReviewList).
  • Reports (FR-005 to FR-007): TAP version 14, JUnit XML, CTRF JSON, SARIF 2.1.0, OSCAL 1.2.3 assessment results and a completion report in Markdown and in self-contained HTML. Each is a function of the model alone (and, for the completion report, of the signatures the ledger holds), with no clock: rendering a record twice gives the same bytes, and the counts of each agree with the model's.
  • Evidence (FR-008 to FR-010): the ledger entries of a run and of a signature with its meaning and role, the in-toto Test Result statement of a run, and verifyEvidence, which checks a record's digest against a ledger, the ledger's chain, each envelope given and, where asked, a signature of meaning reviewed.
import { readRecord, tap14 } from "./test-report.ts";
// const run = readRecord(await readTextFile("out/4bf92f3577b34da6a3ce929d0e0e4736.otlp.json"));
// if (run.kind === "ok") console.log(tap14(run.value));

Source: commons/quality/src/test-report.ts

20 of 96 exports are imported by a test of the run. A test that imports an export may not call it or check what it returns.

testReportnamespace
namespace testReport
testReport.RunTesttypeno test imports it
type RunTest = TestCase & { readonly runtime: string; readonly spanId: string };

A test of a run: the case, the runtime it ran on (its process's process.runtime.name) and the id of its span.

testReport.RequirementStatetypeno test imports it
type RequirementState = "proven" | "failing" | "skipped" | "reviewed" | "unproven";

How far a requirement is met by a run.

testReport.RequirementTracetypeno test imports it
type RequirementTrace = {
  readonly id: string;
  /** Whether it is in the list the run was held to. */
  readonly required: boolean;
  readonly risk: Risk;
  readonly state: RequirementState;
  readonly tests: readonly { readonly id: string; readonly name: string; readonly runtime: string; readonly outcome: Outcome }[];
  readonly reviews: readonly string[];
};

One requirement of a run: the tests that prove it, the reviews of it, its risk and its state.

testReport.Countstypeno test imports it
type Counts = {
  readonly tests: number;
  readonly passed: number;
  readonly failed: number;
  readonly skipped: number;
  readonly todo: number;
  readonly cancelled: number;
};

The counts of a run (FR-007).

testReport.RunStatustypeno test imports it
type RunStatus = "success" | "failure";

A run's status, as test.suite.run.status says it.

testReport.TestRuntypeno test imports it
type TestRun = { readonly traceId: string; readonly spanId: string; readonly start: bigint; readonly end: bigint… (990 characters)
type TestRun = {
  readonly traceId: string;
  readonly spanId: string;
  readonly start: bigint;
  readonly end: bigint;
  readonly status: RunStatus;
  /** Why it failed, one sentence each; empty where it passed. */
  readonly failures: readonly string[];
  /** The requirements of the list neither proven nor reviewed. */
  readonly unproven: readonly string[];
  /** The list the run was held to. */
  readonly required: readonly string[];
  /** The installation of the process that ran the runners. */
  readonly runner: Installation;
  /** The installation of each runtime the tests ran on, by runtime. */
  readonly installations: readonly Installation[];
  readonly runners: readonly RunnerOutcome[];
  /** In a stable order: runtime, file, name, attempt, start. */
  readonly tests: readonly RunTest[];
  readonly reviews: readonly Review[];
  readonly warnings: readonly string[];
  /** By id. */
  readonly requirements: readonly RequirementTrace[];
  readonly counts: Counts;
};

A run's record as a typed model (FR-005).

testReport.testsOffunctionno test imports it
function testsOf(signals: Signals): Result<readonly RunTest[], RecordError>

The tests of some signals, in a stable order, each with its steps (the test step spans under its span).

testReport.reviewsOffunctionno test imports it
function reviewsOf(signals: Signals): readonly Review[]

The reviews some signals declare, each once, by requirement and reason.

testReport.countsOffunctionno test imports it
function countsOf(tests: readonly { readonly outcome: Outcome; }[]): Counts

The counts of some tests.

testReport.requirementsOffunctionno test imports it
function requirementsOf(tests: readonly RunTest[], reviews: readonly Review[], required: readonly string[]): readonly RequirementTrace[]

Each requirement named by a test, a review or the list, with its state (FR-003, FR-006).

testReport.Judgementtypeno test imports it
type Judgement = { readonly status: RunStatus; readonly failures: readonly string[]; readonly unproven: readonly string[] };

What the exit rule says of a run (FR-003).

testReport.judgefunctionno test imports it
function judge(input: { readonly tests: readonly RunTest[]; readonly runners: readonly RunnerOutcome[]; readonly required: readonly string[]; readonly reviews: readonly Review[]; }): Judgement

The exit rule (FR-003): a run fails when a test failed, when a runner exited non-zero or is not installed, when a test was cancelled, or when a requirement of the list is neither proven by a test of the run nor declared reviewed.

testReport.RunInputtypeno test imports it
type RunInput = { readonly traceId: TraceId; readonly spanId: SpanId; readonly start: bigint; readonly end: bigi… (422 characters)
type RunInput = {
  readonly traceId: TraceId;
  readonly spanId: SpanId;
  readonly start: bigint;
  readonly end: bigint;
  readonly runner: Installation;
  readonly signals: Signals;
  readonly runners: readonly RunnerOutcome[];
  readonly required: readonly string[];
  /** Reviews from a list rather than from the tests' own code. */
  readonly reviews?: readonly Review[];
  readonly warnings?: readonly string[];
};

What a run is assembled from: its ids and times, the installation that ran it, the signals its processes wrote, the runners, the list and the reviews of a list.

testReport.assembleRecordfunctionno test imports it
function assembleRecord(input: RunInput): Result<{ readonly record: JsonObject; readonly judgement: Judgement; }, RecordError>

The one OTLP JSON document of a run (FR-002, FR-003): every process's signals, a cancelled span for each test that never ended, and the run's test run span with test.suite.run.status and ifc.test.unproven, its reviews, runners and warnings; with the judgement.

testReport.recordTextfunctionno test imports it
function recordText(record: JsonObject): string

A record as the bytes kept: its JSON and a line feed (FR-004).

testReport.readRecordfunctionno test imports it
function readRecord(input: unknown): Result<TestRun, RecordError>

A run's record read into the typed model (FR-004, FR-005), only through the OTLP decoders of 0006-signals: the run's span, its installation records (a resource that names a person is refused), its tests, its reviews and its requirements. input is the record's text, or its JSON already parsed.

testReport.parseRequirementListfunctionno test imports it
function parseRequirementList(input: string, spec?: string | undefined): readonly string[]

The requirements a run is held to (FR-003): one id per line (0008-test-evidence FR-001; # begins a comment), or a spec-kit enforcement register, whose rows (requirement, a tab, mechanism, ...) of mechanism check are taken. spec keeps only the ids that begin with it (0005-typescript-practice). Each id once, in the order given.

testReport.parseReviewListfunctionno test imports it
function parseReviewList(input: string): readonly Review[]

Requirements declared reviewed in a list (FR-003): a line of the requirement, a tab and the reason (# begins a comment). A line without a reason is left out.

testReport.isoTimefunctionno test imports it
function isoTime(nanos: bigint): string

Nanoseconds since the Unix epoch as ISO 8601 text in UTC with milliseconds.

testReport.secondsfunctionno test imports it
function seconds(start: bigint, end: bigint): string

The seconds between two times in nanoseconds, with three decimals.

testReport.tap14functionno test imports it
function tap14(run: TestRun): string

The run as TAP version 14 (FR-005): a plan, one subtest per runtime and file, each test ok or not ok numbered, # SKIP and # TODO with their reasons, and a YAML diagnostic block for a failure or a cancellation; the counts as closing comments. A test's steps are a subtest of their own before its line (0001-test FR-002).

testReport.junitXmlfunctionno test imports it
function junitXml(run: TestRun): string

The run as JUnit XML (FR-005): one testsuite per file and runtime with the installation as properties, a testcase with its classname (the file) and its time in seconds, the requirements, risk, id and steps (step, as pass outer > inner) as its properties, and failure, error (cancelled) or skipped (skip and todo).

testReport.ctrfJsonfunctionno test imports it
function ctrfJson(run: TestRun): string

The run as CTRF JSON (FR-005): results.tool, summary and tests, with the requirements, risk, id and outcome under extra.

testReport.NO_REQUIREMENT_RULEconstno test imports it
const NO_REQUIREMENT_RULE: "ifc.test.no-requirement"

The rule of a failed test that proves no requirement.

testReport.sariffunctionno test imports it
function sarif(run: TestRun): string

The run as SARIF 2.1.0 (FR-005): each failed or cancelled test a result at its file, its first requirement as its rule; the counts as the run's properties.

testReport.OSCAL_NAMESPACE_URIconstno test imports it
const OSCAL_NAMESPACE_URI: "https://github.com/intellectual-frontiers/.github/tree/main/commons/spec-kit/specs/0008-test-evidence"

The URI the UUIDs and properties of a test run's OSCAL document are named in.

testReport.ControlReftypeno test imports it
type ControlRef = { readonly catalog: string; readonly id: string };

A requirement as a catalog control: its specification and its number (0008-test-evidence FR-001).

testReport.requirementReffunctionno test imports it
function requirementRef(id: string): ControlRef

A requirement as a catalog control: its specification and its number (0008-test-evidence FR-001).

testReport.oscalControlIdfunctionno test imports it
function oscalControlId(r: ControlRef): string

A requirement as an OSCAL token: req- and its words in lower case joined by -.

testReport.oscalAssessmentResultsfunctionno test imports it
function oscalAssessmentResults(run: TestRun, options?: { readonly version?: string | undefined; readonly organization?: string | undefined; }): Promise<string>

The run as OSCAL 1.2.3 assessment results (FR-005): one observation per test (its subject the runtime it ran on) and one finding per requirement, satisfied or not by the outcomes of the tests that prove it, and one for each test that proves none. Every uuid is a version 5 UUID of a name in the run, so the same run gives the same bytes.

testReport.Meaningtypeno test imports it
type Meaning = "executed" | "reviewed" | "approved";

The meaning of a signature on a run (FR-008).

testReport.MEANINGSconstno test imports it
const MEANINGS: readonly Meaning[]

Every meaning.

testReport.isMeaningfunctionno test imports it
function isMeaning(text: string): text is Meaning

Whether a text is a meaning.

testReport.RUN_KINDconstno test imports it
const RUN_KIND: "test-run"

The ledger kind of a run's record.

testReport.signatureKindfunctionno test imports it
function signatureKind(meaning: Meaning, role: string): string

The ledger kind of a signature on a run: test-signature:<meaning>:<role>.

testReport.runEntryfunctionno test imports it
function runEntry(traceId: string, digest: Sha256, at: Instant): LedgerRecord

The ledger entry of a run (FR-008): kind test-run, the run's trace id as its subject and the record's digest as its evidence.

testReport.signatureEntryfunctionno test imports it
function signatureEntry(meaning: Meaning, role: string, recordDigest: Sha256, envelopeDigest: Sha256, at: Instant): LedgerRecord

The ledger entry of a signature (FR-008): its meaning and role in its kind, the record's digest as its subject, the envelope's digest as its evidence.

testReport.SignatureOnRuntypeno test imports it
type SignatureOnRun = {
  readonly seq: number;
  readonly meaning: Meaning;
  readonly role: string;
  readonly at: string;
  readonly envelope: string;
};

A signature on a run as the ledger holds it.

testReport.signaturesOffunctionno test imports it
function signaturesOf(ledger: readonly LedgerEntry[], recordDigest: string): readonly SignatureOnRun[]

The signatures the ledger holds on the record of a digest, in the ledger's order.

testReport.TEST_RESULT_PREDICATEconstno test imports it
const TEST_RESULT_PREDICATE: "https://in-toto.io/attestation/test-result/v0.1"

The predicate type of an in-toto Test Result.

testReport.TestResultPredicatetypeno test imports it
type TestResultPredicate = {
  readonly result: "PASSED" | "WARNED" | "FAILED";
  readonly configuration: readonly ResourceDescriptor[];
  readonly url?: string;
  readonly passedTests: readonly string[];
  readonly warnedTests: readonly string[];
  readonly failedTests: readonly string[];
};

An in-toto Test Result predicate (v0.1).

testReport.testResultStatementfunctionno test imports it
function testResultStatement(run: TestRun, record: { readonly name: string; readonly digest: Sha256; readonly url?: string | undefined; }): Statement<TestResultPredicate>

The in-toto Statement v1 of a run (FR-008): its subject the record (name, the SHA-256 of its bytes), its predicate the Test Result: FAILED where the run failed, WARNED where it passed with tests skipped or todo, else PASSED; the configuration the record and the lockfile and pins it names.

testReport.VerifyProblemtypeno test imports it
type VerifyProblem = | { readonly kind: "not-in-ledger"; readonly digest: string } | { readonly kind: "ledger-broken… (422 characters)
type VerifyProblem =
  | { readonly kind: "not-in-ledger"; readonly digest: string }
  | { readonly kind: "ledger-broken"; readonly break: LedgerBreak }
  | { readonly kind: "envelope"; readonly name: string; readonly reason: string }
  | { readonly kind: "envelope-subject"; readonly name: string; readonly digest: string }
  | { readonly kind: "no-key" }
  | { readonly kind: "review-missing"; readonly digest: string };

Why ifc test verify refused a record (FR-009, FR-010).

testReport.formatVerifyProblemfunctionno test imports it
function formatVerifyProblem(p: VerifyProblem): string

A verification problem as one sentence.

testReport.Verificationtypeno test imports it
type Verification = { readonly seq: number; readonly signatures: readonly SignatureOnRun[] };

What verification found where nothing is wrong: the ledger's head and the signatures on the record.

testReport.verifyEvidencefunctionno test imports it
function verifyEvidence(input: { readonly recordDigest: Sha256; readonly ledger: readonly LedgerEntry[]; readonly envelopes?: readonly { readonly name: string; readonly envelope: DsseEnvelope; readonly digest: Sha256; }[] | undefined; readonly keys?: readonly DsseKey[] | undefined; readonly requireReview?: boolean | undefined; }): Promise<Result<Verification, readonly VerifyProblem[]>>

A record's evidence checked (FR-009, FR-010): its digest is in the ledger as a test-run, the ledger's chain is whole, each envelope given verifies with the keys given and names the digest, and, with requireReview, the ledger holds a signature of meaning reviewed on it (one of the envelopes given, where any are). Every problem is named, not only the first.

testReport.CompletionOptionstypeno test imports it
type CompletionOptions = { readonly digest?: string; readonly signatures?: readonly SignatureOnRun[] };

What a completion report adds to the record: the record's digest and the signatures the ledger holds on it.

testReport.completionMarkdownfunctionno test imports it
function completionMarkdown(run: TestRun, options?: CompletionOptions): string

The test completion report in Markdown (FR-006).

testReport.completionHtmlfunctionno test imports it
function completionHtml(run: TestRun, options?: CompletionOptions): string

The test completion report as one HTML document that needs nothing outside itself (FR-006).

testReport.runIdsfunctionno test imports it
function runIds(traceId: string, spanId: string): Result<{ readonly traceId: TraceId; readonly spanId: SpanId; }, string>

Parses a trace id and a span id (for a runner that holds them as text).

RunTesttype
type RunTest = TestCase & { readonly runtime: string; readonly spanId: string };

A test of a run: the case, the runtime it ran on (its process's process.runtime.name) and the id of its span.

Imported by 15 tests of the run (15 passed, 0 failed): the library's results

RequirementStatetypeno test imports it
type RequirementState = "proven" | "failing" | "skipped" | "reviewed" | "unproven";

How far a requirement is met by a run.

RequirementTracetypeno test imports it
type RequirementTrace = {
  readonly id: string;
  /** Whether it is in the list the run was held to. */
  readonly required: boolean;
  readonly risk: Risk;
  readonly state: RequirementState;
  readonly tests: readonly { readonly id: string; readonly name: string; readonly runtime: string; readonly outcome: Outcome }[];
  readonly reviews: readonly string[];
};

One requirement of a run: the tests that prove it, the reviews of it, its risk and its state.

Countstypeno test imports it
type Counts = {
  readonly tests: number;
  readonly passed: number;
  readonly failed: number;
  readonly skipped: number;
  readonly todo: number;
  readonly cancelled: number;
};

The counts of a run (FR-007).

RunStatustypeno test imports it
type RunStatus = "success" | "failure";

A run's status, as test.suite.run.status says it.

TestRuntype
type TestRun = { readonly traceId: string; readonly spanId: string; readonly start: bigint; readonly end: bigint… (990 characters)
type TestRun = {
  readonly traceId: string;
  readonly spanId: string;
  readonly start: bigint;
  readonly end: bigint;
  readonly status: RunStatus;
  /** Why it failed, one sentence each; empty where it passed. */
  readonly failures: readonly string[];
  /** The requirements of the list neither proven nor reviewed. */
  readonly unproven: readonly string[];
  /** The list the run was held to. */
  readonly required: readonly string[];
  /** The installation of the process that ran the runners. */
  readonly runner: Installation;
  /** The installation of each runtime the tests ran on, by runtime. */
  readonly installations: readonly Installation[];
  readonly runners: readonly RunnerOutcome[];
  /** In a stable order: runtime, file, name, attempt, start. */
  readonly tests: readonly RunTest[];
  readonly reviews: readonly Review[];
  readonly warnings: readonly string[];
  /** By id. */
  readonly requirements: readonly RequirementTrace[];
  readonly counts: Counts;
};

A run's record as a typed model (FR-005).

Imported by 15 tests of the run (15 passed, 0 failed): the library's results

reviewsOffunctionno test imports it
function reviewsOf(signals: Signals): readonly Review[]

The reviews some signals declare, each once, by requirement and reason.

countsOffunctionno test imports it
function countsOf(tests: readonly { readonly outcome: Outcome; }[]): Counts

The counts of some tests.

judgefunction
function judge(input: { readonly tests: readonly RunTest[]; readonly runners: readonly RunnerOutcome[]; readonly required: readonly string[]; readonly reviews: readonly Review[]; }): Judgement

The exit rule (FR-003): a run fails when a test failed, when a runner exited non-zero or is not installed, when a test was cancelled, or when a requirement of the list is neither proven by a test of the run nor declared reviewed.

Imported by 32 tests of the run (32 passed, 0 failed): the library's results

RunInputtypeno test imports it
type RunInput = { readonly traceId: TraceId; readonly spanId: SpanId; readonly start: bigint; readonly end: bigi… (422 characters)
type RunInput = {
  readonly traceId: TraceId;
  readonly spanId: SpanId;
  readonly start: bigint;
  readonly end: bigint;
  readonly runner: Installation;
  readonly signals: Signals;
  readonly runners: readonly RunnerOutcome[];
  readonly required: readonly string[];
  /** Reviews from a list rather than from the tests' own code. */
  readonly reviews?: readonly Review[];
  readonly warnings?: readonly string[];
};

What a run is assembled from: its ids and times, the installation that ran it, the signals its processes wrote, the runners, the list and the reviews of a list.

assembleRecordfunction
function assembleRecord(input: RunInput): Result<{ readonly record: JsonObject; readonly judgement: Judgement; }, RecordError>

The one OTLP JSON document of a run (FR-002, FR-003): every process's signals, a cancelled span for each test that never ended, and the run's test run span with test.suite.run.status and ifc.test.unproven, its reviews, runners and warnings; with the judgement.

Imported by 15 tests of the run (15 passed, 0 failed): the library's results

recordTextfunction
function recordText(record: JsonObject): string

A record as the bytes kept: its JSON and a line feed (FR-004).

Imported by 15 tests of the run (15 passed, 0 failed): the library's results

readRecordfunction
function readRecord(input: unknown): Result<TestRun, RecordError>

A run's record read into the typed model (FR-004, FR-005), only through the OTLP decoders of 0006-signals: the run's span, its installation records (a resource that names a person is refused), its tests, its reviews and its requirements. input is the record's text, or its JSON already parsed.

Imported by 15 tests of the run (15 passed, 0 failed): the library's results

parseRequirementListfunction
function parseRequirementList(input: string, spec?: string | undefined): readonly string[]

The requirements a run is held to (FR-003): one id per line (0008-test-evidence FR-001; # begins a comment), or a spec-kit enforcement register, whose rows (requirement, a tab, mechanism, ...) of mechanism check are taken. spec keeps only the ids that begin with it (0005-typescript-practice). Each id once, in the order given.

Imported by 15 tests of the run (15 passed, 0 failed): the library's results

parseReviewListfunction
function parseReviewList(input: string): readonly Review[]

Requirements declared reviewed in a list (FR-003): a line of the requirement, a tab and the reason (# begins a comment). A line without a reason is left out.

Imported by 15 tests of the run (15 passed, 0 failed): the library's results

isoTimefunctionno test imports it
function isoTime(nanos: bigint): string

Nanoseconds since the Unix epoch as ISO 8601 text in UTC with milliseconds.

secondsfunctionno test imports it
function seconds(start: bigint, end: bigint): string

The seconds between two times in nanoseconds, with three decimals.

tap14function
function tap14(run: TestRun): string

The run as TAP version 14 (FR-005): a plan, one subtest per runtime and file, each test ok or not ok numbered, # SKIP and # TODO with their reasons, and a YAML diagnostic block for a failure or a cancellation; the counts as closing comments. A test's steps are a subtest of their own before its line (0001-test FR-002).

Imported by 15 tests of the run (15 passed, 0 failed): the library's results

junitXmlfunction
function junitXml(run: TestRun): string

The run as JUnit XML (FR-005): one testsuite per file and runtime with the installation as properties, a testcase with its classname (the file) and its time in seconds, the requirements, risk, id and steps (step, as pass outer > inner) as its properties, and failure, error (cancelled) or skipped (skip and todo).

Imported by 15 tests of the run (15 passed, 0 failed): the library's results

ctrfJsonfunction
function ctrfJson(run: TestRun): string

The run as CTRF JSON (FR-005): results.tool, summary and tests, with the requirements, risk, id and outcome under extra.

Imported by 15 tests of the run (15 passed, 0 failed): the library's results

NO_REQUIREMENT_RULEconstno test imports it
const NO_REQUIREMENT_RULE: "ifc.test.no-requirement"

The rule of a failed test that proves no requirement.

OSCAL_NAMESPACE_URIconstno test imports it
const OSCAL_NAMESPACE_URI: "https://github.com/intellectual-frontiers/.github/tree/main/commons/spec-kit/specs/0008-test-evidence"

The URI the UUIDs and properties of a test run's OSCAL document are named in.

ControlReftypeno test imports it
type ControlRef = { readonly catalog: string; readonly id: string };

A requirement as a catalog control: its specification and its number (0008-test-evidence FR-001).

requirementReffunctionno test imports it
function requirementRef(id: string): ControlRef

A requirement as a catalog control: its specification and its number (0008-test-evidence FR-001).

oscalControlIdfunctionno test imports it
function oscalControlId(r: ControlRef): string

A requirement as an OSCAL token: req- and its words in lower case joined by -.

oscalAssessmentResultsfunction
function oscalAssessmentResults(run: TestRun, options?: { readonly version?: string | undefined; readonly organization?: string | undefined; }): Promise<string>

The run as OSCAL 1.2.3 assessment results (FR-005): one observation per test (its subject the runtime it ran on) and one finding per requirement, satisfied or not by the outcomes of the tests that prove it, and one for each test that proves none. Every uuid is a version 5 UUID of a name in the run, so the same run gives the same bytes.

Imported by 15 tests of the run (15 passed, 0 failed): the library's results

Meaningtypeno test imports it
type Meaning = "executed" | "reviewed" | "approved";

The meaning of a signature on a run (FR-008).

MEANINGSconstno test imports it
const MEANINGS: readonly Meaning[]

Every meaning.

isMeaningfunctionno test imports it
function isMeaning(text: string): text is Meaning

Whether a text is a meaning.

RUN_KINDconstno test imports it
const RUN_KIND: "test-run"

The ledger kind of a run's record.

signatureKindfunctionno test imports it
function signatureKind(meaning: Meaning, role: string): string

The ledger kind of a signature on a run: test-signature:<meaning>:<role>.

runEntryfunction
function runEntry(traceId: string, digest: Sha256, at: Instant): LedgerRecord

The ledger entry of a run (FR-008): kind test-run, the run's trace id as its subject and the record's digest as its evidence.

Imported by 15 tests of the run (15 passed, 0 failed): the library's results

signatureEntryfunction
function signatureEntry(meaning: Meaning, role: string, recordDigest: Sha256, envelopeDigest: Sha256, at: Instant): LedgerRecord

The ledger entry of a signature (FR-008): its meaning and role in its kind, the record's digest as its subject, the envelope's digest as its evidence.

Imported by 15 tests of the run (15 passed, 0 failed): the library's results

SignatureOnRuntypeno test imports it
type SignatureOnRun = {
  readonly seq: number;
  readonly meaning: Meaning;
  readonly role: string;
  readonly at: string;
  readonly envelope: string;
};

A signature on a run as the ledger holds it.

signaturesOffunction
function signaturesOf(ledger: readonly LedgerEntry[], recordDigest: string): readonly SignatureOnRun[]

The signatures the ledger holds on the record of a digest, in the ledger's order.

Imported by 15 tests of the run (15 passed, 0 failed): the library's results

TEST_RESULT_PREDICATEconst
const TEST_RESULT_PREDICATE: "https://in-toto.io/attestation/test-result/v0.1"

The predicate type of an in-toto Test Result.

Imported by 15 tests of the run (15 passed, 0 failed): the library's results

TestResultPredicatetypeno test imports it
type TestResultPredicate = {
  readonly result: "PASSED" | "WARNED" | "FAILED";
  readonly configuration: readonly ResourceDescriptor[];
  readonly url?: string;
  readonly passedTests: readonly string[];
  readonly warnedTests: readonly string[];
  readonly failedTests: readonly string[];
};

An in-toto Test Result predicate (v0.1).

testResultStatementfunction
function testResultStatement(run: TestRun, record: { readonly name: string; readonly digest: Sha256; readonly url?: string | undefined; }): Statement<TestResultPredicate>

The in-toto Statement v1 of a run (FR-008): its subject the record (name, the SHA-256 of its bytes), its predicate the Test Result: FAILED where the run failed, WARNED where it passed with tests skipped or todo, else PASSED; the configuration the record and the lockfile and pins it names.

Imported by 15 tests of the run (15 passed, 0 failed): the library's results

VerifyProblemtypeno test imports it
type VerifyProblem = | { readonly kind: "not-in-ledger"; readonly digest: string } | { readonly kind: "ledger-broken… (422 characters)
type VerifyProblem =
  | { readonly kind: "not-in-ledger"; readonly digest: string }
  | { readonly kind: "ledger-broken"; readonly break: LedgerBreak }
  | { readonly kind: "envelope"; readonly name: string; readonly reason: string }
  | { readonly kind: "envelope-subject"; readonly name: string; readonly digest: string }
  | { readonly kind: "no-key" }
  | { readonly kind: "review-missing"; readonly digest: string };

Why ifc test verify refused a record (FR-009, FR-010).

formatVerifyProblemfunctionno test imports it
function formatVerifyProblem(p: VerifyProblem): string

A verification problem as one sentence.

Verificationtypeno test imports it
type Verification = { readonly seq: number; readonly signatures: readonly SignatureOnRun[] };

What verification found where nothing is wrong: the ledger's head and the signatures on the record.

verifyEvidencefunction
function verifyEvidence(input: { readonly recordDigest: Sha256; readonly ledger: readonly LedgerEntry[]; readonly envelopes?: readonly { readonly name: string; readonly envelope: DsseEnvelope; readonly digest: Sha256; }[] | undefined; readonly keys?: readonly DsseKey[] | undefined; readonly requireReview?: boolean | undefined; }): Promise<Result<Verification, readonly VerifyProblem[]>>

A record's evidence checked (FR-009, FR-010): its digest is in the ledger as a test-run, the ledger's chain is whole, each envelope given verifies with the keys given and names the digest, and, with requireReview, the ledger holds a signature of meaning reviewed on it (one of the envelopes given, where any are). Every problem is named, not only the first.

Imported by 15 tests of the run (15 passed, 0 failed): the library's results

CompletionOptionstypeno test imports it
type CompletionOptions = { readonly digest?: string; readonly signatures?: readonly SignatureOnRun[] };

What a completion report adds to the record: the record's digest and the signatures the ledger holds on it.

completionMarkdownfunction
function completionMarkdown(run: TestRun, options?: CompletionOptions): string

The test completion report in Markdown (FR-006).

Imported by 15 tests of the run (15 passed, 0 failed): the library's results

completionHtmlfunction
function completionHtml(run: TestRun, options?: CompletionOptions): string

The test completion report as one HTML document that needs nothing outside itself (FR-006).

Imported by 15 tests of the run (15 passed, 0 failed): the library's results

runIdsfunctionno test imports it
function runIds(traceId: string, spanId: string): Result<{ readonly traceId: TraceId; readonly spanId: SpanId; }, string>

Parses a trace id and a span id (for a runner that holds them as text).

Titles, headings, exports, tables, columns and requirements. Esc closes.