Written to: 3 requirements: 3 proved, 0 partly, 0 failing, 0 with no test recorded. 0001-docs FR-022, 0001-docs FR-009, 0001-docs FR-027
The evidence the site shows beside the code (docs 0001-docs FR-009 to FR-011): the results of the tests, the issues kept as typed code, and the tickets of GitHub, read from files that CI (or a person) leaves in a folder, so that the same files give the same pages. The results are CTRF (the Common Test Report Format: <library>.ctrf.json, one file for each library, written by ifc test run --report ctrf), with the requirements each test proves in extra.requirements; meta.json says which commit they are of. The package reads the format and imports no library that makes it (0011-documentation FR-026). Nothing here runs a test or reaches a network: a missing folder is "no run recorded", and a file that cannot be read is a finding.
Source: commons/docs/src/evidence.ts
5 of 13 exports are imported by a test of the run. A test that imports an export may not call it or check what it returns.
Evidenceinterfaceno test imports itinterface Evidence
Everything the site knows of quality.
| Member | Type | What it is |
|---|
packageReport | PackageReport | undefined | What the package weighs, from the report of its packer (docs 0001-docs FR-032), where one is kept. |
usages | readonly ConsumerUsage[] | What each consumer repository reaches of the package, from usage.<consumer>.json. |
results | ReadonlyMap<string, LibraryResults> | |
run | RunMeta | undefined | |
issues | readonly TypedIssue[] | |
tickets | readonly Ticket[] | |
recorded | boolean | Whether a folder of results was given and read (a run is recorded), as opposed to none. |
status | ReadonlyMap<string, LibraryStatus> | What each library's job says of its tests, by library. |
build | BuildInfo | undefined | |
workflows | readonly WorkflowRuns[] | The workflows and their recent runs; none where runs.json is not there. |
runsError | string | undefined | Why the runs could not be read, where runs.json says so. |
history | readonly HistoryRun[] | The runs of the evidence that were kept (docs 0001-docs FR-016), oldest first. |
coverage | ReadonlyMap<string, LibraryCoverage> | The line coverage of each library whose tests were run with it (docs 0001-docs FR-022). |
mutation | ReadonlyMap<string, LibraryMutation> | The mutation run of each library that has one (docs 0001-docs FR-027). |
findings | readonly FindingsFile[] | The findings of the runs of the workflows, from the history branch, oldest first (docs 0001-docs FR-030). |
acceptances | { readonly accepted: readonly Acceptance[]; readonly problems: readonly string[]; } | The accepted risks of the repository and what was wrong with their files (docs 0001-docs FR-030). |
isOpenconstno test imports itconst isOpen: (i: Pick<IssueLike, "status">) => boolean
Whether an issue's status is one that is still to be dealt with.
LibraryResultsinterfaceno test imports itinterface LibraryResults
The results of one library's suite.
| Member | Type | What it is |
|---|
library | string | The library, by the name of its folder (repository for the tests of the repository itself). |
tests | readonly TestResult[] | |
passed | number | |
failed | number | |
skipped | number | |
other | number | Tests that were planned and not run (pending, todo, other). |
NONEconstno test imports itconst NONE: Evidence
No evidence: nothing recorded.
function parseCtrf(library: string, text: string, prefix: string): { readonly value: LibraryResults; } | { readonly error: string; }
The results of a CTRF file: its tests, or why it cannot be read.
| Parameter | What it is |
|---|
library | the library the file is of |
text | the file's text |
prefix | what is put before a test's path to make it a path from the repository's root |
Imported by 21 tests of the run (21 passed, 0 failed): the library's results
function parseTickets(text: string): { readonly value: readonly Ticket[]; } | { readonly error: string; }
The tickets of a snapshot of the GitHub issues of a repository (a JSON array as the API gives it; pull requests are left out).
| Parameter | What it is |
|---|
text | the snapshot |
Imported by 21 tests of the run (21 passed, 0 failed): the library's results
function readEvidence(root: string, source: EvidenceSource | undefined, found: Finding[], dirOf?: (library: string) => string): Evidence
The evidence the configuration names, read from its files.
| Parameter | What it is |
|---|
root | the repository's root, absolute |
source | what the configuration gives |
found | where findings are added |
dirOf | the folder of a library, from the repository's root (commons/core; "" for the repository itself) |
Imported by 24 tests of the run (24 passed, 0 failed): the library's results
function requirementsIn(text: string): string[]
The requirement keys a text names, as 0009-example FR-001.
Imported by 21 tests of the run (21 passed, 0 failed): the library's results
function standing(tests: readonly TestResult[]): "failing" | "none" | "partly" | "proved"
What the tests of a requirement come to: not run (none recorded), failing, proved (every one passed) or partly (some skipped or planned).
Imported by 21 tests of the run (21 passed, 0 failed): the library's results
TestResultinterfaceno test imports itinterface TestResult
One test of a run.
| Member | Type | What it is |
|---|
name | string | Its name, as proves makes it: the requirements, a colon and what it says. |
status | TestStatus | |
durationMs | number | |
file | string | Its file, from the repository's root. |
requirements | readonly string[] | The requirements it proves, as 0009-example FR-001. |
risk | string | |
runtime | string | |
retries | number | |
message | string | undefined | Why it failed, cut short; for a failed test only. |
TestStatustypeno test imports ittype TestStatus = "passed" | "failed" | "skipped" | "pending" | "other";
How a test ended, in CTRF's words.
Ticketinterfaceno test imports itinterface Ticket
A ticket of GitHub, as its API gives it, read from a snapshot.
| Member | Type | What it is |
|---|
number | number | |
title | string | |
state | "closed" | "open" | |
url | string | |
labels | readonly string[] | |
body | string | |
libraries | readonly string[] | The libraries it is about (a label lib:<name>, or a path of commons/<name>/ in its words). |
requirements | readonly string[] | The requirements it names, as 0009-example FR-001. |
TypedIssueinterfaceno test imports itinterface TypedIssue extends IssueLike
An issue kept as typed code, with the library it is of.
| Member | Type | What it is |
|---|
library | string | |
scope | string | undefined | The path of its set in the library, where the library has more than one. |
id | number | |
slug | string | |
title | string | |
summary | string | |
severity | string | |
priority | string | |
goals? | readonly string[] | |
status | { readonly kind: string; readonly reason?: string | undefined; readonly by?: string | undefined; readonly of?: number | undefined; readonly until?: string | undefined; readonly proof?: readonly { readonly [k: string]: unknown; readonly kind: string; }[] | undefined; } | |
size? | string | |
cites? | readonly string[] | The requirements it is about, as 0043-sql-invariants FR-002. |
evidence? | readonly { readonly [k: string]: unknown; readonly kind: string; }[] | |
links? | readonly { readonly kind: string; readonly to: number; }[] | |
raised | string | |