IF Commons GitHub

merge/layout

Written to: 7 requirements: 7 proved, 0 partly, 0 failing, 0 with no test recorded. 0009-anyfs-merge-physical FR-001, 0009-anyfs-merge-physical FR-002, 0009-anyfs-merge-physical FR-003, 0009-anyfs-merge-physical FR-004, 0009-anyfs-merge-physical FR-005, 0009-anyfs-merge-physical FR-006 and 1 more

The laws a database keeps so that its rows can be dumped and replayed into another and be merged by the replaying alone, as typed code: which tables are the mergeable set (the version rows, whose primary key is the id and the sum with a conflict rule that ignores a second insert, and the blobs, whose primary key is the SHA-256), which are derived (views and caches made only from those rows, never dumped as a source) and which are local (the identity of the replica, whom it trusts, the epochs it holds, its caches: never in a dump and never written by an import); the procedure that makes a dump and replays it; the epoch that lets rows be collected without a replay of an old dump bringing them back; and the identity of a replica, which a copy of a database does not keep.

import { isMergeableTable, dumpCommand } from "./layout.ts";
isMergeableTable("anyfs_versions"); // true
dumpCommand("x.db"); // 'sqlite3 x.db ".dump --data-only anyfs_versions anyfs_blobs"'

Source: commons/anyfs/src/merge/layout.ts

15 of 18 exports are imported by a test of the run. A test that imports an export may not call it or check what it returns.

MERGEABLE_TABLESconst
const MERGEABLE_TABLES: readonly ["anyfs_versions", "anyfs_blobs"]

The tables a dump carries: the version rows (primary key id with sum, on conflict ignore, without a rowid) and the blobs (primary key sha, on conflict ignore). A row of them is immutable and an equal key is an equal row.

Imported by 5 tests of the run (5 passed, 0 failed): the library's results

DERIVED_TABLESconst
const DERIVED_TABLES: readonly ["anyfs_heads", "anyfs_names", "anyfs_siblings", "anyfs_vector"]

The tables made from the rows and nothing else: the heads, the names, the siblings and the vector. Views, or caches a rebuild makes again, and never dumped as a source.

Imported by 5 tests of the run (5 passed, 0 failed): the library's results

LOCAL_TABLESconst
const LOCAL_TABLES: readonly ["anyfs_meta", "anyfs_trust", "anyfs_epochs", "anyfs_cache"]

The tables that belong to one replica: its identity, the trust it gives, the epochs it holds and its caches. An import never writes them.

Imported by 5 tests of the run (5 passed, 0 failed): the library's results

TableKindtypeno test imports it
type TableKind = "mergeable" | "derived" | "local";

The three kinds of table of a merge database.

kindOfTablefunction
function kindOfTable(name: string): TableKind | undefined

What kind of table a table is, or nothing where it is none of ours.

Imported by 5 tests of the run (5 passed, 0 failed): the library's results

isMergeableTableconst
const isMergeableTable: (name: string) => boolean

Whether a table is one a dump carries.

Imported by 5 tests of the run (5 passed, 0 failed): the library's results

VERSION_COLUMNSconst
const VERSION_COLUMNS: readonly ["id", "sum", "replica", "seq", "path", "parent", "merged", "kind", "blob"]

The columns of the version table, in order: no autoincrement, no column unique but the key, nothing that depends on the order of an import. merged is the ids merged with, as a list of text.

Imported by 5 tests of the run (5 passed, 0 failed): the library's results

BLOB_COLUMNSconst
const BLOB_COLUMNS: readonly ["sha", "bytes"]

The columns of the blob table: the digest and the bytes.

Imported by 5 tests of the run (5 passed, 0 failed): the library's results

dumpCommandconst
const dumpCommand: (database: string) => string

The command that makes the dump of a database: the data of the mergeable tables alone, as one INSERT for each row, with no schema.

Imported by 5 tests of the run (5 passed, 0 failed): the library's results

RESTORE_STEPSconst
const RESTORE_STEPS: readonly ["create-schema", "replay-in-one-transaction", "verify"]

The steps of taking a dump into a database, in order: make the schema first (a dump has none, and a full dump with its schema cannot be replayed twice), replay each dump inside one transaction in any order and as often as you like, and then verify, because a raw import passed through no check of ours.

Imported by 5 tests of the run (5 passed, 0 failed): the library's results

CONFLICT_RULEconst
const CONFLICT_RULE: "ON CONFLICT IGNORE"

The conflict rule of the primary key of a mergeable table: ignore the second row, so that replaying a dump twice, or two overlapping dumps, is replaying their union.

Imported by 5 tests of the run (5 passed, 0 failed): the library's results

IMPORT_CHECKSconst
const IMPORT_CHECKS: { readonly trigger: readonly ["replica-in-trust-table"]; readonly verify: readonly ["sum", "id", "blob-digest", "future-id", "duplicate-id", "dangling-parent", "missing-blob"]; }

How trust is held where a raw import bypasses the application: a trigger before the insert refuses a row of a replica that is not in the trust table, and the verify step checks the rest (sums, ids, blobs, time), which a trigger cannot.

Imported by 5 tests of the run (5 passed, 0 failed): the library's results

Epochtype
type Epoch = { readonly n: number; readonly horizon: VersionVector; readonly roster: readonly ReplicaId[]; readonly retained: readonly string[] };

An epoch: a compaction of the rows against a roster of replicas that is known to be complete. A row whose number the horizon covers is dead unless the epoch retains its key, so that replaying an old dump cannot bring back what was collected.

Imported by 5 tests of the run (5 passed, 0 failed): the library's results

horizonOffunction
function horizonOf(reports: Readonly<Record<string, VersionVector>>, roster: readonly ReplicaId[]): Result<VersionVector, string>

The horizon of a roster: for each replica, the least number any replica of the roster holds of its rows. Every replica of the roster must have reported its vector: otherwise the roster is not known to be complete, and the refusal says who is missing.

Imported by 5 tests of the run (5 passed, 0 failed): the library's results

deadAfterfunction
function deadAfter(epoch: Epoch, v: Version): boolean

Whether a row is dead after an epoch: its number is covered by the horizon and the epoch does not retain it.

Imported by 5 tests of the run (5 passed, 0 failed): the library's results

StoredIdentitytypeno test imports it
type StoredIdentity = { readonly replica: ReplicaId; readonly fingerprint: Fingerprint };

What a replica keeps of its identity: who it is and what the host it was made on was.

Identitytypeno test imports it
type Identity = { readonly replica: ReplicaId; readonly fingerprint: Fingerprint; readonly forkedFrom: ReplicaId | undefined };

What opening a database ended with: the identity to use, and whether the database is a copy that was given a new one.

openIdentityfunction
function openIdentity(stored: StoredIdentity | undefined, now: Fingerprint, fresh: () => ReplicaId): Identity

The identity of a replica when its database is opened. Where the database holds one and the fingerprint is the one the host makes now, it is kept. Where the fingerprint is not (the file was copied, or restored from a backup into another place), the replica takes a new identity, so that two copies never make the same ids, and says which it was copied from. Nothing was stored: a new identity.

Imported by 5 tests of the run (5 passed, 0 failed): the library's results

Titles, headings, exports, tables, columns and requirements. Esc closes.