IF Commons GitHub

Convergence

0019Active

0019-sql-convergence · owner if · domain data

Reading a database

FR-001checkpassing

A dialect MUST be able to read its own catalog into a neutral value (the tables, each with its columns' written types and nullability, its primary key, its indexes and, where the dialect can list them, the names and kinds of its foreign keys, uniques and checks) and to say what it compares, so that reading is owned by the dialect and the strategy names no engine.

Enforced (check): .github: cd commons/data && bun run verify — suite/convergence.test.ts: each dialect reads its own catalog into a neutral value and says what it compares

Proved by: data/suite/convergence.test.ts

Written in: data/sql/extensions/convergence/mod.ts, data/sql/extensions/convergence/read.ts

Tests: passed: suite/convergence.test.ts each dialect reads its own catalog into a neutral value and says what it compares

Open: issue data 00012: Two readers of a database's catalog: convergence's `catalog` and the dialects' `observe`

Cited by: Data, sql/extensions/convergence/read, sql/physical/dialect

FR-002checkpassing

readState MUST make a state from that value: a table and a column by the words of its name (with the schema's words taken from a table's where the dialect has none), a type as the desired state's where the two are the same type under another name (character varying and VARCHAR), nullability and key as found, an index and a constraint as the desired state's where one of the same name is found and as unknown where not, and what the dialect cannot list as the desired state's; and the state of a database made from the desired schema MUST have the hash of the desired state.

Enforced (check): .github: cd commons/data && bun run verify — suite/convergence.test.ts: the state of a database made from a schema has the hash of the desired state, with types read under another name found the same, and the words of a table found with the schema's taken off

Proved by: data/suite/convergence.test.ts

Written in: data/sql/extensions/convergence/read.ts

Tests: passed: suite/convergence.test.ts the state of a database made from a schema has the hash of the desired state, with types r

FR-003checkpassing

What a state does not compare (a default, a check's text, a table clause, and a foreign key, unique or check of a dialect that cannot list them) MUST be said by the dialect, and a difference in what is compared MUST be a change: a column added by hand is a column to drop, a column made optional is a change of nullability, a table missing is a table to make.

Enforced (check): .github: cd commons/data && bun run verify — suite/convergence.test.ts: drift made by hand is a difference: a column added is a column to drop, a table missing is a table to make, a table made by hand is a table to drop, and what is not compared is not

Proved by: data/suite/convergence.test.ts

Written in: data/sql/extensions/convergence/read.ts

Tests: passed: suite/convergence.test.ts drift made by hand is a difference: a column added is a column to drop, a table missing is

Cited by: Data, sql/extensions/convergence/read

The plan from what is there

FR-004checkpassing

propose MUST read the database's state, compute the plan from it to the state wanted with the transforms and the fallback it is given, and pin it: the hash of the state it was computed from and the hash of the state it ends in; and a plan between states that are equal MUST have no step.

Enforced (check): .github: cd commons/data && bun run verify — suite/convergence.test.ts: a proposal is the plan from the state read, pinned to it and to the state it ends in, empty between equal states, and refused as stale when the database moved

Proved by: data/suite/convergence.test.ts

Written in: data/sql/extensions/convergence/plan.ts, data/sql/extensions/convergence/propose.ts

Tests: passed: suite/convergence.test.ts a proposal is the plan from the state read, pinned to it and to the state it ends in, empt

Cited by: Data, sql/extensions/convergence/propose

FR-005checkpassing

Applying a proposal MUST read the state again and refuse, as an error value naming both hashes, a database that is no longer in the state the proposal was pinned to.

Enforced (check): .github: cd commons/data && bun run verify — suite/convergence.test.ts: a proposal is the plan from the state read, pinned to it and to the state it ends in, empty between equal states, and refused as stale when the database moved

Proved by: data/suite/convergence.test.ts

Written in: data/sql/extensions/convergence/propose.ts

Tests: passed: suite/convergence.test.ts a proposal is the plan from the state read, pinned to it and to the state it ends in, empt

Reviewed

FR-006checkpassing

A reviewed proposal MUST be a file of text (the pinned hashes, each step's name, class, reversibility, idempotency and why, its statements and its verification) that is the same every time for the same proposal; apply MUST take the hash of the text that was approved and refuse, as an error value, text that does not have it, and MUST then run the steps through the lifecycle's run, so that a second apply finds nothing to do.

Enforced (check): .github: cd commons/data && bun run verify — suite/convergence.test.ts: a reviewed proposal is text a person approves by its hash, the same every time; it is applied only if the text is the approved one, leaves the database as bind makes it, and a second apply finds nothing to do

Proved by: data/suite/convergence.test.ts

Written in: data/sql/extensions/convergence/propose.ts

Tests: passed: suite/convergence.test.ts a reviewed proposal is text a person approves by its hash, the same every time; it is appl

Cited by: Data, sql/extensions/convergence/propose

Autonomous

FR-007checkpassing

connect MUST read the state, compute the plan and, where the policy allows every step (by default safe steps that are lossless, of the kinds that only add), apply it, with a lock that keeps one instance applying where the dialect has one; where the policy does not allow a step it MUST change nothing and return the plan as held for a person, naming the steps it did not allow; and where a step fails it MUST return the error and leave the database at the step before.

Enforced (check): .github: cd commons/data && bun run verify — suite/convergence.test.ts: autonomous convergence applies what only adds, holds a plan that renames or drops and changes nothing, holds everything when the policy is off, and says which steps it did not allow; suite/convergence.test.ts: a step that fails leaves the database at the step before and the next access resumes, a lock is taken where the dialect has one, and a rehearsal that says no holds the plan

Proved by: data/suite/convergence.test.ts

Written in: data/sql/extensions/convergence/connect.ts, data/sql/extensions/convergence/plan.ts

Tests: passed: suite/convergence.test.ts autonomous convergence applies what only adds, holds a plan that renames or drops and chan; passed: suite/convergence.test.ts a step that fails leaves the database at the step before and the next access resumes, a lo

Cited by: Data, sql/extensions/convergence/connect

FR-008checkpassing

The policy MUST be a setting (the classes, the reversibilities and the kinds of change allowed, and whether it is on at all), with a default that only adds, and a policy that is off MUST apply nothing and hold every plan.

Enforced (check): .github: cd commons/data && bun run verify — suite/convergence.test.ts: autonomous convergence applies what only adds, holds a plan that renames or drops and changes nothing, holds everything when the policy is off, and says which steps it did not allow

Proved by: data/suite/convergence.test.ts

Written in: data/sql/extensions/convergence/connect.ts, data/sql/extensions/convergence/settings.ts

Tests: passed: suite/convergence.test.ts autonomous convergence applies what only adds, holds a plan that renames or drops and chan

Cited by: Data, sql/extensions/convergence/settings

FR-009checkpassing

A rehearsal MAY be given: where it is, connect MUST run it on the plan before it applies it and MUST hold the plan where it says no.

Enforced (check): .github: cd commons/data && bun run verify — suite/convergence.test.ts: a step that fails leaves the database at the step before and the next access resumes, a lock is taken where the dialect has one, and a rehearsal that says no holds the plan

Proved by: data/suite/convergence.test.ts

Written in: data/sql/extensions/convergence/connect.ts

Tests: passed: suite/convergence.test.ts a step that fails leaves the database at the step before and the next access resumes, a lo

The version gate

FR-010checkpassing

The version gate MUST say whether a database is in the state an application expects, by the hash, and MUST return an error value naming both where it is not, so that an application can refuse to start instead of migrating.

Enforced (check): .github: cd commons/data && bun run verify — suite/convergence.test.ts: the version gate says whether a database is in the state an application expects, and names both states where it is not

Proved by: data/suite/convergence.test.ts

Written in: data/sql/extensions/convergence/connect.ts

Tests: passed: suite/convergence.test.ts the version gate says whether a database is in the state an application expects, and names

Cited by: Data, sql/extensions/convergence/connect

Settings

FR-011checkpassing

The approval (reviewed or autonomous), the policy, the ledger's table and the names the reader leaves out (by default the ledger's table and the tables that quarantine makes by its default suffixes, so that what the library keeps aside is not read as drift; a name that begins with * leaving out every table whose name ends with the rest of it, so that what a feature keeps aside is not read as drift) MUST each have a default and be changeable.

Enforced (check): .github: cd commons/data && bun run verify — suite/convergence.test.ts: autonomous convergence applies what only adds, holds a plan that renames or drops and changes nothing, holds everything when the policy is off, and says which steps it did not allow

Proved by: data/suite/convergence.test.ts

Written in: data/sql/extensions/convergence/settings.ts

Tests: passed: suite/convergence.test.ts autonomous convergence applies what only adds, holds a plan that renames or drops and chan

Cited by: Data, sql/extensions/convergence/settings

What the database holds

FR-012checkpassing

On SQLite, DuckDB and PostgreSQL, a database made by bind for one version of a domain (outside the lifecycle) MUST be read as the state of that version, drift made by hand MUST be found, reviewed convergence to the next version MUST leave the database as bind makes it for that version, autonomous convergence MUST apply what only adds and hold what does not, and a stale proposal MUST be refused.

Enforced (check): .github: cd commons/data && bun run verify — suite/convergence.test.ts: drift made by hand is a difference: a column added is a column to drop, a table missing is a table to make, a table made by hand is a table to drop, and what is not compared is not; suite/convergence.test.ts: a reviewed proposal is text a person approves by its hash, the same every time; it is applied only if the text is the approved one, leaves the database as bind makes it, and a second apply finds nothing to do

Proved by: data/suite/convergence.test.ts

Tests: passed: suite/convergence.test.ts drift made by hand is a difference: a column added is a column to drop, a table missing is; passed: suite/convergence.test.ts a reviewed proposal is text a person approves by its hash, the same every time; it is appl

FR-013checkpassing

The reader MUST read each column's default as the catalog says it and compare it with the state wanted after casts, outer parentheses, spaces and case outside quotes are set aside, so that a default changed by hand is a difference and a default the model made is not; and where a project says unmanaged: hold, a table, a column, an index or a constraint that is in the database and not in the model MUST be left out of the state read, so that no step touches it, and MUST be named in the proposal as held.

Enforced (check): .github: cd commons/data && bun run verify — suite/convergence.test.ts: a default changed by hand is drift on the engines that can change one, and the defaults the model made are not drift on any engine; suite/convergence.test.ts: what a person made outside the model is held, not dropped, where the project says to hold it: a table, a column and an index are left alone and named, and the plan has no step for them

Proved by: data/suite/convergence.test.ts

Tests: passed: suite/convergence.test.ts a default changed by hand is drift on the engines that can change one, and the defaults th; passed: suite/convergence.test.ts what a person made outside the model is held, not dropped, where the project says to hold

Out of scope

  • Comparing defaults, the text of checks and partitions with the catalog: said by each dialect and left to the strategy that reads.
  • Signing the approval: the approved hash is what is checked; who approved it is the project's workflow.
  • Converging many databases at once: each is met on its own.

Edge cases

  • A table made by hand that governance does not know: a table to drop, quarantined, per FR-003.
  • A database restored from a backup of an older release: the plan from that state, per FR-004.
  • A proposal approved and then the database changed: refused as stale, per FR-005.
  • Two application instances starting together: one applies and the other waits, then finds nothing to do, per FR-007.
  • Autonomous on a plan that renames a column: held, per FR-007.
  • The policy off: every plan held, per FR-008.

Assumptions

  • The catalog is read inside no transaction of its own and is the state of the moment.
  • A code table's rows are part of the state; other data is not.

Open questions

OQ-1open question

Whether the held plans are also written as proposals to a place a person is told of, and by whom.

Key entities

  • Live state - the state read from a database's catalog.
  • Proposal - a plan pinned to the state it was computed from.
  • Policy - what may be applied with no one's approval.
  • Version gate - the check that a database is where an application expects.

Success criteria

SC-001success criterion

Any database, however it came to be, is brought to the state wanted by a plan someone can read first, or by one that only adds.

Review & acceptance checklist

  • [x] Every requirement is testable (MUST / MUST NOT), not aspirational
  • [x] No company fact is asserted here
  • [x] Every open item is marked, not silently decided

Cited by

Data, sql/extensions/convergence/connect, sql/extensions/convergence/plan, sql/extensions/convergence/propose, sql/extensions/convergence/read, sql/extensions/convergence/settings, sql/physical/dialect, Presets

Titles, headings, exports, tables, columns and requirements. Esc closes.