IF Commons GitHub

sql/loads/source

Written to: 2 requirements: 2 proved, 0 partly, 0 failing, 0 with no test recorded. 0054-sql-loads FR-001, 0054-sql-loads FR-002

Sources of a load (commons data 0054-sql-loads): where the rows come from, as batches of rows of a type. A source is a value: rows in memory, CSV text read with a typed parser for each column, lines of JSON, or a select of another database. A line that cannot be read is not a thrown error but a rejected line, with its number and why, so that a load says what it did not take.

Source: commons/data/sql/loads/source.ts

8 of 13 exports are imported by a test of the run. A test that imports an export may not call it or check what it returns.

Batchinterfaceno test imports it
interface Batch<R>

A batch: rows and the lines that were rejected on the way.

MemberTypeWhat it is
rowsreadonly R[]
rejectedreadonly Rejected[]
boolconst
const bool: CellParser<boolean>

true or false (also yes, no, 1, 0, t, f, in any case).

Imported by 8 tests of the run (8 passed, 0 failed): the library's results

CellParsertypeno test imports it
type CellParser<T> = (cell: string) => Result<T, string>;

How one column of a CSV is read: the text of a cell to a value, or why not.

csvRecordsfunction
function csvRecords(textIn: string): readonly { readonly line: number; readonly cells: readonly string[]; }[]

The cells of CSV text (RFC 4180: quoted cells, doubled quotes, line breaks inside quotes), with the line each record starts on.

Imported by 8 tests of the run (8 passed, 0 failed): the library's results

fromCsvfunction
function fromCsv<R>(csv: string, columns: { readonly [K in keyof R & string]: CellParser<R[K]> | { readonly parse: CellParser<R[K]>; readonly column: string; }; }, name?: string): Source<R>

CSV text whose first line names the columns, read into rows of type R: each key of R has the parser of its cell and the name of its column in the file (the key where not given). A line with a cell that does not parse, or the wrong number of cells, is rejected with its line and why; the rest are taken.

Imported by 8 tests of the run (8 passed, 0 failed): the library's results

fromJsonLinesfunction
function fromJsonLines<R>(jsonl: string, read: (value: unknown) => Result<R, string>, name?: string): Source<R>

Lines of JSON, one record each, read by a function that checks a value is a row of R; a line that is not JSON or not a row is rejected.

Imported by 8 tests of the run (8 passed, 0 failed): the library's results

fromQueryconst
const fromQuery: <R>(select: Select<Scope, R>, on: Runner, name?: string) => Source<R>

The rows of a select, run on a database: the way to copy between databases.

Imported by 8 tests of the run (8 passed, 0 failed): the library's results

fromRowsconst
const fromRows: <R>(rows: readonly R[], name?: string) => Source<R>

Rows in memory.

Imported by 13 tests of the run (13 passed, 0 failed): the library's results

oneOfconst
const oneOf: <T extends string>(...codes: readonly T[]) => CellParser<T>

One of a closed list of codes.

Imported by 8 tests of the run (8 passed, 0 failed): the library's results

optionalconst
const optional: <T>(p: CellParser<T>) => CellParser<T | null>

A cell that may be empty: empty is null.

Imported by 8 tests of the run (8 passed, 0 failed): the library's results

Rejectedinterfaceno test imports it
interface Rejected

A line that was not taken.

MemberTypeWhat it is
atnumberWhere it was: the line (CSV, JSON lines) or the row (others), from 1.
reasonstring
rawstringWhat it was, as text.
Sourceinterfaceno test imports it
interface Source<R>

Where rows come from.

MemberTypeWhat it is
namestring
batches(size: number) => AsyncIterable<Batch<R>>The rows, in batches of at most size.
timestampconstno test imports it
const timestamp: CellParser<string>

An instant as ISO 8601.

Titles, headings, exports, tables, columns and requirements. Esc closes.