endAtfunctionno test imports itfunction endAt(t: Tokens, i: number): numberWhere the token at i ends; the length of the text past the last.
The tokens of SQL text (commons lang-sql 0001-sql-lang FR-005): a closed set of kinds and the parallel columns the lexer fills, so that a token costs no object, no string and no allocation of its own. A column is read with its accessor, which answers the end of the text past the last token, so that a parser may look ahead without a bounds check.
The columns are arrays of small integers, which the engine keeps packed; they carry the types of the kinds, so that no cast is needed to read one.
const t = lex("select a", ansi.compiled).tokens;
kindAt(t, 0) === Tok.Word; // true
textAt(t, "select a", 1); // "a"Spec: 0001-sql-lang FR-005
endAtfunctionno test imports itfunction endAt(t: Tokens, i: number): numberWhere the token at i ends; the length of the text past the last.
idAtfunctionfunction idAt(t: Tokens, i: number): numberThe id of the token at i (see Tokens); 0 past the last.
kindAtfunctionfunction kindAt(t: Tokens, i: number): TokKindThe kind of the token at i; the end of the text past the last.
startAtfunctionno test imports itfunction startAt(t: Tokens, i: number): numberWhere the token at i begins; the length of the text past the last.
textAtfunctionno test imports itfunction textAt(t: Tokens, source: string, i: number): stringThe text of the token at i, cut from the source only now.
TokKindtypeno test imports ittype TokKind = (typeof Tok)[keyof typeof Tok];A kind of token.
Tokconstconst Tok: { readonly Eof: 0; readonly Word: 1; readonly QuotedIdent: 2; readonly Number: 3; readonly String: 4; readonly Param: 5; readonly Op: 6; readonly LParen: 7; readonly RParen: 8; readonly LBracket: 9; readonly RBracket: 10; readonly LBrace: 11; readonly RBrace: 12; readonly Comma: 13; readonly Semicolon: 14; readonly Dot: 15; readonly Error: 16; }The kinds of token.
tokenCountfunctionfunction tokenCount(t: Tokens): numberHow many tokens there are, the end of the text included.
Tokensinterfaceno test imports itinterface TokensThe tokens of a text, as columns. id is, by kind: for a word the keyword's id (0 where it is none); for an operator the operator's id; for a parameter its style (ParamStyleId); for a string the index of its syntax plus one (0 for a dollar quote); for a quoted identifier the index of its syntax; otherwise 0.
| Member | Type | What it is |
|---|---|---|
kind | readonly TokKind[] | |
start | readonly number[] | |
end | readonly number[] | |
id | readonly number[] |