describeLexErrorfunctionfunction describeLexError(e: LexError): stringOne sentence for an error of the lexer.
The lexer of SQL (commons lang-sql 0001-sql-lang): one pass over the character codes of a text, by the tables of a compiled lexical, that returns the tokens, the comments and the errors, so that every character is in exactly one token, one comment or white space. It does not throw, does not judge where a word is a keyword, and names no dialect: what a dialect writes is its lexical.
The hot path reads each character once: white space, then a comment (the one table lookup that says whether a character may begin one), a word with its keyword found by a hash of its upper-cased characters, a number, a quoted string or identifier, a parameter, punctuation, then an operator, the longest of the dialect's first. A token is four pushes onto arrays of small integers; its text is cut from the source only when someone asks (textAt).
const { tokens, comments, errors } = lex("select 1 -- one", ansi.compiled);
errors.length; // 0Spec: 0001-sql-lang FR-004, FR-005, FR-006, FR-007, FR-008, FR-009, FR-010, FR-011, FR-012, FR-013, FR-018
describeLexErrorfunctionfunction describeLexError(e: LexError): stringOne sentence for an error of the lexer.
LexErrortypeno test imports ittype LexError =
| { readonly kind: "unterminated-comment"; readonly span: Span }
| { readonly kind: "unterminated-string"; readonly span: Span }
| { readonly kind: "unterminated-identifier"; readonly span: Span }
| { readonly kind: "unexpected-character"; readonly span: Span; readonly char: string };What the lexer found that the dialect cannot read.
Lexedinterfaceno test imports itinterface LexedThe result of lexing a text.
| Member | Type | What it is |
|---|---|---|
text | string | |
tokens | Tokens | |
comments | Comments | |
errors | readonly LexError[] |
lexfunctionfunction lex(text: string, lx: CompiledLexical, from?: number, to?: number): LexedLexes a text.
| Parameter | What it is |
|---|---|
text | SQL, in any dialect |
lx | how that dialect is written, compiled |
from | where to begin, 0 by default; the tokens, comments and errors have offsets of the whole text |
to | where to end, the end of the text by default |
Returns: its tokens (the last is the end of the text), its comments and what it holds that the dialect cannot read