httpnamespacenamespace httpHow a typed client reaches a system over HTTP (commons 0001-net FR-003): a transport, live (fetch) or recorded (the exchanges a test keeps, so it reaches no host), and over it the guards and helpers every client otherwise rewrites: a read-only guard (safe methods only, POST to the token endpoints named, and the POSTs a predicate says only read, FR-066), AWS Signature Version 4 on each request (FR-069), backoff on 429 and 503 that honours Retry-After, a JSON read decoded at the boundary with typed failures (a denied call is told apart, so a caller can record a gap instead of failing), and every item of a list that pages by Link: rel="next" or by a cursor in the body. A request that carries a credential is refused before it is sent unless the transport is made to allow it, so a harness whose proxy injects credentials never sends one by mistake. Taken from erga's lib/ts/http-client and Stolos's connector SDK (src/archon/connect/sdk.ts, connector.ts).
import { array, object, string } from "@intellectual-frontiers/core";
import { getJson, liveTransport, withBackoff } from "./http-client.ts";
const http = withBackoff(liveTransport());
const repo = await getJson(http, "https://api.github.com/repos/oven-sh/bun", object({ full_name: string }));httpnamespacenamespace httphttp.OutRequesttypeno test imports ittype OutRequest = {
readonly method: "GET" | "HEAD" | "POST" | "PUT" | "PATCH" | "DELETE";
readonly url: string;
readonly headers: Readonly<Record<string, string>>;
readonly body?: string;
/** Read the answer as bytes (`Exchange.bytes`), for an artifact that is not text (a tarball, a zip); the text `body` of the answer is then empty. */
readonly binary?: boolean;
};A request a client makes.
http.Exchangetypeno test imports ittype Exchange = {
readonly method: OutRequest["method"];
readonly url: string;
readonly status: number;
/** The … (429 characters)type Exchange = {
readonly method: OutRequest["method"];
readonly url: string;
readonly status: number;
/** The headers kept: the content type, paging, rate limits and `Retry-After`. */
readonly headers: Readonly<Record<string, string>>;
readonly body: string;
/** The answer's bytes: present where the request asked for `binary` (a recorded exchange of an artifact holds them). */
readonly bytes?: Uint8Array;
};One exchange: a request and what answered it.
http.TransportErrortypeno test imports ittype TransportError =
{ readonly kind: "refused"; readonly message: string } | { readonly kind: "network"; readonly message: string };Why a request got no answer at all (a refusal before it was sent, or the network).
http.Transporttypeno test imports ittype Transport = { readonly kind: "live" | "recorded"; readonly send: (r: OutRequest) => Promise<Result<Exchange, TransportError>> };A way to reach a system.
http.CREDENTIAL_HEADERSconstno test imports itconst CREDENTIAL_HEADERS: readonly string[]The headers a credential travels in.
http.carriesCredentialfunctionno test imports itfunction carriesCredential(r: OutRequest): string | undefinedWhy a request may not be sent where credentials are refused: it carries one in a header or in its query. Undefined where it carries none.
http.LiveOptionstypeno test imports ittype LiveOptions = {
/** The `fetch` to use (the platform's by default). */
readonly fetch?: typeof fetch;
/** Ref… (465 characters)type LiveOptions = {
/** The `fetch` to use (the platform's by default). */
readonly fetch?: typeof fetch;
/** Refuse a request that carries a credential (the default), or send it. */
readonly credentials?: "refuse" | "allow";
/** The `User-Agent` said to every system: a generic name, never a person's. */
readonly userAgent?: string;
/** No answer within this many seconds fails the request (30 by default). */
readonly timeoutSeconds?: number;
};How the live transport sends.
http.liveTransportfunctionno test imports itfunction liveTransport(options?: LiveOptions): TransportThe live transport, over fetch.
http.recordedTransportfunctionno test imports itfunction recordedTransport(exchanges: readonly Exchange[]): TransportA transport that answers from recorded exchanges, by method and URL, and refuses a request it has none for, naming it.
http.ReadPosttypeno test imports ittype ReadPost = (r: OutRequest) => boolean;Whether a POST only reads (FR-066): a GraphQL query, a search, a JSON API's Describe and Get actions. It is given the whole request (method, URL, headers and body), as an action may be named in a header (x-amz-target) or the body.
http.readOnlyfunctionno test imports itfunction readOnly(t: Transport, tokenEndpoints?: readonly string[], readPost?: ReadPost | undefined): TransportA transport that sends only safe methods (GET, HEAD, ...), POST to URLs that start with one of tokenEndpoints, and the POSTs that readPost says only read (FR-066). Every other request (PUT, PATCH, DELETE, any other POST) is refused without being sent.
http.SigV4TransportOptionstypeno test imports ittype SigV4TransportOptions = {
readonly credentials: AwsCredentials;
readonly region: string;
readonly service: string;
/** The clock read as each request is sent (`new Date()` by default; a test gives its own). */
readonly now?: () => Date;
/** Sign `UNSIGNED-PAYLOAD` instead of the body's hash (S3 allows it). */
readonly unsignedPayload?: boolean;
};How withSigV4 signs.
http.withSigV4functionno test imports itfunction withSigV4(t: Transport, options: SigV4TransportOptions): TransportA transport that signs each request by AWS Signature Version 4 (FR-069, by the kernel's sigv4, FR-068) when it is sent. A signed request carries a credential, so the live transport under it must be made with credentials: "allow", or it refuses the request.
http.BackoffOptionstypeno test imports ittype BackoffOptions = {
/** Tries after the first (3 by default). */
readonly retries?: number;
/** How a wait is spent (a timer by default; a test passes one that returns at once). */
readonly pause?: (ms: number) => Promise<void>;
/** The longest wait, whatever `Retry-After` asks (60 s by default). */
readonly capMs?: number;
};How long to back off.
http.withBackofffunctionno test imports itfunction withBackoff(t: Transport, options?: BackoffOptions): TransportA transport that waits and tries again on 429 and 503: as long as Retry-After asks (up to the cap), else 1, 2, 4 seconds.
http.ReadErrortypeno test imports ittype ReadError =
| TransportError
| { readonly kind: "denied"; readonly status: 401 | 403; readonly url: string }
| { readonly kind: "status"; readonly status: number; readonly url: string }
| { readonly kind: "decode"; readonly url: string; readonly message: string };Why a JSON read failed.
http.formatReadErrorfunctionno test imports itfunction formatReadError(e: ReadError): stringA read error as one sentence; never a header or a body, so no credential or personal datum is said.
http.getExchangefunctionno test imports itfunction getExchange<T>(t: Transport, url: string, decoder: Decoder<T>, headers?: Readonly<Record<string, string>>): Promise<Result<{ readonly value: T; readonly exchange: Exchange; }, ReadError>>A GET whose JSON body is decoded, with the exchange; 401 and 403 are denied, any other status from 400 up is status.
http.getJsonfunctionno test imports itfunction getJson<T>(t: Transport, url: string, decoder: Decoder<T>, headers?: Readonly<Record<string, string>>): Promise<Result<T, ReadError>>A GET whose JSON body is decoded.
http.linkPagesfunctionno test imports itfunction linkPages<T>(t: Transport, url: string, item: Decoder<T>, options?: { readonly headers?: Readonly<Record<string, string>> | undefined; readonly maxPages?: number | undefined; }): AsyncGenerator<T, Result<number, ReadError>, any>Every page of a list that pages by Link: <...>; rel="next", at most maxPages (200 by default); the first failure ends it, as an error.
http.cursorPagesfunctionno test imports itfunction cursorPages<B, T>(t: Transport, first: string, page: Decoder<B>, items: (body: B) => readonly T[], next: (body: B, url: string) => string | undefined, options?: { readonly headers?: Readonly<Record<string, string>> | undefined; readonly maxPages?: number | undefined; }): AsyncGenerator<T, Result<number, ReadError>, any>Every item of a list that pages by a cursor in its body: items takes a page's items, next the next page's URL or undefined.
http.decodeRecordedfunctionno test imports itfunction decodeRecorded(text: string): Result<Exchange, string>A recorded exchange read from the JSON5 text recordedText writes, or why it is not one.
http.recordedTextfunctionno test imports itfunction recordedText(x: Exchange, note: string): stringThe text a recorded exchange is kept in: a comment that says where and when it was recorded, then the exchange as JSON5.
http.bearerfunctionno test imports itfunction bearer(token: string, scheme?: string): Readonly<Record<string, string>>Headers that give a bearer token (for a transport made to allow credentials).
OutRequesttypeno test imports ittype OutRequest = {
readonly method: "GET" | "HEAD" | "POST" | "PUT" | "PATCH" | "DELETE";
readonly url: string;
readonly headers: Readonly<Record<string, string>>;
readonly body?: string;
/** Read the answer as bytes (`Exchange.bytes`), for an artifact that is not text (a tarball, a zip); the text `body` of the answer is then empty. */
readonly binary?: boolean;
};A request a client makes.
Exchangetypetype Exchange = {
readonly method: OutRequest["method"];
readonly url: string;
readonly status: number;
/** The … (429 characters)type Exchange = {
readonly method: OutRequest["method"];
readonly url: string;
readonly status: number;
/** The headers kept: the content type, paging, rate limits and `Retry-After`. */
readonly headers: Readonly<Record<string, string>>;
readonly body: string;
/** The answer's bytes: present where the request asked for `binary` (a recorded exchange of an artifact holds them). */
readonly bytes?: Uint8Array;
};One exchange: a request and what answered it.
TransportErrortypeno test imports ittype TransportError =
{ readonly kind: "refused"; readonly message: string } | { readonly kind: "network"; readonly message: string };Why a request got no answer at all (a refusal before it was sent, or the network).
Transporttypeno test imports ittype Transport = { readonly kind: "live" | "recorded"; readonly send: (r: OutRequest) => Promise<Result<Exchange, TransportError>> };A way to reach a system.
CREDENTIAL_HEADERSconstno test imports itconst CREDENTIAL_HEADERS: readonly string[]The headers a credential travels in.
carriesCredentialfunctionfunction carriesCredential(r: OutRequest): string | undefinedWhy a request may not be sent where credentials are refused: it carries one in a header or in its query. Undefined where it carries none.
LiveOptionstypeno test imports ittype LiveOptions = {
/** The `fetch` to use (the platform's by default). */
readonly fetch?: typeof fetch;
/** Ref… (465 characters)type LiveOptions = {
/** The `fetch` to use (the platform's by default). */
readonly fetch?: typeof fetch;
/** Refuse a request that carries a credential (the default), or send it. */
readonly credentials?: "refuse" | "allow";
/** The `User-Agent` said to every system: a generic name, never a person's. */
readonly userAgent?: string;
/** No answer within this many seconds fails the request (30 by default). */
readonly timeoutSeconds?: number;
};How the live transport sends.
liveTransportfunctionfunction liveTransport(options?: LiveOptions): TransportThe live transport, over fetch.
recordedTransportfunctionfunction recordedTransport(exchanges: readonly Exchange[]): TransportA transport that answers from recorded exchanges, by method and URL, and refuses a request it has none for, naming it.
ReadPosttypetype ReadPost = (r: OutRequest) => boolean;Whether a POST only reads (FR-066): a GraphQL query, a search, a JSON API's Describe and Get actions. It is given the whole request (method, URL, headers and body), as an action may be named in a header (x-amz-target) or the body.
readOnlyfunctionfunction readOnly(t: Transport, tokenEndpoints?: readonly string[], readPost?: ReadPost | undefined): TransportA transport that sends only safe methods (GET, HEAD, ...), POST to URLs that start with one of tokenEndpoints, and the POSTs that readPost says only read (FR-066). Every other request (PUT, PATCH, DELETE, any other POST) is refused without being sent.
SigV4TransportOptionstypeno test imports ittype SigV4TransportOptions = {
readonly credentials: AwsCredentials;
readonly region: string;
readonly service: string;
/** The clock read as each request is sent (`new Date()` by default; a test gives its own). */
readonly now?: () => Date;
/** Sign `UNSIGNED-PAYLOAD` instead of the body's hash (S3 allows it). */
readonly unsignedPayload?: boolean;
};How withSigV4 signs.
withSigV4functionfunction withSigV4(t: Transport, options: SigV4TransportOptions): TransportA transport that signs each request by AWS Signature Version 4 (FR-069, by the kernel's sigv4, FR-068) when it is sent. A signed request carries a credential, so the live transport under it must be made with credentials: "allow", or it refuses the request.
BackoffOptionstypeno test imports ittype BackoffOptions = {
/** Tries after the first (3 by default). */
readonly retries?: number;
/** How a wait is spent (a timer by default; a test passes one that returns at once). */
readonly pause?: (ms: number) => Promise<void>;
/** The longest wait, whatever `Retry-After` asks (60 s by default). */
readonly capMs?: number;
};How long to back off.
withBackofffunctionfunction withBackoff(t: Transport, options?: BackoffOptions): TransportA transport that waits and tries again on 429 and 503: as long as Retry-After asks (up to the cap), else 1, 2, 4 seconds.
ReadErrortypeno test imports ittype ReadError =
| TransportError
| { readonly kind: "denied"; readonly status: 401 | 403; readonly url: string }
| { readonly kind: "status"; readonly status: number; readonly url: string }
| { readonly kind: "decode"; readonly url: string; readonly message: string };Why a JSON read failed.
formatReadErrorfunctionfunction formatReadError(e: ReadError): stringA read error as one sentence; never a header or a body, so no credential or personal datum is said.
getExchangefunctionno test imports itfunction getExchange<T>(t: Transport, url: string, decoder: Decoder<T>, headers?: Readonly<Record<string, string>>): Promise<Result<{ readonly value: T; readonly exchange: Exchange; }, ReadError>>A GET whose JSON body is decoded, with the exchange; 401 and 403 are denied, any other status from 400 up is status.
getJsonfunctionfunction getJson<T>(t: Transport, url: string, decoder: Decoder<T>, headers?: Readonly<Record<string, string>>): Promise<Result<T, ReadError>>A GET whose JSON body is decoded.
linkPagesfunctionfunction linkPages<T>(t: Transport, url: string, item: Decoder<T>, options?: { readonly headers?: Readonly<Record<string, string>> | undefined; readonly maxPages?: number | undefined; }): AsyncGenerator<T, Result<number, ReadError>, any>Every page of a list that pages by Link: <...>; rel="next", at most maxPages (200 by default); the first failure ends it, as an error.
cursorPagesfunctionfunction cursorPages<B, T>(t: Transport, first: string, page: Decoder<B>, items: (body: B) => readonly T[], next: (body: B, url: string) => string | undefined, options?: { readonly headers?: Readonly<Record<string, string>> | undefined; readonly maxPages?: number | undefined; }): AsyncGenerator<T, Result<number, ReadError>, any>Every item of a list that pages by a cursor in its body: items takes a page's items, next the next page's URL or undefined.
decodeRecordedfunctionfunction decodeRecorded(text: string): Result<Exchange, string>A recorded exchange read from the JSON5 text recordedText writes, or why it is not one.
recordedTextfunctionfunction recordedText(x: Exchange, note: string): stringThe text a recorded exchange is kept in: a comment that says where and when it was recorded, then the exchange as JSON5.
bearerfunctionfunction bearer(token: string, scheme?: string): Readonly<Record<string, string>>Headers that give a bearer token (for a transport made to allow credentials).