IF Commons GitHub

http-client

Written to: 1 requirement: 1 proved, 0 partly, 0 failing, 0 with no test recorded. 0001-net FR-003

How a typed client reaches a system over HTTP (commons 0001-net FR-003): a transport, live (fetch) or recorded (the exchanges a test keeps, so it reaches no host), and over it the guards and helpers every client otherwise rewrites: a read-only guard (safe methods only, POST to the token endpoints named, and the POSTs a predicate says only read, FR-066), AWS Signature Version 4 on each request (FR-069), backoff on 429 and 503 that honours Retry-After, a JSON read decoded at the boundary with typed failures (a denied call is told apart, so a caller can record a gap instead of failing), and every item of a list that pages by Link: rel="next" or by a cursor in the body. A request that carries a credential is refused before it is sent unless the transport is made to allow it, so a harness whose proxy injects credentials never sends one by mistake. Taken from erga's lib/ts/http-client and Stolos's connector SDK (src/archon/connect/sdk.ts, connector.ts).

import { array, object, string } from "@intellectual-frontiers/core";
import { getJson, liveTransport, withBackoff } from "./http-client.ts";
const http = withBackoff(liveTransport());
const repo = await getJson(http, "https://api.github.com/repos/oven-sh/bun", object({ full_name: string }));

Source: commons/net/src/http-client.ts

15 of 48 exports are imported by a test of the run. A test that imports an export may not call it or check what it returns.

httpnamespace
namespace http
http.OutRequesttypeno test imports it
type OutRequest = {
  readonly method: "GET" | "HEAD" | "POST" | "PUT" | "PATCH" | "DELETE";
  readonly url: string;
  readonly headers: Readonly<Record<string, string>>;
  readonly body?: string;
  /** Read the answer as bytes (`Exchange.bytes`), for an artifact that is not text (a tarball, a zip); the text `body` of the answer is then empty. */
  readonly binary?: boolean;
};

A request a client makes.

http.Exchangetypeno test imports it
type Exchange = { readonly method: OutRequest["method"]; readonly url: string; readonly status: number; /** The … (429 characters)
type Exchange = {
  readonly method: OutRequest["method"];
  readonly url: string;
  readonly status: number;
  /** The headers kept: the content type, paging, rate limits and `Retry-After`. */
  readonly headers: Readonly<Record<string, string>>;
  readonly body: string;
  /** The answer's bytes: present where the request asked for `binary` (a recorded exchange of an artifact holds them). */
  readonly bytes?: Uint8Array;
};

One exchange: a request and what answered it.

http.TransportErrortypeno test imports it
type TransportError =
  { readonly kind: "refused"; readonly message: string } | { readonly kind: "network"; readonly message: string };

Why a request got no answer at all (a refusal before it was sent, or the network).

http.Transporttypeno test imports it
type Transport = { readonly kind: "live" | "recorded"; readonly send: (r: OutRequest) => Promise<Result<Exchange, TransportError>> };

A way to reach a system.

http.CREDENTIAL_HEADERSconstno test imports it
const CREDENTIAL_HEADERS: readonly string[]

The headers a credential travels in.

http.carriesCredentialfunctionno test imports it
function carriesCredential(r: OutRequest): string | undefined

Why a request may not be sent where credentials are refused: it carries one in a header or in its query. Undefined where it carries none.

http.LiveOptionstypeno test imports it
type LiveOptions = { /** The `fetch` to use (the platform's by default). */ readonly fetch?: typeof fetch; /** Ref… (465 characters)
type LiveOptions = {
  /** The `fetch` to use (the platform's by default). */
  readonly fetch?: typeof fetch;
  /** Refuse a request that carries a credential (the default), or send it. */
  readonly credentials?: "refuse" | "allow";
  /** The `User-Agent` said to every system: a generic name, never a person's. */
  readonly userAgent?: string;
  /** No answer within this many seconds fails the request (30 by default). */
  readonly timeoutSeconds?: number;
};

How the live transport sends.

http.liveTransportfunctionno test imports it
function liveTransport(options?: LiveOptions): Transport

The live transport, over fetch.

http.recordedTransportfunctionno test imports it
function recordedTransport(exchanges: readonly Exchange[]): Transport

A transport that answers from recorded exchanges, by method and URL, and refuses a request it has none for, naming it.

http.ReadPosttypeno test imports it
type ReadPost = (r: OutRequest) => boolean;

Whether a POST only reads (FR-066): a GraphQL query, a search, a JSON API's Describe and Get actions. It is given the whole request (method, URL, headers and body), as an action may be named in a header (x-amz-target) or the body.

http.readOnlyfunctionno test imports it
function readOnly(t: Transport, tokenEndpoints?: readonly string[], readPost?: ReadPost | undefined): Transport

A transport that sends only safe methods (GET, HEAD, ...), POST to URLs that start with one of tokenEndpoints, and the POSTs that readPost says only read (FR-066). Every other request (PUT, PATCH, DELETE, any other POST) is refused without being sent.

http.SigV4TransportOptionstypeno test imports it
type SigV4TransportOptions = {
  readonly credentials: AwsCredentials;
  readonly region: string;
  readonly service: string;
  /** The clock read as each request is sent (`new Date()` by default; a test gives its own). */
  readonly now?: () => Date;
  /** Sign `UNSIGNED-PAYLOAD` instead of the body's hash (S3 allows it). */
  readonly unsignedPayload?: boolean;
};

How withSigV4 signs.

http.withSigV4functionno test imports it
function withSigV4(t: Transport, options: SigV4TransportOptions): Transport

A transport that signs each request by AWS Signature Version 4 (FR-069, by the kernel's sigv4, FR-068) when it is sent. A signed request carries a credential, so the live transport under it must be made with credentials: "allow", or it refuses the request.

http.BackoffOptionstypeno test imports it
type BackoffOptions = {
  /** Tries after the first (3 by default). */
  readonly retries?: number;
  /** How a wait is spent (a timer by default; a test passes one that returns at once). */
  readonly pause?: (ms: number) => Promise<void>;
  /** The longest wait, whatever `Retry-After` asks (60 s by default). */
  readonly capMs?: number;
};

How long to back off.

http.withBackofffunctionno test imports it
function withBackoff(t: Transport, options?: BackoffOptions): Transport

A transport that waits and tries again on 429 and 503: as long as Retry-After asks (up to the cap), else 1, 2, 4 seconds.

http.ReadErrortypeno test imports it
type ReadError =
  | TransportError
  | { readonly kind: "denied"; readonly status: 401 | 403; readonly url: string }
  | { readonly kind: "status"; readonly status: number; readonly url: string }
  | { readonly kind: "decode"; readonly url: string; readonly message: string };

Why a JSON read failed.

http.formatReadErrorfunctionno test imports it
function formatReadError(e: ReadError): string

A read error as one sentence; never a header or a body, so no credential or personal datum is said.

http.getExchangefunctionno test imports it
function getExchange<T>(t: Transport, url: string, decoder: Decoder<T>, headers?: Readonly<Record<string, string>>): Promise<Result<{ readonly value: T; readonly exchange: Exchange; }, ReadError>>

A GET whose JSON body is decoded, with the exchange; 401 and 403 are denied, any other status from 400 up is status.

http.getJsonfunctionno test imports it
function getJson<T>(t: Transport, url: string, decoder: Decoder<T>, headers?: Readonly<Record<string, string>>): Promise<Result<T, ReadError>>

A GET whose JSON body is decoded.

http.linkPagesfunctionno test imports it
function linkPages<T>(t: Transport, url: string, item: Decoder<T>, options?: { readonly headers?: Readonly<Record<string, string>> | undefined; readonly maxPages?: number | undefined; }): AsyncGenerator<T, Result<number, ReadError>, any>

Every page of a list that pages by Link: <...>; rel="next", at most maxPages (200 by default); the first failure ends it, as an error.

http.cursorPagesfunctionno test imports it
function cursorPages<B, T>(t: Transport, first: string, page: Decoder<B>, items: (body: B) => readonly T[], next: (body: B, url: string) => string | undefined, options?: { readonly headers?: Readonly<Record<string, string>> | undefined; readonly maxPages?: number | undefined; }): AsyncGenerator<T, Result<number, ReadError>, any>

Every item of a list that pages by a cursor in its body: items takes a page's items, next the next page's URL or undefined.

http.decodeRecordedfunctionno test imports it
function decodeRecorded(text: string): Result<Exchange, string>

A recorded exchange read from the JSON5 text recordedText writes, or why it is not one.

http.recordedTextfunctionno test imports it
function recordedText(x: Exchange, note: string): string

The text a recorded exchange is kept in: a comment that says where and when it was recorded, then the exchange as JSON5.

http.bearerfunctionno test imports it
function bearer(token: string, scheme?: string): Readonly<Record<string, string>>

Headers that give a bearer token (for a transport made to allow credentials).

OutRequesttypeno test imports it
type OutRequest = {
  readonly method: "GET" | "HEAD" | "POST" | "PUT" | "PATCH" | "DELETE";
  readonly url: string;
  readonly headers: Readonly<Record<string, string>>;
  readonly body?: string;
  /** Read the answer as bytes (`Exchange.bytes`), for an artifact that is not text (a tarball, a zip); the text `body` of the answer is then empty. */
  readonly binary?: boolean;
};

A request a client makes.

Exchangetype
type Exchange = { readonly method: OutRequest["method"]; readonly url: string; readonly status: number; /** The … (429 characters)
type Exchange = {
  readonly method: OutRequest["method"];
  readonly url: string;
  readonly status: number;
  /** The headers kept: the content type, paging, rate limits and `Retry-After`. */
  readonly headers: Readonly<Record<string, string>>;
  readonly body: string;
  /** The answer's bytes: present where the request asked for `binary` (a recorded exchange of an artifact holds them). */
  readonly bytes?: Uint8Array;
};

One exchange: a request and what answered it.

Imported by 7 tests of the run (7 passed, 0 failed): the library's results

TransportErrortypeno test imports it
type TransportError =
  { readonly kind: "refused"; readonly message: string } | { readonly kind: "network"; readonly message: string };

Why a request got no answer at all (a refusal before it was sent, or the network).

Transporttypeno test imports it
type Transport = { readonly kind: "live" | "recorded"; readonly send: (r: OutRequest) => Promise<Result<Exchange, TransportError>> };

A way to reach a system.

CREDENTIAL_HEADERSconstno test imports it
const CREDENTIAL_HEADERS: readonly string[]

The headers a credential travels in.

carriesCredentialfunction
function carriesCredential(r: OutRequest): string | undefined

Why a request may not be sent where credentials are refused: it carries one in a header or in its query. Undefined where it carries none.

Imported by 7 tests of the run (7 passed, 0 failed): the library's results

LiveOptionstypeno test imports it
type LiveOptions = { /** The `fetch` to use (the platform's by default). */ readonly fetch?: typeof fetch; /** Ref… (465 characters)
type LiveOptions = {
  /** The `fetch` to use (the platform's by default). */
  readonly fetch?: typeof fetch;
  /** Refuse a request that carries a credential (the default), or send it. */
  readonly credentials?: "refuse" | "allow";
  /** The `User-Agent` said to every system: a generic name, never a person's. */
  readonly userAgent?: string;
  /** No answer within this many seconds fails the request (30 by default). */
  readonly timeoutSeconds?: number;
};

How the live transport sends.

liveTransportfunction
function liveTransport(options?: LiveOptions): Transport

The live transport, over fetch.

Imported by 10 tests of the run (10 passed, 0 failed): the library's results

recordedTransportfunction
function recordedTransport(exchanges: readonly Exchange[]): Transport

A transport that answers from recorded exchanges, by method and URL, and refuses a request it has none for, naming it.

Imported by 7 tests of the run (7 passed, 0 failed): the library's results

ReadPosttype
type ReadPost = (r: OutRequest) => boolean;

Whether a POST only reads (FR-066): a GraphQL query, a search, a JSON API's Describe and Get actions. It is given the whole request (method, URL, headers and body), as an action may be named in a header (x-amz-target) or the body.

Imported by 7 tests of the run (7 passed, 0 failed): the library's results

readOnlyfunction
function readOnly(t: Transport, tokenEndpoints?: readonly string[], readPost?: ReadPost | undefined): Transport

A transport that sends only safe methods (GET, HEAD, ...), POST to URLs that start with one of tokenEndpoints, and the POSTs that readPost says only read (FR-066). Every other request (PUT, PATCH, DELETE, any other POST) is refused without being sent.

Imported by 7 tests of the run (7 passed, 0 failed): the library's results

SigV4TransportOptionstypeno test imports it
type SigV4TransportOptions = {
  readonly credentials: AwsCredentials;
  readonly region: string;
  readonly service: string;
  /** The clock read as each request is sent (`new Date()` by default; a test gives its own). */
  readonly now?: () => Date;
  /** Sign `UNSIGNED-PAYLOAD` instead of the body's hash (S3 allows it). */
  readonly unsignedPayload?: boolean;
};

How withSigV4 signs.

withSigV4function
function withSigV4(t: Transport, options: SigV4TransportOptions): Transport

A transport that signs each request by AWS Signature Version 4 (FR-069, by the kernel's sigv4, FR-068) when it is sent. A signed request carries a credential, so the live transport under it must be made with credentials: "allow", or it refuses the request.

Imported by 7 tests of the run (7 passed, 0 failed): the library's results

BackoffOptionstypeno test imports it
type BackoffOptions = {
  /** Tries after the first (3 by default). */
  readonly retries?: number;
  /** How a wait is spent (a timer by default; a test passes one that returns at once). */
  readonly pause?: (ms: number) => Promise<void>;
  /** The longest wait, whatever `Retry-After` asks (60 s by default). */
  readonly capMs?: number;
};

How long to back off.

withBackofffunction
function withBackoff(t: Transport, options?: BackoffOptions): Transport

A transport that waits and tries again on 429 and 503: as long as Retry-After asks (up to the cap), else 1, 2, 4 seconds.

Imported by 7 tests of the run (7 passed, 0 failed): the library's results

ReadErrortypeno test imports it
type ReadError =
  | TransportError
  | { readonly kind: "denied"; readonly status: 401 | 403; readonly url: string }
  | { readonly kind: "status"; readonly status: number; readonly url: string }
  | { readonly kind: "decode"; readonly url: string; readonly message: string };

Why a JSON read failed.

formatReadErrorfunction
function formatReadError(e: ReadError): string

A read error as one sentence; never a header or a body, so no credential or personal datum is said.

Imported by 7 tests of the run (7 passed, 0 failed): the library's results

getExchangefunctionno test imports it
function getExchange<T>(t: Transport, url: string, decoder: Decoder<T>, headers?: Readonly<Record<string, string>>): Promise<Result<{ readonly value: T; readonly exchange: Exchange; }, ReadError>>

A GET whose JSON body is decoded, with the exchange; 401 and 403 are denied, any other status from 400 up is status.

getJsonfunction
function getJson<T>(t: Transport, url: string, decoder: Decoder<T>, headers?: Readonly<Record<string, string>>): Promise<Result<T, ReadError>>

A GET whose JSON body is decoded.

Imported by 7 tests of the run (7 passed, 0 failed): the library's results

linkPagesfunction
function linkPages<T>(t: Transport, url: string, item: Decoder<T>, options?: { readonly headers?: Readonly<Record<string, string>> | undefined; readonly maxPages?: number | undefined; }): AsyncGenerator<T, Result<number, ReadError>, any>

Every page of a list that pages by Link: <...>; rel="next", at most maxPages (200 by default); the first failure ends it, as an error.

Imported by 7 tests of the run (7 passed, 0 failed): the library's results

cursorPagesfunction
function cursorPages<B, T>(t: Transport, first: string, page: Decoder<B>, items: (body: B) => readonly T[], next: (body: B, url: string) => string | undefined, options?: { readonly headers?: Readonly<Record<string, string>> | undefined; readonly maxPages?: number | undefined; }): AsyncGenerator<T, Result<number, ReadError>, any>

Every item of a list that pages by a cursor in its body: items takes a page's items, next the next page's URL or undefined.

Imported by 7 tests of the run (7 passed, 0 failed): the library's results

decodeRecordedfunction
function decodeRecorded(text: string): Result<Exchange, string>

A recorded exchange read from the JSON5 text recordedText writes, or why it is not one.

Imported by 7 tests of the run (7 passed, 0 failed): the library's results

recordedTextfunction
function recordedText(x: Exchange, note: string): string

The text a recorded exchange is kept in: a comment that says where and when it was recorded, then the exchange as JSON5.

Imported by 7 tests of the run (7 passed, 0 failed): the library's results

bearerfunction
function bearer(token: string, scheme?: string): Readonly<Record<string, string>>

Headers that give a bearer token (for a transport made to allow credentials).

Imported by 7 tests of the run (7 passed, 0 failed): the library's results

Titles, headings, exports, tables, columns and requirements. Esc closes.