IF Commons GitHub

store

Written to: 6 requirements: 6 proved, 0 partly, 0 failing, 0 with no test recorded. 0002-anyfs-s3-store FR-009, 0002-anyfs-s3-store FR-010, 0002-anyfs-s3-store FR-011, 0002-anyfs-s3-store FR-012, 0002-anyfs-s3-store FR-013, 0002-anyfs-s3-store FR-014

The store over a bucket and a prefix of any S3-compatible service. It holds whole objects (up to maxBytes, 64 MiB where it is left out; multipart upload is a later capability and a larger object is refused), and it says what it is by what it was found to keep: the probe of the endpoint decides whether it has cas, and a store that cannot keep a condition is made without it, in its type, so that code which needs compare-and-write cannot be given one. A write expecting no object is If-None-Match: * and one expecting a revision is If-Match on the entity tag, so that of several writers exactly one wins; where the endpoint ignores If-None-Match the store refuses to write expecting no object, and says why, and never lets two writers both think they made it.

The revision of an object is, by default, sha256: and the digest of its bytes, kept in the metadata x-amz-meta-anyfs-sha256 and checked on every read (and, where the service supports it, sent as x-amz-checksum-sha256 so that the service checks the bytes it received): the same bytes are the same revision here as in memory, on a disk or on a host, and a bridge compares them without reading them. An object written by another tool has none, and its revision is the digest of what a read gives. With revisions: "etag" the revision is etag: and the entity tag, which costs no read and no request for a listing, and is the same only between stores of S3.

Limits, said once: a read keeps the whole object in memory; the metadata of an object is trusted by stat and a listing, and checked only by a read; a listing in the default mode asks for each object it lists (a few at a time, listConcurrency) because a listing does not carry the metadata; a rename, a batch, a history, a range and an append are not here; a call is not retried here, and the layer retrying does it for the problems that say they are worth one.

import { openS3Store } from "./store.ts";
// const opened = await openS3Store({ place, credentials, endpoint: "http://127.0.0.1:9000" });
// opened.kind === "ok" && opened.value.mode; // "cas"

Source: commons/anyfs-s3/src/store.ts

4 of 12 exports are imported by a test of the run. A test that imports an export may not call it or check what it returns.

CAS_CAPABILITIESconst
const CAS_CAPABILITIES: readonly ["read", "write", "list", "stat", "delete", "cas"]

The capabilities of a store of an endpoint that keeps every condition.

Imported by 17 tests of the run (17 passed, 0 failed): the library's results

openS3Storefunction
function openS3Store(options: S3Options): Promise<Result<Opened, string>>

A store of S3, or why there is none. The probe runs once, here, and the store is made as what the probe found.

Imported by 17 tests of the run (17 passed, 0 failed): the library's results

Openedtypeno test imports it
type Opened =
  | { readonly mode: "cas"; readonly store: S3CasStore; readonly facts: S3Facts; readonly client: S3Client }
  | { readonly mode: "plain"; readonly store: S3PlainStore; readonly facts: S3Facts; readonly client: S3Client }
  | { readonly mode: "read-only"; readonly store: S3ReadStore; readonly facts: S3Facts; readonly client: S3Client };

A store of S3 and what it was found to be: the mode says which capabilities it has, and the type follows the mode.

PLAIN_CAPABILITIESconst
const PLAIN_CAPABILITIES: readonly ["read", "write", "list", "stat", "delete"]

The capabilities of a store of an endpoint that does not keep every condition.

Imported by 17 tests of the run (17 passed, 0 failed): the library's results

READ_CAPABILITIESconst
const READ_CAPABILITIES: readonly ["read", "list", "stat"]

The capabilities of a store that can only read.

Imported by 17 tests of the run (17 passed, 0 failed): the library's results

RevisionModetypeno test imports it
type RevisionMode = "sha256" | "etag";

What a revision is made of: the digest of the bytes, or the entity tag.

S3CasStoretypeno test imports it
type S3CasStore = Store<(typeof CAS_CAPABILITIES)[number]>;

A store with compare-and-write.

S3Factstypeno test imports it
type S3Facts = {
  readonly probe: Facts;
  readonly revisions: RevisionMode;
  /** What a bridge may take a revision for: the digest of the bytes, or an entity tag. */
  readonly digest: "sha256-native" | "etag";
  readonly endpoint: Endpoint;
  readonly addressing: Addressing;
};

What the store was found to be.

S3Optionstypeno test imports it
type S3Options = { /** The bucket and the prefix: `parsePlace("s3://bucket/prefix")`. */ readonly place: S3Place; … (1095 characters)
type S3Options = {
  /** The bucket and the prefix: `parsePlace("s3://bucket/prefix")`. */
  readonly place: S3Place;
  readonly credentials: CredentialSource;
  /** Where the service listens. Amazon S3 in the region where it is left out. */
  readonly endpoint?: string;
  /** The region of the signature. `us-east-1` where it is left out. */
  readonly region?: string;
  /** How the bucket is written in a URL: virtual-hosted style for Amazon S3 with a bucket that has no dot, path style otherwise. */
  readonly addressing?: Addressing;
  /** What a revision is. `sha256` where it is left out. */
  readonly revisions?: RevisionMode;
  /** The most bytes an object may have. 64 MiB where it is left out. */
  readonly maxBytes?: number;
  readonly transport?: Transport;
  /** How long a request may take. 30 s where it is left out. */
  readonly timeoutMs?: number;
  readonly payload?: PayloadSigning;
  /** How many objects a listing asks about at once. 16 where it is left out. */
  readonly listConcurrency?: number;
  readonly consistency?: Consistency;
  readonly now?: () => Date;
};

What a store is made with.

S3Placetypeno test imports it
type S3Place = Extract<Place, { readonly kind: "s3" }>;

The place of a store of S3.

S3PlainStoretypeno test imports it
type S3PlainStore = Store<(typeof PLAIN_CAPABILITIES)[number]>;

A store without compare-and-write.

S3ReadStoretypeno test imports it
type S3ReadStore = Store<(typeof READ_CAPABILITIES)[number]>;

A store that can only read.

Titles, headings, exports, tables, columns and requirements. Esc closes.