IF Commons GitHub

archive

Written to: 2 requirements: 2 proved, 0 partly, 0 failing, 0 with no test recorded. 0001-formats FR-010, 0001-formats FR-023

Archives in memory, with no outside package (commons 0001-formats FR-010). Writing: a POSIX ustar archive, a gzip stream of stored deflate blocks and a zip of stored entries (zip), each entry with a fixed time and mode and in the byte order of its path, so the bytes depend on the files alone (taken from erga's lib/ts/archive); and, where size matters more than the same bytes on every runtime, a zip of deflated entries (zipDeflated, through the web standard CompressionStream), the same bytes for the same files on one runtime and the same files unzipped on every runtime. Reading: a tar (ustar, pax and GNU long names), a gzip and a zip whose entries are stored or deflated (through the web standard DecompressionStream), each checked (header checksums, CRC-32) and every path refused that would leave the folder it is unpacked into. This is what lets Stolos drop fflate and unpack a verified copy of its engine. A tar, gzipped or not, is also read as a stream (untarStream, readTarStream), an entry at a time with each entry's bytes a stream of their own, with the same checks, so that an archive larger than memory can be read (FR-059).

import { tar, untar, zip, unzip } from "./archive.ts";
const files = new Map([["a/b.txt", new TextEncoder().encode("hello")]]);
const z = zip(files);
const back = await unzip(z); // { kind: "ok", value: Map { "a/b.txt" => Uint8Array [104, 101, 108, 108, 111] } }

Source: commons/formats/src/archive.ts

18 of 31 exports are imported by a test of the run. A test that imports an export may not call it or check what it returns.

ArchiveFailureclass
class ArchiveFailure extends Error

An archive error raised as an Error, which is how a stream reports one: the error itself is in error.

MemberTypeWhat it is
errorArchiveErrorWhy the archive could not be read.
namestring
messagestring
stack?string
cause?unknown

Imported by 14 tests of the run (14 passed, 0 failed): the library's results

concatfunctionno test imports it
function concat(parts: readonly Uint8Array<ArrayBufferLike>[]): Uint8Array<ArrayBufferLike>

Byte arrays joined end to end.

crc32function
function crc32(bytes: Uint8Array<ArrayBufferLike>): number

The CRC-32 (ISO 3309) that gzip and zip both carry.

Imported by 31 tests of the run (31 passed, 0 failed): the library's results

deflateRawfunctionno test imports it
function deflateRaw(data: Uint8Array<ArrayBufferLike>): Promise<Uint8Array<ArrayBufferLike>>

The raw deflate stream (RFC 1951) a web CompressionStream gives for the bytes.

dosDatefunctionno test imports it
function dosDate(year: number, month: number, day: number): number

A date as a zip writes it (MS-DOS): (year - 1980) << 9 | month << 5 | day.

filesOffunction
function filesOf(entries: readonly TarEntry[]): Files

The files of a list of tar entries, by path: folders and links left out.

Imported by 19 tests of the run (19 passed, 0 failed): the library's results

formatArchiveErrorfunction
function formatArchiveError(e: ArchiveError): string

An archive error as one sentence.

Imported by 19 tests of the run (19 passed, 0 failed): the library's results

gunzipfunction
function gunzip(data: Uint8Array<ArrayBufferLike>): Promise<Result<Uint8Array<ArrayBufferLike>, ArchiveError>>

The bytes of a gzip stream, stored or deflated, its CRC checked by the decompressor.

Imported by 14 tests of the run (14 passed, 0 failed): the library's results

gzipfunction
function gzip(data: Uint8Array<ArrayBufferLike>): Uint8Array<ArrayBufferLike>

A gzip stream (RFC 1952) of the bytes in stored deflate blocks (RFC 1951 §3.2.4), with no name and no time: the same bytes every time.

Imported by 14 tests of the run (14 passed, 0 failed): the library's results

isSafePathfunction
function isSafePath(path: string): boolean

Whether a path inside an archive stays inside the folder it is unpacked into: relative, with / between its parts, no part .., no empty part but a trailing one, no backslash, no drive letter and no NUL.

Imported by 14 tests of the run (14 passed, 0 failed): the library's results

linksOffunction
function linksOf(entries: readonly TarEntry[]): readonly TarLink[]

The links of a list of tar entries, in the order of the archive, so that a consumer decides what to do with each and finds none among the files.

Imported by 14 tests of the run (14 passed, 0 failed): the library's results

readTarStreamfunctionno test imports it
function readTarStream(stream: ReadableStream<Uint8Array<ArrayBufferLike>>): ReadableStream<TarStreamEntry>

The entries of a tar archive given as a stream of bytes, gzipped or not (told by its first two bytes), one at a time as they are asked for, so that an archive larger than memory is read with one chunk held (commons 0001-formats FR-011). Every header is checked as untar checks it (checksum, pax and GNU long names, a safe path); a failure errors the stream with an ArchiveFailure. Each entry's readable MUST be read to its end or cancelled before the next entry is given; cancelling the stream of entries cancels the bytes.

sortedPathsfunction
function sortedPaths(paths: Iterable<string>): readonly string[]

The paths of the files in the byte order of their UTF-8, the order every writer here uses.

Imported by 14 tests of the run (14 passed, 0 failed): the library's results

splitNamefunction
function splitName(path: string): readonly [string, string] | undefined

A path too long for the 100 bytes of a ustar name, split into its prefix (up to 155 bytes) and name; undefined where it cannot be.

Imported by 14 tests of the run (14 passed, 0 failed): the library's results

tarfunction
function tar(files: Files): Result<Uint8Array<ArrayBufferLike>, ArchiveError>

A ustar archive of the files, each folder they lie in first given its own entry; or the path that does not fit or is not safe.

Imported by 19 tests of the run (19 passed, 0 failed): the library's results

tarHeaderfunctionno test imports it
function tarHeader(path: string, size: number, dir: boolean): Uint8Array<ArrayBufferLike> | undefined

One ustar header block of a file (mode 0644) or a folder (0755), owned by user 0, dated 0; undefined where the path does not fit.

untarfunction
function untar(data: Uint8Array<ArrayBufferLike>): Result<readonly TarEntry[], ArchiveError>

The entries of a tar archive (ustar, with pax extended headers and GNU long names), each header's checksum checked and each path checked safe.

Imported by 19 tests of the run (19 passed, 0 failed): the library's results

untarStreamfunction
function untarStream(): ReadableWritablePair<TarStreamEntry, Uint8Array<ArrayBufferLike>>

A stream from a tar archive's bytes, gzipped or not, to its entries, to put in a pipeThrough as @std/tar's UntarStream is: see readTarStream for what it checks and how each entry's body is read (commons 0001-formats FR-011).

for await (const entry of file.readable.pipeThrough(untarStream())) {
  if (entry.path === "bin/tool") await save(entry.readable);
  else await entry.readable.cancel();
}

Imported by 14 tests of the run (14 passed, 0 failed): the library's results

unzipfunction
function unzip(data: Uint8Array<ArrayBufferLike>): Promise<Result<Files, ArchiveError>>

Every file of a zip by its path, folders left out, each read and checked, and every path checked safe.

Imported by 14 tests of the run (14 passed, 0 failed): the library's results

zipfunction
function zip(files: Files, date?: number): Uint8Array<ArrayBufferLike>

A zip (APPNOTE 6.3) of the files, each stored, dated date (1980-01-01 unless another is given) at midnight and readable by everyone.

Imported by 14 tests of the run (14 passed, 0 failed): the library's results

ZIP_EPOCHconstno test imports it
const ZIP_EPOCH: number

The zip's fixed date: 1980-01-01, the first a zip can hold.

zipDeflatedfunction
function zipDeflated(files: Files, date?: number): Promise<Uint8Array<ArrayBufferLike>>

A zip (APPNOTE 6.3) of the files as zip lays them out, but each entry deflated (method 8) through the web standard CompressionStream("deflate-raw"), or stored where deflating would not make it smaller (an empty or incompressible file). The bytes depend on the files alone on one runtime; another runtime's deflater may choose other (equally valid) bytes, so only what they unzip to is the same everywhere.

Imported by 14 tests of the run (14 passed, 0 failed): the library's results

zipEntriesfunction
function zipEntries(data: Uint8Array<ArrayBufferLike>): Result<readonly ZipEntry[], ArchiveError>

The entries of a zip, from its central directory; or why they cannot be read (a ZIP64 archive among the reasons).

Imported by 14 tests of the run (14 passed, 0 failed): the library's results

zipReadfunctionno test imports it
function zipRead(data: Uint8Array<ArrayBufferLike>, e: ZipEntry): Promise<Result<Uint8Array<ArrayBufferLike>, ArchiveError>>

The bytes of one entry of a zip, stored (method 0) or deflated (method 8), its size and CRC-32 checked.

ArchiveErrortype
type ArchiveError =
  | { readonly kind: "damaged"; readonly message: string }
  | { readonly kind: "unsupported"; readonly message: string }
  | { readonly kind: "unsafe-path"; readonly path: string }
  | { readonly kind: "too-long"; readonly path: string };

Why an archive could not be read or written.

Imported by 14 tests of the run (14 passed, 0 failed): the library's results

Filestypeno test imports it
type Files = ReadonlyMap<string, Uint8Array>;

The files of an archive, by their path inside it.

TarEntrytypeno test imports it
type TarEntry = {
  readonly path: string;
  readonly kind: "file" | "directory" | "symlink" | "hardlink";
  readonly mode: number;
  readonly data: Uint8Array;
  readonly linkName: string;
};

One entry of a tar archive. A folder's path has no trailing /; a link's target is in linkName.

TarHeadertypeno test imports it
type TarHeader = {
  readonly path: string;
  readonly kind: TarEntry["kind"];
  readonly mode: number;
  readonly size: number;
  /** Seconds since 1970, as the header (or a pax `mtime`) gives them. */
  readonly mtime: number;
  readonly linkName: string;
};

The header of one entry of a tar archive: its path (a folder's with no trailing /), kind, mode, size of body, time and link target.

TarLinktypeno test imports it
type TarLink =
  | { readonly kind: "symlink"; readonly path: string; readonly target: string }
  | { readonly kind: "hardlink"; readonly path: string; readonly target: string };

One link of a tar: a symbolic link names a path (relative to the link's own folder, or absolute), a hard link names the path of another entry of the archive.

TarStreamEntrytypeno test imports it
type TarStreamEntry = TarHeader & { readonly readable: ReadableStream<Uint8Array> };

One entry of a tar archive read as a stream: its header and its body as a stream (empty but for a file). The body MUST be read to its end or cancelled before the next entry is given, as with @std/tar's UntarStream.

ZipEntrytypeno test imports it
type ZipEntry = {
  readonly name: string;
  readonly method: number;
  readonly crc: number;
  readonly compressed: number;
  readonly size: number;
  readonly offset: number;
};

One entry of a zip's central directory.

Titles, headings, exports, tables, columns and requirements. Esc closes.