[0.1 draft] [guide 15 · spec 15 pages]

spec/tables

Tables (table)

txtab

P table name=orders rows=2

Every table, including a single one, starts with a P table line. The lines that follow, up to the table's data or the next P table, are the table's block.

Keys

Key Requirement Default Meaning
rows MUST — Number of data records, not counting the header row
header MAY 1 1: the data starts with a header row; 0: it does not (below)
name MAY — Table name; unique within the file
columns MAY — Physical column count: stored columns plus materialized computed columns
reference MAY — External data: a file (orders.csv) or a directory (orders/)
storage MAY rows rows or columns (archive.md)
section-size MAY, only with storage=columns — Records per section of a column-stored table (archive.md)
data-size MAY — Approximate unpacked size of the table's data in bytes, see below
delim, quote, null MAY from the first line Override the file's defaults for this table (first-line.md)
encoding MAY the file's encoding Encoding of this table's data, see below

texttables and tables are file keys; on P table they are key-misplaced (WARN, ignored).

rows

  • Required on every P table (table-missing-rows, FAIL).
  • A non-negative integer: the number of data records, excluding the header row.
  • For an inline table the parser reads exactly rows records (file-structure.md). For a reference= table, rows describes the external data.
  • A mismatch with the actual count is rows-mismatch: WARN on read, FAIL in verify.
  • 1 — the table's data starts with a header row of column names; 0 — it does not. Any other value is value-invalid (FAIL).
  • Default 1.
  • A table key only (decision 0015): it describes this table's data, not the file. On the first line it is key-misplaced (WARN, ignored).
  • For an inline table the data section is the header row (when header=1), then rows records. For a reference= table it says whether the external file starts with a header row.
  • It decides how column lines bind to the data: by header cell with header=1, by index= with header=0 (columns.md).

name

  • Optional, also when the file has several tables.
  • If present: MUST match [A-Za-z_][A-Za-z0-9_]* (table-name-invalid, FAIL) and MUST be unique in the file (table-name-duplicate, FAIL). Names are user data and are not subject to kebab-case.
  • A table without a name is addressed by its zero-based position among the file's tables. A table that is the target of references= MUST have a name (columns.md).

columns

The physical column count: stored columns plus computed columns with materialized=1; virtual computed columns do not count (columns.md). A mismatch with the data is columns-mismatch: WARN on read, FAIL in verify.

reference — external data

  • reference= names the table's data outside this file: a CSV/TSV file, or with storage=columns a directory of .col files (archive.md).
  • The path is relative to the directory of the file that contains the P table line. It MUST NOT be absolute and MUST NOT escape that directory with .. (reference-escapes-dir, FAIL). Inside an archive it resolves against archive entries (archive.md).
  • A reference= table has no data section in this file. A record in its block is data-in-reference-table (FAIL).
  • The external file is ordinary CSV/TSV in the table's effective dialect, with no Text Tables lines. rows, columns, data-size and agg-* describe it.
  • Parsing reads only this file and always succeeds without the external data. Opening the table's data resolves reference=; if the target is missing, the parser MUST report reference-not-found (WARN) and treat the table as metadata-only: reading rows, verifying counts and aggregates, and materializing are unavailable.
  • Tools MUST NOT modify a referenced file as a side effect of any operation on this file. Operations that change data (for example materializing a computed column) produce a new inline table or a new file.
  • Inline tables and reference= tables MAY be mixed in one file, and one file MAY describe several external files.
  • Discovery: when opening sales.csv, a tool MAY look for sales.txtab in the same directory and use the table whose reference= names sales.csv.

data-size

  • Optional and approximate: the number of bytes the table's data occupies on disk when unpacked — for an inline table, its data section (header row, records, line breaks); for reference=<file>, that file; for storage=columns, the sum of its .col files.
  • The value depends on storage; tools that convert a table between storages recompute or drop it.
  • For estimates only (disk, memory, progress, choice of reading strategy). Parsers MUST NOT report a mismatch.

Dialect overrides

delim, quote and null on P table override the first line's values for this table only. A table's effective dialect is the first line's values with its own overrides applied, plus its own header. One file can therefore describe CSV and TSV tables together:

txtab

# texttables=0.1 tables=2
# table name=orders reference=orders.csv rows=48213 delim=comma
# table name=events reference=events.tsv rows=1200 delim=tab

encoding

  • Applies to the table's data: inline records, the reference= file, or the .col files. The table's system lines are in the file's encoding.
  • Default: the file's encoding.
  • A referenced file or .col file that starts with a BOM: the BOM wins (encoding-mismatch, WARN if it contradicts encoding=), and the BOM is stripped.
  • An inline table MAY declare an encoding different from the file's only if both are ASCII-compatible; otherwise encoding-mixed-inline (FAIL): a UTF-16 data section cannot be split into lines inside a UTF-8 file, or the reverse.
  • Typical use: one file describing external CSV files exported by different systems — a UTF-8 file from a web service next to a Windows-1251 file from an accounting system.

Number of tables

The first line's tables= is strict (first-line.md): the file MUST contain exactly that many P table lines.

spec/tables.md on GitHub