[0.1 draft] [guide 15 · spec 15 pages]

spec/introduction

Introduction

Text Tables is a line-oriented text format for tables. A file holds one or more tables; each table's rows are ordinary CSV/TSV, and system lines — lines that start with the file's prefix — describe the file and its tables: dialect, column schema, aggregates, metadata, SQL, notes and charts.

A descriptive format (normative)

Text Tables describes data; it does not define or alter it.

  • A table's data rows MUST be valid CSV/TSV in the table's declared dialect. System lines only annotate them.
  • A parser MUST NOT rewrite values from metadata. default= is a schema/DDL attribute and is not substituted on read; null markers read as null.
  • Consistency signals (rows= against the actual row count, default= against nulls, agg-* against the data) are advisory unless a rule says otherwise: a mismatch is a warning and MUST NOT block reading. The exceptions are listed in errors.md (for example, tables-mismatch is always an error).
  • A file asserts its own data. Aggregates, row counts and other derived facts describe exactly the rows the file declares — see aggregates.md.
  • Rewriting data (materializing a computed column, slicing, filtering) is an explicit tool operation, never a side effect of reading.

Forms

Form Extension MIME type
Text .txtab text/txtab
Archive .txtab.zip application/txtab+zip

A table's data can live in three places, declared per table (see tables.md and archive.md):

  • inline — rows in the same file, after the table's block;
  • external rows — an ordinary CSV/TSV file named by reference=;
  • external columns — a directory of .col files, one per column (storage=columns), normally inside a .txtab.zip.

The vocabulary is the same everywhere; only the packaging differs. There is no JSON form.

Conformance

  • A parser reads Text Tables files and reports the errors and warnings defined in errors.md. A conforming parser MUST reject a file on any FAIL-level error and MUST NOT reject it on a WARN-level one.
  • A writer produces Text Tables files. A conforming writer MUST produce files that a conforming parser reads without FAIL-level errors, and SHOULD produce files without warnings.
  • A validator (txtab verify and similar) is a parser that additionally escalates the warnings marked "→ FAIL" in errors.md.

Text Tables is not ExCSV. A conforming parser MUST NOT treat an ExCSV file (#!excsv …) as Text Tables; such a file is plain CSV/TSV from its point of view (file-structure.md).

Terminology

The key words MUST, MUST NOT, REQUIRED, SHOULD, SHOULD NOT and MAY are to be interpreted as described in RFC 2119.

Term Meaning
prefix (P) The characters before the first space of the first line; starts every system line
system line A line that starts with P followed by a space, or that is exactly P
first line Line 1: P texttables=<version> …
keyword The first token after P on a system line, e.g. table, column, agg-sum
comment A system line whose first token is --
file block System lines between the first line and the first P table
table block System lines from a P table line up to the table's data (or the next P table)
data section A table's inline rows: the header row (when header=1) followed by rows data rows
record One line of a data section

Prior work

Text Tables is a new take on ExCSV 0.6, which built on Astropy ECSV, InfluxDB Annotated CSV, MetaCSV, CSVW, CSVY and Frictionless Table Schema. It does not read ExCSV files.

spec/introduction.md on GitHub