guide/prior-art
Prior art
Text Tables is a new take on ExCSV 0.6, which in turn stood on the shoulders of earlier work showing that a CSV can carry its own metadata — inline or next to it — without ceasing to be CSV. This page lists the ideas we borrowed, the ones we left out on purpose, and the ones still missing.
✅ have · ◐ partly · ❌ not yet · 🚫 left out on purpose
Metadata for CSV
| Source | Idea | Text Tables |
|---|---|---|
| Astropy ECSV | Versioned signature line, #-prefixed metadata, column types and units |
✅ first line, column type/unit/format/description, meta-* |
| Metadata as YAML | 🚫 line-oriented key=value — easier to grep, diff and hand-write |
|
| Masked and multidimensional columns, Python object serialization | 🚫 an interchange format, not an object serializer | |
| InfluxDB Annotated CSV | Annotation rows with one value per column | ✅ agg-* |
#group |
◐ role=dimension |
|
#default applied when reading |
🚫 default= is a DDL fact; an empty cell is null |
|
| MetaCSV | Metadata in a sidecar file | ✅ reference= — and one file can describe many data files |
Locale-aware types (boolean/vrai/faux, decimal comma, currency symbols) |
◐ format= and ISO 4217 in unit=; no locale parameters |
|
| CSVW (W3C) | Detached JSON-LD schema: types, required, foreign keys |
✅ in our own vocabulary; references= for foreign keys |
lang per column |
❌ | |
Discovery via HTTP Link headers |
❌ | |
| CSVY | YAML front matter above the CSV | 🚫 not skippable by CSV tools |
| Frictionless Table Schema / Data Package | Constraints: enum, min/max, pattern, unique |
✅ |
primaryKey, foreignKeys, multi-resource packages |
✅ references=, several tables per file; primary keys via SQL |
|
dbt schema.yml |
not_null, unique, accepted_values, relationships |
✅ same vocabulary — required, unique, enum, references |
| Tests executed against a warehouse | 🚫 descriptive hints, no query engine | |
| Croissant (MLCommons) | ML dataset description: distributions, record sets, splits | ◐ meta-license; no splits — a different level, complementary |
Several tables and other separators
| Source | Idea | Text Tables |
|---|---|---|
| CSV Next | Comments, types, groups | ✅ -- comments and column; ◐ groups |
| CSVX | A strict CSV subset, naming rules | ◐ column and table name rules; no strict profile |
| MTSV (IETF draft) | Several named sheets in one file | ✅ table with row counts and tables= |
| Multiple-TSV | Runs of tabs as one separator | ❌ |
| PTT (Rohde & Schwarz) | Comments vs directives; several lists per file | ✅ configurable prefix, -- vs keywords, several tables |
workspace.sh .table |
Typed columns, relations, packaging options | ✅ types, references=, .txtab.zip with row or column storage |
| ASV / USV | Control characters / Unicode marks as separators | ◐ any literal delim=; no named ASV/USV delimiters, no group/file levels |
| Perl Text::CSV | Arbitrary delimiter, quote and escape characters | ◐ delim and quote; no separate escape character |
| Miller | Conversion between CSV/TSV/ASV/USV | ◐ tools convert CSV/TSV |
What Text Tables adds
- A configurable prefix, declared by the first line — so the data can contain anything, and legacy tools still skip system lines.
- Several tables per file, read by row count, with a strict table count that catches truncation.
- One file describing many external CSV/TSV files, each with its own dialect and encoding.
- Pre-computed aggregates as facts about exactly the rows declared.
- SQL in
pg_dumpsections — pre-data and post-data — tagged per line with a dialect. - Computed columns,
role=/agg=analytical hints, money with per-row currencies. - Notes, links and charts that never touch the data.
- One archive form with row or column storage and a schema preview in the ZIP comment.
Left out of 0.1
- A JSON form (ExCSV had one) — Text Tables is text.
- Checksums — deferred until it's clear what to hash for column storage.
- Byte-offset indexes for parallel reads.