spec/file-structure
File structure and reading
Layout
A Text Tables file consists of, in order:
text
┌──────────────────────────────┐
│ First line │ P texttables=0.1 … exactly one
├──────────────────────────────┤
│ File block │ system lines zero or more
├──────────────────────────────┤
│ Table 1 │
│ table block │ P table … + system lines
│ data section │ inline rows (if any)
├──────────────────────────────┤
│ Table 2 … │ zero or more further tables
└──────────────────────────────┘- The first line MUST be line 1 (first-line.md).
- Every table MUST start with a
P tableline (tables.md). There is no implicit table. - A file with no
P tablelines is valid only withtables=0(an empty stub).
Recognition
- Detect the encoding from a BOM, if present, and strip the BOM (Encoding).
- Take line 1. It MUST be ASCII (
first-line-not-ascii, FAIL). Let P be every character before the first space (U+0020). - The file is Text Tables if and only if line 1 contains a space and the characters right after the first space start with
texttables=.
A file that does not match is not a Text Tables document — including an empty file and an ExCSV file. A tool MAY still open it as plain CSV/TSV, but that is outside this specification and no Text Tables rule applies to it.
The prefix
- P MUST be 1–8 characters long.
- P MUST consist of printable ASCII characters (U+0021–U+007E): no whitespace, no control characters.
- The first character of P MUST NOT be a letter or a digit.
- P MUST NOT contain the delimiter or the quote character of the file, or of any table that overrides them.
- Violations are
prefix-invalid(FAIL).
Writers SHOULD use #. Legacy CSV readers skip comment lines by a single character (pandas comment=, R comment.char, DuckDB comment), so a single-table file is readable by them only when they are told the first character of P.
Line classes
Every line of the file, other than records of a data section, is one of:
| Class | Rule |
|---|---|
| first line | line 1 |
| system line | starts with P followed by a space, or is exactly P |
| comment | a system line whose first token after P is -- |
| directive | any other system line; its first token is the keyword (lines.md) |
A line that is exactly P is an empty comment.
Reading
A parser reads the file top to bottom:
- Read the first line (first-line.md).
- Read system lines into the file block until the first
P tableline. - For each
P tableline: read the following system lines into the table block. The first line that is not a system line starts the table's data section. - A table with inline data reads its data section by count: the header row when
header=1, then exactlyrowsrecords (tables.md). - After the last record of a table, the next line MUST be a
P tableline, a comment, or end of file. Directives other thantableare not allowed after a data section (directive-after-data, FAIL). - A table with
reference=has no inline data: its block ends at the nextP tableline or end of file. A line that is not a system line there isdata-in-reference-table(FAIL).
Count mismatch and recovery
Records are read by count, not by looking at the prefix, so a record may begin with any characters. When the count is wrong:
- Fewer records than
rows— aP tableline, or end of file, is reached beforerowsrecords were read. The parser MUST treat theP tableline as the start of the next table, keep the records read so far, and reportrows-mismatch(WARN). - More records than
rows— afterrowsrecords the next line is not a system line. The parser MUST keep reading records until aP tableline or end of file and reportrows-mismatch(WARN).
Recovery relies on data records never looking like a P table line. Writers MUST choose P so that no record of any data section starts with P followed by a space or equals P. This is what a configurable prefix is for: if the data contains lines starting with # , choose another prefix. A parser that reads such a record by count reports record-starts-with-prefix (WARN).
Data section
- Records MUST follow the table's effective dialect (
delim,quote,null,header). - Values MUST NOT contain raw line breaks: one record is one line.
- With
quote=none, values MUST NOT contain the delimiter. - An empty line inside a data section is a record with one empty field.
- A trailing line break after the last line of the file is optional.
Line endings
Lines end with LF or CRLF. Parsers MUST accept both, including mixed.
Encoding
BOM. If the file starts with a BOM, the BOM decides the encoding and is stripped:
Bytes Encoding 00 00 FE FFUTF-32BE FF FE 00 00UTF-32LE (checked before UTF-16LE) FE FFUTF-16BE FF FEUTF-16LE EF BB BFUTF-8 No BOM. The file is in an ASCII-compatible encoding (UTF-8, ISO-8859-1, Windows-1251, …). The first line is read as ASCII;
encoding=on the first line (defaultUTF-8) applies to the rest of the file.UTF-16 and UTF-32 files MUST start with a BOM. Without one, line 1 does not read as
P texttables=and the file is not Text Tables.An
encoding=that contradicts the BOM isencoding-mismatch(WARN); the BOM wins. An unknown encoding name isencoding-unknown(WARN); UTF-8 is used. Bytes that do not decode areencoding-invalid-bytes(FAIL).A table MAY declare its own
encoding=for its data (tables.md). System lines are always in the file's encoding.
Writers SHOULD write UTF-8; a writer targeting Excel MAY add a UTF-8 BOM.
Compatibility with CSV/TSV tools
A single-table file with inline data is a valid CSV/TSV file for any tool that skips lines starting with the first character of P. A multi-table file read by such a tool concatenates all tables' data and reads the second table's header row as a data record.