What is actually built

Every row below was derived from the code or measured on this machine, with the file and line or the command that produced it. Nothing here is quoted from another document. It exists because the design documents in this repository have different provenance and different trustworthiness, and that was not written down anywhere.

0. Provenance of the documents

Document First committed Origin
plan/txz-spec.md 2026-02-26 M. Chatterjee’s notes, 2001–2018, consolidated. Original, but explicitly a draft to start a conversation — not hardened.
dev/bitfile.h, dev/VBEDict 2026-02-26 Original dio_BitFile lineage, 2000–2001
docs/triepack-northstar.md 2026-09-18 Generated by Claude, two days ago, at the author’s request to articulate v2 goals. A restatement, not a source.
docs/status.md 2026-09-18 Generated by Claude, same commit
docs/internals/format-spec-v2.md 2026-09-18 Generated by Claude, same commit. A proposal.
docs/internals/format-spec-v2-draft2.md 2026-09-20 Generated by Claude. A proposal, under review.

Reproduce: git log --diff-filter=A --format='%ad %h %s' --date=short -- <file>

The three 2026-09-18 documents are not design authority and should not be cited as if they record original intent. They were written during a session about what v2 should be, and then quoted back as though they were what the project had decided. Only the 2026-02 material predates the implementation.

1. The stated intent, tested against the code

“a well designed bit aligned trie with skips and support for suffix tries and arbitrary (binary or ascii or utf8) sortable keys and values. Clever use of tries and bits was meant to make them compact and ROMmable (read in place).”

# Claim Built? Evidence
1 Bit-aligned trie Yes Symbols are bits_per_symbol-wide fields packed with no padding, width chosen per dictionary (encoder.c:265). src/bitstream/ is 1,028 lines implementing the GetNBitQty/PutNBitQty primitives.
2 Skips Yes TP_CTRL_SKIP is emitted by the encoder and read by the decoder. This is what makes lookup O(key length) without decompressing.
3 Suffix tries No src/core/suffix.c is 14 lines ending in /* TODO: suffix table implementation */. TP_CTRL_SUFFIX appears only as a #define — never emitted, never read. The header flag has_suffix_table is never set.
4 Arbitrary binary keys No TP_MAX_ALPHABET_SIZE 249. Measured: 249 distinct byte values builds and every key reads back; 250 returns TP_ERR_ALPHABET. Keys spanning all 256 values are refused.
5 UTF-8 keys Yes, incidentally UTF-8 text works because it is bytes, and most single-language text stays under 249 distinct values. The declared UTF-8 symbol mode does not exist: TP_ADDR_SYMBOL_UTF8 is read by 0 files in src/.
6 Sortable keys Yes tp_dict_iterate yields keys in lexicographic byte order; tp_dict_find_prefix walks a subtree.
7 Values Yes, with a serious defect Typed values work (null, bool, int, uint, float32/64, string, blob). But each terminal stores a varint index into a flat store, so resolving one value decodes every preceding one: 328 µs at 10k keys, 30,500 µs at 699k. Full decode is O(n²).
8 Compact Partly Keys-only, measured: 71,824 bytes of keys → 45,054 bytes, 0.627 of raw. With values it gets worse than the input, because every key pays a varint index on top of its value.
9 ROMmable (read in place) Yes, for lookup tp_dict_lookup_n contains zero allocations. tp_dict_open allocates one control struct; iteration allocates an iterator and a growable key buffer. Lookup itself runs from the buffer you hand it.

Five of nine as intended, two partly, two not.

The two that are not — suffix tries and arbitrary binary keys — are the two that most distinguish the format from an ordinary prefix trie, and they are related: SUFFIX and ESCAPE reserve two of the six control codes, and those six are what reduce the alphabet from 255 to 249.

2. Reproducing the measurements

# claim 4 — the alphabet boundary
#   build a dictionary with n distinct byte values for n = 248..250
# claim 7 — value lookup latency
ctest --test-dir build -R test_scale --output-on-failure
# claim 8 — keys-only compression
#   encode tests/data/common_words_10k.txt with TP_NULL values
# claim 9 — allocation in the lookup path
awk '/^tp_result tp_dict_lookup_n/,/^}/' src/core/decoder.c | grep -cE 'malloc|calloc|realloc'

3. Declared but absent

Each is reachable from a public header or an original-draft section, and nothing implements it.

Thing Where it is named Reality
Suffix table TP_CTRL_SUFFIX, has_suffix_table, suffix.c 14-line TODO
ESCAPE control code TP_CTRL_ESCAPE #define only. Unnecessary by construction — controls occupy codes 0–5 and the alphabet starts at 6, so no collision can occur.
Addressing modes tp_addr_mode (4 values) 2 occur, both hardcoded
Checksums tp_checksum_type (3 values) CRC-32 only
Huffman / multi-width symbols txz-spec.md §3.3.1 Fixed width only
array, dict value types tp_value_type Return TP_ERR_INVALID_PARAM
has_nested_dicts, compact_mode header flags Never set, never read
Fuzzy search tp_dict_find_fuzzy Returns TP_ERR_UNSUPPORTED
Two-trie architecture (keys + values) txz-spec.md §1.1 One trie plus a flat value store

As of 2.0.0 the encoder refuses options naming these rather than accepting and ignoring them. The enum values and struct fields remain.

4. terseml is not part of this

terseml/ is a separate subproject in the same repository. It links nothing from TriePack, TriePack links nothing from it, it is not published anywhere, and nothing on this page describes it. It is a tree serialisation; TriePack is a dictionary format. They share a repository and nothing else.

5. Summary

What you have is a bit-packed fixed-width prefix trie with skip pointers, typed values, CRC-32, lexicographic iteration, prefix search, and an allocation-free lookup path — implemented ten times, byte-identically, enforced on every build.

What you do not have, and were meant to: suffix sharing, binary-safe keys, and a value path that is not O(n).