mirror of
https://gitlab.com/the-no-frauds-club/cosmonarchy-bw-prerelease.git
synced 2026-09-17 01:01:12 +00:00
36cc26b464
Ported from cosmonarchy-bw-release fe42f19. The .tbl.txt sources were regenerated from this repo's own .tbl binaries rather than copied, so images.tbl.txt and unitnames.tbl.txt carry the prerelease strings (Askosi Sons of Ash, Faction Selection Askosi, Terran Fleshthief, Forerunner Lumen) instead of the release ones. tools/tbl is committed as real files here; in the release repo it is a bare gitlink with no .gitmodules, so it comes out empty on clone.
168 lines
6.4 KiB
Markdown
168 lines
6.4 KiB
Markdown
# tbl
|
|
|
|
A round-trip compiler between StarCraft: Brood War `.tbl` string tables and a
|
|
plain-text `.tbl.txt` source format. TypeScript, [Effect](https://effect.website) v4, Bun.
|
|
|
|
Every `.tbl` shipped in this repo decompiles and recompiles **byte for byte
|
|
identically** — that is enforced by the test suite.
|
|
|
|
## Usage
|
|
|
|
```sh
|
|
bun install
|
|
|
|
bun run tbl decompile ../../mpq/arr/images.tbl # -> images.tbl.txt
|
|
$EDITOR ../../mpq/arr/images.tbl.txt
|
|
bun run tbl compile ../../mpq/arr/images.tbl.txt # -> images.tbl
|
|
|
|
bun run tbl verify ../../mpq/rez/stat_txt.tbl # prove a round trip is lossless
|
|
bun run tbl info ../../mpq/rez/stat_txt.tbl # entry counts, offset headroom
|
|
```
|
|
|
|
| Command | Flags |
|
|
| --- | --- |
|
|
| `decompile <in.tbl>` | `-o <out>`, `--annotate` (`# [index]` above each entry), `--bare` (no banner), `--dialect` |
|
|
| `compile <in.tbl.txt>` | `-o <out>`, `--dedupe` (share identical entries), `--dialect` |
|
|
| `build [root]` | `--check` (fail instead of writing), `--dedupe` |
|
|
| `verify <in.tbl>` | — |
|
|
| `info <in.tbl>` | — |
|
|
|
|
`build` walks a directory for `.tbl.txt` sources and recompiles each one in place —
|
|
the whole `mpq` tree in one go:
|
|
|
|
```sh
|
|
bun run build # recompile every table under ../../mpq
|
|
bun run check # fail if any committed .tbl is out of date
|
|
```
|
|
|
|
`tools/hooks/pre-commit` runs `check` so a stale `.tbl` cannot be committed beside an
|
|
edited `.tbl.txt`. Install it with
|
|
`ln -sf ../../tools/hooks/pre-commit .git/hooks/pre-commit`.
|
|
|
|
`decompile` defaults to `<input>.txt`, so `images.tbl` becomes `images.tbl.txt`
|
|
and never lands on the unrelated `images.txt` that DatEdit uses for its name
|
|
lists. `compile` strips that `.txt` back off.
|
|
|
|
## The binary format
|
|
|
|
```
|
|
u16le count
|
|
u16le offset[count] // absolute, from the start of the file
|
|
... NUL-terminated string data
|
|
```
|
|
|
|
One index does not always mean one string. `unitnames.tbl` stores
|
|
`name\0subname\0group\0` under a single index, and `stat_txt.tbl` stores a
|
|
hotkey byte followed by button text. Everything from `offset[i]` up to the next
|
|
offset in the file therefore belongs to entry `i`, which is how this tool reads
|
|
them — including bytes past the last offset, which `stat_txt.tbl` relies on.
|
|
|
|
## The text format
|
|
|
|
One entry per line, in index order. `.tbl` strings are raw bytes in a legacy
|
|
code page and are full of Windows paths, so backslash escaping would be both
|
|
lossy and miserable to hand-edit. Angle brackets are the only special
|
|
characters:
|
|
|
|
| Token | Meaning |
|
|
| --- | --- |
|
|
| `<0>` | separates the parts of a multi-part entry |
|
|
| `<N>` | the raw byte `N` (decimal 0-255) — colour codes, `<10>` newline |
|
|
| `<<` | a literal `<` |
|
|
| `<>` | nothing; marks an otherwise empty line |
|
|
|
|
Everything else is literal, so a GRP path stays readable:
|
|
|
|
```
|
|
zerg\avenger.grp
|
|
Terran Phalanx<0>Tank Mode<0>Ground Units
|
|
e<0><4>Tank Mode <1>(<3>E<1>)<10>Returns the Phalanx's weapon and mobility to normal.
|
|
```
|
|
|
|
Lines starting with `#` are comments and blank lines are ignored. An entry that
|
|
would begin with `#` or a space, or end with a space, has that character written
|
|
as `<35>` / `<32>` so no editor can quietly eat it. Output is ASCII by
|
|
construction, so re-saving the file as UTF-8 cannot corrupt it — a stray
|
|
non-ASCII character is a parse error pointing at the line and column.
|
|
|
|
## The GPTP dialect
|
|
|
|
Two tables in this repo are *already* loaded from text by the engine, not from a
|
|
`.tbl` at all: `rez/stat.txt` and `arr/sounds.txt`. CM-GPTP's `load_stat_txt`
|
|
(`hooks/limits/tbl_extender.cpp`) parses them with `collapse_control_codes` and
|
|
builds a **u32-offset** string table in memory, which is how they escape the
|
|
ceiling described below. `--dialect gptp` reads and writes that format.
|
|
|
|
The two dialects agree on what an entry is — bytes, NUL-separated — and differ
|
|
only in escaping:
|
|
|
|
| | `--dialect gptp` | tbl-text v1 |
|
|
| --- | --- | --- |
|
|
| `<N>` | byte N; **wraps** above 255 (`char()` narrowing in the C++) | byte N; over 255 is an error |
|
|
| `<<` | not special — two literal `<` | a literal `<` |
|
|
| `<>` | not special — two literal chars | nothing (empty-entry marker) |
|
|
| `\n`, `\\` | newline, one backslash | no backslash escapes |
|
|
| bytes > 127 | allowed raw | must be written `<N>` |
|
|
| `#` line | **an entry** whose text is `#…` | a comment |
|
|
| blank line | **an entry** | ignored |
|
|
|
|
That last pair is the dangerous one: the engine turns *every* line into an entry,
|
|
so a comment inserted into `stat.txt` would become a string and shift every index
|
|
after it. `--dialect gptp` therefore refuses `--annotate` and writes no banner.
|
|
|
|
Writing is unambiguous rather than byte-faithful to any hand-written file: `<` is
|
|
written `<60>` (this dialect has no `<<`, so a literal `<` before digits would be
|
|
read back as a control code), `\` is written `\\` only when the next byte would
|
|
make it an escape, and bytes above 127 are written `<N>` so output stays ASCII.
|
|
Round trips are checked against both shipped files in `test/dialect.test.ts`.
|
|
|
|
## The 64 KiB ceiling
|
|
|
|
Offsets are `u16`, so no string may start past byte 65535. `stat_txt.tbl` is
|
|
currently **26 bytes** away from that limit:
|
|
|
|
```
|
|
$ bun run tbl info ../../mpq/rez/stat_txt.tbl
|
|
entries 1813
|
|
size 65590 bytes (3628 header + 61962 strings)
|
|
highest offset 65509 of 65535 (26 bytes of headroom)
|
|
```
|
|
|
|
`compile --dedupe` points identical entries at one copy of the string data,
|
|
which buys room back without changing a single string:
|
|
|
|
```
|
|
entries 1813 (904 sharing string data)
|
|
size 64325 bytes (3628 header + 60697 strings)
|
|
highest offset 64244 of 65535 (1291 bytes of headroom)
|
|
```
|
|
|
|
It is off by default so that a plain recompile reproduces the original bytes.
|
|
Overflowing the ceiling is a hard error that names the offending entry and
|
|
suggests the flag.
|
|
|
|
## Library
|
|
|
|
```ts
|
|
import { Effect } from "effect"
|
|
import { Tbl, TblFiles, TblText } from "./src/index.ts"
|
|
|
|
const program = Effect.gen(function*() {
|
|
const files = yield* TblFiles
|
|
const table = yield* files.readTbl("images.tbl") // ReadonlyArray<ReadonlyArray<Uint8Array>>
|
|
yield* files.writeText("images.tbl.txt", table)
|
|
})
|
|
```
|
|
|
|
`Tbl.decode` / `Tbl.encode` and `TblText.decode` / `TblText.encode` are pure and
|
|
need no services. Failures are tagged errors — `TblBinaryError`,
|
|
`TblTextError`, `TblEncodeError` — so `Effect.catchTag` can pick them apart.
|
|
|
|
## Development
|
|
|
|
```sh
|
|
bun test # 76 tests: byte-exact round trips over all 8 shipped .tbl files,
|
|
# plus the GPTP dialect against rez/stat.txt and arr/sounds.txt
|
|
bun run typecheck
|
|
```
|