Taco
Open source · Rust · Alpha
A compressor that runs GPT-2 on integers, so every machine gets the same bits.
The model predicts the next token and a range coder stores only what it did not predict. Its engine, HardShell, uses integer arithmetic only, so a file compressed on a Mac comes back bit for bit on Linux, on a CPU or a GPU.
Rust · CUDA · wgpu · GPT-2
- Taco8,432
- xz -9e12,400
- zstd -1912,463
- 32%
- smaller than xz on English text
- 10.8%
- smaller than xz on server logs
- 12/12
- identical files, Mac ARM and Linux x86