FlashLM v4 "Bolt" — Ternary Language Model Demo
4.3M parameters | Ternary weights (-1, 0, +1) | Trained on CPU in 2 hours | No GPU used
This model uses only additions and subtractions for inference. Trained on TinyStories dataset on a free-tier 2-thread CPU (Deepnote).
| Metric | Value |
|---|---|
| Parameters | 4.3M (ternary) |
| BPC | 0.8798 |
| Training | 2h on 2-thread CPU |
| Dataset | TinyStories (10.6M tokens) |
| Val Loss | 2.0976 |
50 500
0.1 2
0 100
Note: Some words may be missing due to the 10K vocabulary limitation. The model was trained on children's stories and works best with story-like prompts.
Model Card | v3 Demo | GitHub
Examples