nanoGPT-Seis
A 113M-parameter GPT pretrained from scratch on earthquake science and general text, with a crawler, MinHash deduplication, a BPE tokenizer, a GQA and RoPE decoder, two-GPU DDP training, and a scaling-law sweep.
- Parameters
- 113M
- Training tokens
- 822.7M
- Hardware
- 2× A30, 6.5 h
- Optimal size
- N ∝ C0.69











