Small model pretraining experiments - a amang1802 Collection

amang1802 's Collections

ThinkTransformer experiments

Small model pretraining experiments

PPO experiments

Synthetic Data rewrite (model checkpoints)

Synthetic Data rewrite research (training and eval datasets)

WildeWeb Research

Small model pretraining experiments

updated Feb 9, 2025

amang1802/llama_162M_fineweb100BT

Text Generation • 0.2B • Updated Aug 27, 2025 • 1
amang1802/llama_162M_fineweb10BT

Text Generation • 0.2B • Updated Dec 22, 2024