OpenLLM-France
/

luciole-ablation-1B-fr1.0

Text Generation

Model card Files Files and versions

charlotte9901 commited on Sep 9

Commit

78d5d95

·

verified ·

1 Parent(s): f04aebb

Update Readme.md

Files changed (1) hide show

README.md +1 -2

README.md CHANGED Viewed

@@ -164,8 +164,7 @@ Each model is trained on 100 billion tokens using the following hyperparameters:
 | Maximum Learning rate  | 3e-4 |
 | Final Learning rate    | 3e-5 |
-*For [OpenLLM-France/luciole-ablation-1B-en0.33-fr0.66](https://huggingface.co/OpenLLM-France/luciole-ablation-1B-en0.33-fr0.66), we added a warmup of 500 steps.
 ### Compute Infrastructure
 Each model was trained on 64 H100 GPUs

 | Maximum Learning rate  | 3e-4 |
 | Final Learning rate    | 3e-5 |
+*For [OpenLLM-France/luciole-ablation-1B-en0.33-fr0.66](https://huggingface.co/OpenLLM-France/luciole-ablation-1B-en0.33-fr0.66) and [OpenLLM-France/luciole-ablation-1B-en0.80-fr0.20](https://huggingface.co/OpenLLM-France/luciole-ablation-1B-en0.80-fr0.20), we added a warmup of 500 steps.
 ### Compute Infrastructure
 Each model was trained on 64 H100 GPUs