hf-carbon/generator2-opengenome2-eukaryote-20B-hfjson-ddp
Updated
β’
15
AI & ML interests
None defined yet.
Recent Activity
View all activity
kashifΒ
updated
a
model
7 days ago
kashifΒ
published
a
model
7 days ago
lvwerraΒ
authored
a
paper
2 months ago
lvwerraΒ
authored
a
paper
6 months ago
guipenedoΒ
authored
a
paper
6 months ago
loubnabnlΒ
authored
a
paper
7 months ago
guipenedoΒ
authored
a
paper
7 months ago
Post
6013
SmolVLM is now available on PocketPal β you can run it offline on your smartphone to interpret the world around you. ππ±
And check out this real-time camera demo by @ngxson , powered by llama.cpp:
https://github.com/ngxson/smolvlm-realtime-webcam
https://x.com/pocketpal_ai
And check out this real-time camera demo by @ngxson , powered by llama.cpp:
https://github.com/ngxson/smolvlm-realtime-webcam
https://x.com/pocketpal_ai
-
- 4 replies
anton-lΒ
authored
a
paper
9 months ago
lvwerraΒ
authored
a
paper
9 months ago
loubnabnlΒ
authored
a
paper
9 months ago
loubnabnlΒ
authored
a
paper
11 months ago
guipenedoΒ
authored
a
paper
11 months ago
anton-lΒ
authored
a
paper
11 months ago
lvwerraΒ
authored
2
papers
11 months ago
guipenedoΒ
authored
a
paper
11 months ago
Post
3633
Introducing ππ
π’π§πππππ‘: the best public math pre-training dataset with 50B+ tokens!
HuggingFaceTB/finemath
Math remains challenging for LLMs and by training on FineMath we see considerable gains over other math datasets, especially on GSM8K and MATH.
We build the dataset by:
π οΈ carefully extracting math data from Common Crawl;
π iteratively filtering and recalling high quality math pages using a classifier trained on synthetic annotations to identify math reasoning and deduction.
We conducted a series of ablations comparing the performance of Llama-3.2-3B-Base after continued pre-training on FineMath and observe notable gains compared to the baseline model and other public math datasets.
We hope this helps advance the performance of LLMs on math and reasoning! π
Weβre also releasing all the ablation models as well as the evaluation code.
HuggingFaceTB/finemath-6763fb8f71b6439b653482c2
HuggingFaceTB/finemath
Math remains challenging for LLMs and by training on FineMath we see considerable gains over other math datasets, especially on GSM8K and MATH.
We build the dataset by:
π οΈ carefully extracting math data from Common Crawl;
π iteratively filtering and recalling high quality math pages using a classifier trained on synthetic annotations to identify math reasoning and deduction.
We conducted a series of ablations comparing the performance of Llama-3.2-3B-Base after continued pre-training on FineMath and observe notable gains compared to the baseline model and other public math datasets.
We hope this helps advance the performance of LLMs on math and reasoning! π
Weβre also releasing all the ablation models as well as the evaluation code.
HuggingFaceTB/finemath-6763fb8f71b6439b653482c2
Post
3743
Making SmolLM2 reproducible: open-sourcing our training & evaluation toolkit π οΈ https://github.com/huggingface/smollm/
- Pre-training code with nanotron
- Evaluation suite with lighteval
- Synthetic data generation using distilabel (powers our new SFT dataset HuggingFaceTB/smoltalk)
- Post-training scripts with TRL & the alignment handbook
- On-device tools with llama.cpp for summarization, rewriting & agents
Apache 2.0 licensed. V2 pre-training data mix coming soon!
Which other tools should we add next?
- Pre-training code with nanotron
- Evaluation suite with lighteval
- Synthetic data generation using distilabel (powers our new SFT dataset HuggingFaceTB/smoltalk)
- Post-training scripts with TRL & the alignment handbook
- On-device tools with llama.cpp for summarization, rewriting & agents
Apache 2.0 licensed. V2 pre-training data mix coming soon!
Which other tools should we add next?
lvwerraΒ
authored
a
paper
about 1 year ago