unsloth/MiniMax-M2.1-GGUF
Text Generation
•
229B
•
Updated
•
51.8k
•
128
Milan Kryl
mikr
AI & ML interests
None yet
Recent Activity
liked
a model
about 7 hours ago
unsloth/MiniMax-M2.1-GGUF
liked
a model
about 7 hours ago
0xSero/GLM-4.7-REAP-50-W4A16
liked
a model
about 22 hours ago
0xSero/MiniMax-M2.1-REAP-50-W4A16
Organizations
liked
2
models
about 7 hours ago
liked
a
model
about 22 hours ago
upvoted
an
article
2 days ago
Article
Reachy Mini - The Open-Source Robot for Today's and Tomorrow's AI Builders
•
750
upvoted
a
paper
10 days ago
upvoted
an
article
about 1 month ago
Article
We Got Claude to Fine-Tune an Open Source LLM
•
558
upvoted
a
paper
about 1 month ago
liked
a
model
about 1 month ago
upvoted
a
collection
about 1 month ago
liked
2
models
about 2 months ago
upvoted
a
paper
about 2 months ago
liked
2
models
2 months ago
liked
a
model
3 months ago
upvoted
a
collection
3 months ago
upvoted
an
article
3 months ago
Article
mem-agent: Persistent, Human Readable Memory Agent Trained with Online RL
•
25
reacted to
tomaarsen's
post with ❤️
3 months ago
Post
5734
ModernBERT goes MULTILINGUAL! One of the most requested models I've seen, The Johns Hopkins University's CLSP has trained state-of-the-art massively multilingual encoders using the ModernBERT architecture: mmBERT.
Model details:
- 2 model sizes:
- jhu-clsp/mmBERT-small
- jhu-clsp/mmBERT-base
- Uses the ModernBERT architecture, but with the Gemma2 multilingual tokenizer (so: flash attention, alternating global/local attention, unpadding/sequence packing, etc.)
- Maximum sequence length of 8192 tokens, on the high end for encoders
- Trained on 1833 languages using DCLM, FineWeb2, and many more sources
- 3 training phases: 2.3T tokens pretraining on 60 languages, 600B tokens mid-training on 110 languages, and 100B tokens decay training on all 1833 languages.
- Both models are MIT Licensed, and the full datasets and intermediary checkpoints are also publicly released
Evaluation details:
- Very competitive with ModernBERT at equivalent sizes on English (GLUE, MTEB v2 English after finetuning)
- Consistently outperforms equivalently sized models on all Multilingual tasks (XTREME, classification, MTEB v2 Multilingual after finetuning)
- In short: beats commonly used multilingual base models like mDistilBERT, XLM-R (multilingual RoBERTa), multilingual MiniLM, etc.
- Additionally: the ModernBERT-based mmBERT is much faster than the alternatives due to its architectural benefits. Easily up to 2x throughput in common scenarios.
Check out the full blogpost with more details. It's super dense & gets straight to the point: https://huggingface.co/blog/mmbert
Based on these results, mmBERT should be the new go-to multilingual encoder base models at 300M and below. Do note that the mmBERT models are "base" models, i.e. they're currently only trained to perform Mask Filling. They'll need to be finetuned for downstream tasks like semantic search, classification, clustering, etc.
Model details:
- 2 model sizes:
- jhu-clsp/mmBERT-small
- jhu-clsp/mmBERT-base
- Uses the ModernBERT architecture, but with the Gemma2 multilingual tokenizer (so: flash attention, alternating global/local attention, unpadding/sequence packing, etc.)
- Maximum sequence length of 8192 tokens, on the high end for encoders
- Trained on 1833 languages using DCLM, FineWeb2, and many more sources
- 3 training phases: 2.3T tokens pretraining on 60 languages, 600B tokens mid-training on 110 languages, and 100B tokens decay training on all 1833 languages.
- Both models are MIT Licensed, and the full datasets and intermediary checkpoints are also publicly released
Evaluation details:
- Very competitive with ModernBERT at equivalent sizes on English (GLUE, MTEB v2 English after finetuning)
- Consistently outperforms equivalently sized models on all Multilingual tasks (XTREME, classification, MTEB v2 Multilingual after finetuning)
- In short: beats commonly used multilingual base models like mDistilBERT, XLM-R (multilingual RoBERTa), multilingual MiniLM, etc.
- Additionally: the ModernBERT-based mmBERT is much faster than the alternatives due to its architectural benefits. Easily up to 2x throughput in common scenarios.
Check out the full blogpost with more details. It's super dense & gets straight to the point: https://huggingface.co/blog/mmbert
Based on these results, mmBERT should be the new go-to multilingual encoder base models at 300M and below. Do note that the mmBERT models are "base" models, i.e. they're currently only trained to perform Mask Filling. They'll need to be finetuned for downstream tasks like semantic search, classification, clustering, etc.
upvoted
an
article
3 months ago
Article
RexBERT: Encoders for a brave new world of E-Commerce
•
50
upvoted
a
collection
3 months ago